feat: fiscal_series, fiscal_certificates, fiscal_documents models + migrations
Ports the three tables that change owner per the design spec (decision #2): FiscalSeries (was tenants.FiscalDocumentSeries), FiscalCertificate, FiscalDocument (both from fiscal.models) — organization_id/branch_id replaced by product_id (FK products) + tenant_ref/branch_ref (opaque strings) per the porte table. sale_id/service_order_id dropped (no Sale concept here). document_model is now a plain String(2) + Python enum, never a Postgres enum (Global Constraints: zero enum PG — the auto's own version of this column was a real PG enum, a documented debt not repeated here). Constraints preserved: UNIQUE chave_acesso, partial-unique cert-vivo-per-(product_id, branch_ref) (product-scoped in addition to the auto's branch_id, since branch_ref is an opaque string two different products could coincidentally share), UNIQUE (product_id, tenant_ref, branch_ref, document_model, serie) on the series. documents.service.allocate_fiscal_number ported verbatim (mechanism + contract): SELECT ... FOR UPDATE + populate_existing=True, no-commit contract (caller commits together with the FiscalDocument insert, Task 5). The next_number regression guard is deliberately deferred to Task 4's PATCH /v1/series endpoint (needs FiscalDocument, which now exists). AST guard (tests/shared/test_for_update_populate_existing.py) ported and adapted to scan src/fiscal_svc/, plus two new self-tests proving the detection logic itself in both directions (flags a missing fix, does not false-positive on a correctly fixed multi-line chain) — the ported guard alone only proves "currently green", not "actually detects". 33 tests green via `make k8s-test` (real-migration round trips + unique constraint violations, N=10 concurrency, identity-map staleness repro, tenancy-scoping not-found across product/tenant_ref/branch_ref).
This commit is contained in:
jonatanritter
@@ -0,0 +1,217 @@
|
||||
"""Task 3: the three fiscal tables that "change owner" per the design spec
|
||||
(decision #2) -- they used to live in the auto (`app/modules/tenants/
|
||||
models.py`'s `FiscalDocumentSeries`, `app/modules/fiscal/models.py`'s
|
||||
`FiscalCertificate`/`FiscalDocument`) and now live HERE, as the source of
|
||||
truth this service was built to be. Ported with the porte table
|
||||
(2026-07-17-fiscal-svc-f2-servico.md) applied throughout:
|
||||
|
||||
| No auto | No serviço |
|
||||
|-----------------------------------|-----------------------------------|
|
||||
| organization_id (FK organizations)| product_id (FK products) + |
|
||||
| | tenant_ref: String(64) |
|
||||
| branch_id (FK branches) | branch_ref: String(64) + cnpj |
|
||||
| | (the strong link) |
|
||||
| Sale/rotas por venda | não existem (sale_id/ |
|
||||
| | service_order_id DROPPED -- |
|
||||
| | emissão recebe DadosEmissao |
|
||||
| | completo, sem correlação a uma |
|
||||
| | linha de venda deste lado) |
|
||||
|
||||
Global Constraints (zero enum PG): unlike the auto's `FiscalDocumentSeries.
|
||||
document_model` (a REAL Postgres `Enum` -- a documented, deliberately
|
||||
un-repeated debt there), `document_model` here is plain `String(2)` +
|
||||
Python enum at the Pydantic border ONLY, same convention the auto's own
|
||||
`FiscalDocument.document_model`/`.status` already used (see their
|
||||
docstrings) -- this port does not re-introduce the PG enum anywhere."""
|
||||
import enum
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import (
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
LargeBinary,
|
||||
String,
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from fiscal_svc.core.db import Base
|
||||
from fiscal_svc.shared.base_model import SoftDeleteMixin, TimestampMixin, UUIDPKMixin
|
||||
|
||||
|
||||
class FiscalDocumentModel(str, enum.Enum):
|
||||
"""Ported from `auto/backend/app/modules/tenants/models.py`. Python
|
||||
enum only -- see this module's docstring for why the auto's Postgres
|
||||
`Enum` column for this same enum is NOT repeated here."""
|
||||
|
||||
NFE_55 = "55"
|
||||
NFCE_65 = "65"
|
||||
|
||||
|
||||
class FiscalDocumentStatus(str, enum.Enum):
|
||||
"""Ported verbatim from `auto/backend/app/modules/fiscal/models.py` --
|
||||
already `String` + Python enum there (ZERO Postgres enum), so no
|
||||
adaptation needed. State machine: `ASSINADO -> TRANSMITINDO ->
|
||||
AUTORIZADA | REJEITADA | PENDENTE_CONSULTA` (+ `CANCELADA`/`DENEGADA`).
|
||||
Task 3 (this port) only ever WRITES `ASSINADO` -- transmission (F4)
|
||||
writes the rest; the full vocabulary exists now so a later phase never
|
||||
needs a column-shape migration for a new status."""
|
||||
|
||||
ASSINADO = "ASSINADO"
|
||||
TRANSMITINDO = "TRANSMITINDO"
|
||||
AUTORIZADA = "AUTORIZADA"
|
||||
REJEITADA = "REJEITADA"
|
||||
PENDENTE_CONSULTA = "PENDENTE_CONSULTA"
|
||||
CANCELADA = "CANCELADA"
|
||||
DENEGADA = "DENEGADA"
|
||||
|
||||
|
||||
class FiscalSeries(Base, UUIDPKMixin, TimestampMixin, SoftDeleteMixin):
|
||||
"""Ported from the auto's `FiscalDocumentSeries` (`tenants/models.py`) --
|
||||
per the design spec this table is the NEW owner of fiscal numbering
|
||||
(decision #2: "mudam de dono"). `product_id`/`tenant_ref`/`branch_ref`
|
||||
replace `organization_id`/`branch_id` per the porte table (this
|
||||
module's docstring).
|
||||
|
||||
Atomic allocation (`SELECT ... FOR UPDATE` + `populate_existing`) is
|
||||
`documents.service.allocate_fiscal_number` (Task 3, ported with its
|
||||
exact no-commit contract -- see that function's docstring). The
|
||||
"guard retroativo" (`next_number` can never regress below the highest
|
||||
`numero` this series has already emitted, per `FiscalDocument.numero`)
|
||||
is deliberately NOT implemented in this task -- it belongs to the
|
||||
`PATCH /v1/series` endpoint (Task 4), which needs this exact model
|
||||
plus `FiscalDocument` (both now exist, Task 3) to check against. Same
|
||||
"documented now, wired later" shape as the auto's own
|
||||
`FiscalDocumentSeries` docstring ("a emissão em si nasce no módulo
|
||||
fiscal da 1b")."""
|
||||
|
||||
__tablename__ = "fiscal_series"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"product_id", "tenant_ref", "branch_ref", "document_model", "serie",
|
||||
name="uq_fiscal_series_product_tenant_branch_model_serie",
|
||||
),
|
||||
)
|
||||
|
||||
product_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("products.id"), nullable=False)
|
||||
tenant_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
branch_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
document_model: Mapped[str] = mapped_column(String(2), nullable=False)
|
||||
serie: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
next_number: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
|
||||
|
||||
class FiscalCertificate(Base, UUIDPKMixin, TimestampMixin, SoftDeleteMixin):
|
||||
"""Ported from the auto's `FiscalCertificate` (`app/modules/fiscal/
|
||||
models.py`) -- A1 certificate (.pfx) for a `branch_ref`. Per the porte
|
||||
table, the emitter is `(product_id, branch_ref)`, with `cnpj` carried
|
||||
on the row itself (the "vínculo forte", design spec decision #4: the
|
||||
CNPJ is what's validated against the certificate at upload and against
|
||||
the emitente at emission -- `branch_ref` alone is an opaque string this
|
||||
service never interprets). No fallback to a tenant-level or
|
||||
product-level certificate -- fail-closed by construction, same as the
|
||||
auto: a branch_ref without its own live certificate cannot emit.
|
||||
|
||||
One VIVO (`deleted_at IS NULL`) row per `(product_id, branch_ref)`: a
|
||||
second upload soft-deletes the previous live row (Task 4's
|
||||
`certificates.service`). `ix_fiscal_certificates_product_branch_live`
|
||||
is a PARTIAL UNIQUE index enforcing that at the database level (same
|
||||
"two concurrent uploads must not both land a live row" reasoning as the
|
||||
auto's own migration `a1b2c3d4e5f6`'s fix) -- scoped by `product_id` in
|
||||
ADDITION to `branch_ref` (the auto's version only needed `branch_id`,
|
||||
already product-scoped by being a real FK; here `branch_ref` is an
|
||||
OPAQUE string owned by the calling product, so two DIFFERENT products
|
||||
could coincidentally pick the identical string for two DIFFERENT real
|
||||
branches -- scoping the uniqueness by `product_id` too is what keeps
|
||||
that from cross-contaminating one product's certificate slot with
|
||||
another's).
|
||||
|
||||
`pfx_encrypted`/`password_encrypted` are Fernet ciphertext (Task 4,
|
||||
`FISCAL_CERT_ENCRYPTION_KEY` env var, own key -- never
|
||||
`app.core.config.Settings`, same "secrets never live in Settings"
|
||||
convention as the auto)."""
|
||||
|
||||
__tablename__ = "fiscal_certificates"
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_fiscal_certificates_product_branch_live",
|
||||
"product_id",
|
||||
"branch_ref",
|
||||
unique=True,
|
||||
postgresql_where=text("deleted_at IS NULL"),
|
||||
),
|
||||
)
|
||||
|
||||
product_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("products.id"), nullable=False)
|
||||
tenant_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
branch_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
cnpj: Mapped[str] = mapped_column(String(14), nullable=False)
|
||||
pfx_encrypted: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
|
||||
password_encrypted: Mapped[bytes] = mapped_column(LargeBinary, nullable=False)
|
||||
subject_cn: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
cnpj_certificado: Mapped[str] = mapped_column(String(14), nullable=False)
|
||||
not_valid_before: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
not_valid_after: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
|
||||
class FiscalDocument(Base, UUIDPKMixin, TimestampMixin, SoftDeleteMixin):
|
||||
"""Ported from the auto's `FiscalDocument` (`app/modules/fiscal/
|
||||
models.py`). Per the porte table, `sale_id`/`service_order_id` are
|
||||
DROPPED -- this service has no notion of a Sale/ServiceOrder (decision
|
||||
#3 of the design spec: it receives a complete `DadosEmissao`, never
|
||||
resolves or references the caller's own domain objects). A product's
|
||||
own correlation to ITS sale/OS is the product's problem, carried via
|
||||
the `Idempotency-Key` (Task 5) it sends, not a column here.
|
||||
|
||||
Invariants preserved from the auto (contract, not mere convention):
|
||||
(a) a row is only inserted in the SAME commit that allocates `numero`
|
||||
via `documents.service.allocate_fiscal_number` (which does not
|
||||
commit by itself -- see its docstring) -- outbox transactional,
|
||||
exactly as ported;
|
||||
(b) documento é IMUTÁVEL após `ASSINADO` -- correção é cancelar (evento
|
||||
futuro) e emitir outro, nunca um UPDATE no XML/chave já assinados;
|
||||
(c) `chave_acesso` is UNIQUE GLOBALLY (not just per product) -- it
|
||||
already embeds the emitente's CNPJ in its own digits, so two
|
||||
DIFFERENT products (or tenants) can never legitimately produce the
|
||||
same one; a collision here can only be a generation bug, never a
|
||||
false positive between tenants OR between products.
|
||||
|
||||
`ix_fiscal_documents_product_tenant_branch_status` replaces the auto's
|
||||
`ix_fiscal_documents_org_branch_status` (porte table: anti-oracle 404
|
||||
is now by `(product_id, tenant_ref)`, and `GET /v1/documentos` filters
|
||||
by `tenant_ref`/`branch_ref`/`status` per the design spec's API
|
||||
table) -- the index covers exactly that filter shape."""
|
||||
|
||||
__tablename__ = "fiscal_documents"
|
||||
__table_args__ = (
|
||||
Index(
|
||||
"ix_fiscal_documents_product_tenant_branch_status",
|
||||
"product_id",
|
||||
"tenant_ref",
|
||||
"branch_ref",
|
||||
"status",
|
||||
),
|
||||
UniqueConstraint("chave_acesso", name="uq_fiscal_documents_chave_acesso"),
|
||||
)
|
||||
|
||||
product_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("products.id"), nullable=False)
|
||||
tenant_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
branch_ref: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
series_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("fiscal_series.id"), nullable=False)
|
||||
document_model: Mapped[str] = mapped_column(String(2), nullable=False)
|
||||
serie: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
numero: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
chave_acesso: Mapped[str] = mapped_column(String(44), nullable=False)
|
||||
codigo_numerico: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||
status: Mapped[str] = mapped_column(String(20), nullable=False)
|
||||
ambiente: Mapped[str] = mapped_column(String(12), nullable=False)
|
||||
xml_assinado: Mapped[str] = mapped_column(Text, nullable=False)
|
||||
rejeicao_codigo: Mapped[str | None] = mapped_column(String(10), nullable=True)
|
||||
rejeicao_motivo: Mapped[str | None] = mapped_column(String(500), nullable=True)
|
||||
protocolo: Mapped[str | None] = mapped_column(String(20), nullable=True)
|
||||
autorizada_em: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
@@ -0,0 +1,124 @@
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from fiscal_svc.documents.models import FiscalDocumentModel, FiscalSeries
|
||||
|
||||
|
||||
class FiscalSeriesNotFoundError(Exception):
|
||||
"""Ported from the auto's `tenants.service.FiscalSeriesNotFoundError`.
|
||||
Raised when a `(product_id, tenant_ref, branch_ref, document_model,
|
||||
serie)` tuple does not resolve to a live row -- deliberately raised
|
||||
both when the series truly does not exist and when it belongs to
|
||||
another product/tenant/branch, so a cross-tenant lookup can't be used
|
||||
to distinguish "not found" from "not yours" (anti-oracle, porte table:
|
||||
"por org" -> "por (product_id, tenant_ref)")."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
product_id: uuid.UUID,
|
||||
tenant_ref: str,
|
||||
branch_ref: str,
|
||||
document_model: FiscalDocumentModel | str,
|
||||
serie: int,
|
||||
):
|
||||
self.product_id = product_id
|
||||
self.tenant_ref = tenant_ref
|
||||
self.branch_ref = branch_ref
|
||||
self.document_model = document_model
|
||||
self.serie = serie
|
||||
super().__init__(
|
||||
f"Série fiscal não encontrada para product_id={product_id}, "
|
||||
f"tenant_ref={tenant_ref}, branch_ref={branch_ref}, "
|
||||
f"modelo={document_model}, série={serie}"
|
||||
)
|
||||
|
||||
|
||||
async def allocate_fiscal_number(
|
||||
session: AsyncSession,
|
||||
product_id: uuid.UUID,
|
||||
tenant_ref: str,
|
||||
branch_ref: str,
|
||||
document_model: FiscalDocumentModel | str,
|
||||
serie: int,
|
||||
) -> int:
|
||||
"""Ported verbatim (mechanism + contract) from the auto's
|
||||
`tenants.service.allocate_fiscal_number`, with the porte table's
|
||||
tenancy adaptation applied to the lookup (`organization_id`/`branch_id`
|
||||
-> `product_id`/`tenant_ref`/`branch_ref`). Every invariant from the
|
||||
original is preserved:
|
||||
|
||||
`SELECT ... FOR UPDATE` locks the SERIES ROW: two concurrent emissions
|
||||
for the same `(product_id, tenant_ref, branch_ref, document_model,
|
||||
serie)` never receive the same number -- the second waits for the
|
||||
first's lock and reads the already-incremented `next_number`.
|
||||
|
||||
`.execution_options(populate_existing=True)` is OBRIGATÓRIO: the lock
|
||||
happens in Postgres, but if this `FiscalSeries` object is already in
|
||||
THIS session's identity map (e.g. an earlier unlocked read of the same
|
||||
row, same session), SQLAlchemy's default behavior hands back the
|
||||
CACHED object without repopulating it from the freshly-(re)locked row
|
||||
-- even though the `FOR UPDATE` really did lock the real row. Without
|
||||
this, `next_number` can stay pinned to a stale value and two callers
|
||||
can allocate the SAME number. `fiscal_svc.core.db`'s
|
||||
`expire_on_commit=False` makes this worse (the object never expires on
|
||||
its own after a commit either). See
|
||||
`auto/backend/app/modules/tenants/service.py::allocate_fiscal_number`'s
|
||||
docstring and `tests/modules/tenants/test_fiscal_series.py::
|
||||
test_allocate_fiscal_number_repopulates_identity_mapped_object` for the
|
||||
original reproduction this ports the fix (and the test) for.
|
||||
|
||||
CONTRATO -- ESTA FUNÇÃO NÃO COMMITA. Only `flush()`s: the number is
|
||||
allocated in memory (and the row stays LOCKED by this transaction's
|
||||
`FOR UPDATE`) but nothing is durable until the CALLER commits. This is
|
||||
deliberate: the real emission flow (Task 5) is "alocar número -> montar
|
||||
o XML -> assinar -> persistir o `FiscalDocument` (com esse número)",
|
||||
and allocation + the document INSERT must commit TOGETHER, atomically,
|
||||
in the SAME transaction -- otherwise a failure between allocating and
|
||||
inserting the document would burn a number with no record of what it
|
||||
was for (and, worse, no `FiscalDocument` row to even know it needs
|
||||
inutilização at SEFAZ later). A caller that never commits (a test that
|
||||
only calls this and rolls back, for instance) leaves the allocation
|
||||
with zero durable effect -- expected, not a bug.
|
||||
|
||||
Not an endpoint -- an INTERNAL function the emission service (Task 5)
|
||||
calls. Each caller must pass its OWN `AsyncSession` (one transaction per
|
||||
call): it's one session's `FOR UPDATE` blocking another session's
|
||||
`SELECT` that actually serializes two concurrent allocations; two calls
|
||||
sharing one session/transaction would have no real DB concurrency to
|
||||
serialize."""
|
||||
result = await session.execute(
|
||||
select(FiscalSeries)
|
||||
.where(
|
||||
FiscalSeries.product_id == product_id,
|
||||
FiscalSeries.tenant_ref == tenant_ref,
|
||||
FiscalSeries.branch_ref == branch_ref,
|
||||
FiscalSeries.document_model == (
|
||||
document_model.value
|
||||
if isinstance(document_model, FiscalDocumentModel)
|
||||
else document_model
|
||||
),
|
||||
FiscalSeries.serie == serie,
|
||||
FiscalSeries.deleted_at.is_(None),
|
||||
)
|
||||
.with_for_update()
|
||||
.execution_options(populate_existing=True)
|
||||
)
|
||||
series = result.scalar_one_or_none()
|
||||
if series is None:
|
||||
raise FiscalSeriesNotFoundError(
|
||||
product_id=product_id,
|
||||
tenant_ref=tenant_ref,
|
||||
branch_ref=branch_ref,
|
||||
document_model=document_model,
|
||||
serie=serie,
|
||||
)
|
||||
allocated = series.next_number
|
||||
series.next_number = allocated + 1
|
||||
# NÃO commita -- ver a seção "CONTRATO" do docstring. O caller commita,
|
||||
# idealmente junto com o INSERT do FiscalDocument que consome este
|
||||
# número. `flush()` garante que o UPDATE já foi enviado ao Postgres (o
|
||||
# lock do FOR UPDATE segue travado até o commit do caller).
|
||||
await session.flush()
|
||||
return allocated
|
||||
Reference in New Issue
Block a user