fix: harden week-two ordering, artwork and operations
This commit is contained in:
215
docs/ROADMAP.md
215
docs/ROADMAP.md
@@ -7,18 +7,37 @@
|
||||
> Update the **Current step** line and the item status every time something moves.
|
||||
> Add new findings at the bottom of the relevant block rather than rewriting history.
|
||||
|
||||
**Current step:** Block 0 closed; Block 2 closed except 2.9–2.11; 4.1, 4.2, 4.5,
|
||||
5.1, 5.3, 5.4 and 5.8 done. Remaining work needs decisions (3.1, 3.2, 3.3) or
|
||||
client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
|
||||
1.1/1.2.
|
||||
**Current step (2026-09-23, Week 2):** Payment safety fixes 2.13 and 2.14 and
|
||||
the local order-correctness work in 3.6/3.9 have passed integration checks.
|
||||
Production specification v2 now records each copy's film coordinates and is
|
||||
kept through the approved order; 4.6 now pages pending and approved unpaid
|
||||
quotes, including a tested 101st pending quote. Operational entrypoints in
|
||||
5.12 are repaired and locally exercised. Image decoding, mixed-sheet grading,
|
||||
rotation-sensitive DPI, and PDF page geometry are corrected in 3.9/4.4.
|
||||
Next address the upload/scanner safety gate and unsupported PDF image evidence.
|
||||
The customer/API upload admission now stops above the scanner's effective limit
|
||||
before transfer; the 5 GiB large-file product path still needs agreement and
|
||||
implementation.
|
||||
Unfinished upload reservations now expire after one hour or can be cancelled
|
||||
explicitly; anonymous admission and browser resource bounds stay open.
|
||||
Generated print output, lifecycle/recovery, and provider work remain open.
|
||||
Obtain decisions for unattended pricing, print-file acceptance and large files,
|
||||
plus sandbox inputs for freight and Mercado Pago. Week 2 delivery items 1.1–1.5
|
||||
remain open; 1.6 is complete.
|
||||
|
||||
> Paths in closed items are written as they were when the finding was made.
|
||||
> The repository was laid out by role on 2026-09-21 (`local/` became `app/`,
|
||||
> with `tests/`, `ops/`, `infra/` and `web/` beside it); the history is left
|
||||
> as recorded rather than rewritten.
|
||||
|
||||
**Last audit:** 2026-09-18, full read of `app/`, `dtf-site.html`, `deploy/`,
|
||||
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
|
||||
**Last audit:** 2026-09-18, full read of the then-current tree. The 2026-09-21
|
||||
full review is `docs/REVIEW-2026-09-21.md`; the 2026-09-22 review and payment
|
||||
probes added new findings to Blocks 2–5 below. Historical paths in closed items
|
||||
remain as recorded.
|
||||
|
||||
**Full remediation register:** `docs/REMEDIATION-2026-09-22.md` maps every one
|
||||
of the 37 review findings to an action and a release gate. Use it alongside
|
||||
this Week 2 tracker; a green milestone here does not close the production gate.
|
||||
|
||||
| Status | Meaning |
|
||||
|---|---|
|
||||
@@ -29,6 +48,29 @@ client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
|
||||
|
||||
---
|
||||
|
||||
## Week 2 execution sequence
|
||||
|
||||
This sequence keeps the client commitments in Block 1 visible while correcting
|
||||
defects that would make those commitments unsafe or impossible to operate.
|
||||
Do not mark a provider item complete from a fake-adapter test or a healthy page.
|
||||
|
||||
| Order | Work | Exit evidence |
|
||||
|---|---|---|
|
||||
| 1. Immediate safety — done 2026-09-22 | Close 2.13 and 2.14; cover foreign quote IDs, missing/invalid amounts, duplicates and valid approvals. | Local integration checks pass and no other customer's order is returned. |
|
||||
| 2. Order correctness | Fix 3.6 and 4.6: one current cart/quote snapshot, versioned per-file production instructions, correction/final revision binding, visible actionable quotes. | The approved quote, order and final file can be traced back to the same reviewed layout; edits cannot buy an old cart. |
|
||||
| 3. Resolve product contracts | Decide 3.1–3.3: which quotes may auto-approve, what generates the print file, and which sizes the upload and scanner can release. | Written acceptance rules and representative artwork/large-file cases before enabling unattended payment. |
|
||||
| 4. Week 2 integrations | Add destination data and real freight first, then Mercado Pago payment intents/webhooks/reconciliation, then Tiny/Olist order creation. Keep the four agreed WhatsApp events in the same delivery contract. | Sandbox flows and failure/retry cases pass; no fake provider is presented as production ready. |
|
||||
| 5. Operability and release | Repair 5.12–5.14, signatures, proxy trust and backup/restore; gate browser tests and the exact deployed images. | Fresh install, upgrade, recovery and deployed release checks pass with alert ownership recorded. |
|
||||
|
||||
Client inputs needed for steps 3–4 are listed in `docs/PRODUCTION_INPUTS.md`.
|
||||
Engineering can complete steps 1–2 and repair local operational commands while
|
||||
those inputs are gathered. The full disposition of architecture, security,
|
||||
quality, operational, and delivery findings is in
|
||||
`docs/REMEDIATION-2026-09-22.md`; all release gates there must be met before
|
||||
accepting real customer work.
|
||||
|
||||
---
|
||||
|
||||
## Block 0 · Broken right now
|
||||
|
||||
Nothing in this block is optional. Until it is closed, the system cannot be
|
||||
@@ -158,29 +200,28 @@ Ports 8090/8091/8010 were used; 8080 was held by an unrelated preview server.
|
||||
From the report already sent. These are dated promises, not backlog.
|
||||
|
||||
- `[~]` 1.1 — Mercado Pago transparent checkout, signed and idempotent webhooks.
|
||||
**The provider-independent half is built** (2026-09-22): `POST /api/payments/webhook`
|
||||
verifies a signature before parsing, records every delivery under the provider's
|
||||
own event id, and applies it in one transaction. A duplicate is a no-op, a
|
||||
re-sent approval finds the order already there, and an approval whose amount
|
||||
disagrees with the reviewed quote is refused rather than shipped. Order creation
|
||||
moved to `app/payments.py` so the webhook and the local checkout cannot drift.
|
||||
Exercised end to end by `tests/payment_test.py` against a fake signer.
|
||||
|
||||
What remains needs the client: a sandbox account, webhook administration, the
|
||||
event/status mapping and the refund policy. In code it is one adapter supplying
|
||||
`create`, `verify` and `parse` — nothing in the service changes.
|
||||
**Current foundation (2026-09-22):** a fake signer exercises signature rejection,
|
||||
event-ID deduplication, amount comparison and transactional order creation.
|
||||
This is not a Mercado Pago integration. Complete a durable payment intent,
|
||||
provider payment ID and currency binding, real verification and status lookup,
|
||||
delayed/duplicate event handling, refund/cancellation rules and reconciliation.
|
||||
A refused paid event must be visible for operator resolution rather than silently
|
||||
treated as finished. See 2.14 and 3.7. Requires sandbox access, webhook
|
||||
administration, event mapping and an approved refund policy.
|
||||
- `[ ]` 1.2 — Real freight quotation. **Blocked on client inputs** (see
|
||||
`PRODUCTION_INPUTS.md`): source platform, credentials, origin CEP, services,
|
||||
packaging weight/dimensions per length, subsidy policy.
|
||||
- `[ ]` 1.3 — Idempotent Tiny/Olist order creation with order-number traceability.
|
||||
Confirm endpoints, tag behaviour and rate limits first.
|
||||
- `[ ]` 1.4 — Final print-file generation (see 3.2 — this is the same problem).
|
||||
- `[ ]` 1.4 — Final print-file generation (see 3.2 and 3.6: production instructions
|
||||
must survive checkout before an output engine can reproduce the approved job).
|
||||
- `[ ]` 1.5 — Main Kanban production states consolidated.
|
||||
- `[ ]` 1.6 — **Block 0.2 + 0.3**, promised as "início da próxima semana".
|
||||
- `[x]` 1.6 — **Block 0.2 + 0.3** were completed and verified on 2026-09-18.
|
||||
|
||||
`[!]` The production compose currently blocks `dev_paid` (`ENVIRONMENT != 'local'`)
|
||||
and ships only fake adapters, so the deployed system cannot take an order at all.
|
||||
1.1 is what unblocks it.
|
||||
Real freight, payment initiation and verified provider events are required to
|
||||
unblock it; the fake webhook alone does not.
|
||||
|
||||
---
|
||||
|
||||
@@ -335,8 +376,36 @@ proving control of the e-mail. Needs a transactional mail provider — **client
|
||||
### `[ ]` 2.11 — LGPD `(F15)`
|
||||
|
||||
CNPJ, phone and e-mail are kept indefinitely in `accounts.profile` and
|
||||
`orders.snapshot`. Artwork has a 30-day policy; personal data has none, and there is
|
||||
no privacy notice, consent record or deletion path.
|
||||
`orders.snapshot`. Artwork has a 30-day policy; personal data has no defined
|
||||
retention, export or deletion path. The Site links an external privacy notice;
|
||||
confirm that it covers this processing and define the required records and
|
||||
customer rights flow before production activation.
|
||||
|
||||
### `[x]` 2.13 — A foreign paid quote ID exposes an order (2026-09-22 review)
|
||||
|
||||
The `dev-paid` refusal fallback fetched `orders` by `quote_id` without `owner`.
|
||||
A separate local customer session received the full paid order when supplied
|
||||
another customer's quote ID. `app/api/orders.py` now includes the owner in the
|
||||
fallback query. The local payment integration test confirms a foreign ID returns
|
||||
404 while the owner can still retrieve the already-paid order.
|
||||
|
||||
### `[x]` 2.14 — A signed approval without a paid amount creates an order
|
||||
|
||||
`app/payments.py` compared amounts only when the event contained one. A local
|
||||
signed `approved` event without `amount_cents` created an order. The service now
|
||||
requires an actual integer amount equal to the approved total; local integration
|
||||
tests cover missing, non-integer, underpaid and correct values. Currency and
|
||||
provider payment identity belong to the wider contract in 3.7; this gate does
|
||||
not complete 1.1.
|
||||
|
||||
### `[~]` 2.15 — Public intake controls need operational proof
|
||||
|
||||
Unfinished reservations now have a one-hour lease and owner-scoped cancellation;
|
||||
anonymous admission capacity still needs a firm bound. Keep ClamAV signatures
|
||||
current and alert on stale data; scope reverse-proxy IP trust
|
||||
to the actual hop and verify it through Swarm ingress. These are separate
|
||||
controls, but all must work before public large-file intake is considered safe.
|
||||
The production topology has not been verified by the repository review.
|
||||
|
||||
---
|
||||
|
||||
@@ -372,8 +441,10 @@ what was promised in the meeting and what exists.
|
||||
### `[?]` 3.3 — The 5 GB problem is unsolved `(F19)`
|
||||
|
||||
Transport accepts 5 GiB; `SCAN_MAX_BYTES` / ClamAV `StreamMaxLength` release only
|
||||
≤ 128 MiB. Files above that are quarantined permanently with no path forward. This
|
||||
is exactly the risk Jorge raised in the meeting.
|
||||
≤ 128 MiB. As of 2026-09-23, customer selection and API reservation reject files
|
||||
above the effective scan limit before transfer, and the Site displays the current
|
||||
limit. This prevents a doomed upload; it does not deliver the promised 5 GiB path.
|
||||
This is exactly the risk Jorge raised in the meeting.
|
||||
|
||||
**Decide:** raise the scan ceiling with a resource/timeout design, or define an
|
||||
explicit large-file path (staged scan, sampled scan, operator override with audit).
|
||||
@@ -384,11 +455,69 @@ explicit large-file path (staged scan, sampled scan, operator override with audi
|
||||
round-trip per part → ~640 sequential API calls for a 5 GB file, through an nginx
|
||||
`limit_req` of 20r/s. Add parallelism (4–6 in flight) and batch presigning.
|
||||
|
||||
### `[ ]` 3.5 — Payment ordering `(F27)`
|
||||
### `[~]` 3.5 — Payment ordering `(F27)`
|
||||
|
||||
`dev_paid` charges before persisting the order and passes no idempotency key.
|
||||
Harmless with `FakePayment`; with Mercado Pago that ordering is how you get double
|
||||
charges. Fix as part of 1.1.
|
||||
The local fake `pay` call now runs inside the order transaction, and the inbound
|
||||
webhook records and applies a delivery transactionally. This does not make an
|
||||
external charge atomic with PostgreSQL: a provider can succeed while the database
|
||||
write fails, or deliver the approval later. Add a durable payment intent,
|
||||
provider idempotency key and reconciliation as part of 1.1 and 3.7.
|
||||
|
||||
### `[~]` 3.6 — Preserve and bind the order the customer actually reviewed
|
||||
|
||||
The browser's width, copies, rotation, mirroring and repetitions are absent from
|
||||
the API item, so the factory cannot reproduce the priced layout. Removing an
|
||||
artwork can leave a stale cart item; editing after quote creation can leave the
|
||||
old quote payable; a new correction can leave an obsolete final active. Persist
|
||||
a versioned per-file production specification, tie the displayed cart to its
|
||||
immutable quote, and tie final approval to the latest correction revision.
|
||||
Cover the real editor-to-quote-to-final journey, not only the pricing table.
|
||||
|
||||
**Local progress 2026-09-23:** The Site includes per-upload width, length,
|
||||
copies, rotation, mirroring, measurement source, and the exact placement of
|
||||
each copy in production specification v2. The API checks coverage, dimensions,
|
||||
film bounds, and quote height; commercial review cannot replace the layout.
|
||||
The order snapshot and downloadable Kanban manifest retain it. Browser quote actions are disabled when
|
||||
the cart differs, including same-price changes. Editor changes invalidate the
|
||||
current cart item immediately; a new customer correction deactivates prior
|
||||
finals. Browser and local API regressions pass. **Still open:** generate and
|
||||
validate the final print file from the approved source revision, and
|
||||
make quote/cart continuity work across devices through a server-authoritative
|
||||
confirmation flow. Current quote binding is a browser guard.
|
||||
|
||||
### `[?]` 3.7 — Complete payment state and reconciliation rules
|
||||
|
||||
Event-ID deduplication does not establish which provider payment settled which
|
||||
quote. Define intent creation, provider transaction ID, currency, paid-at time,
|
||||
pending/rejected/refunded/cancelled states, late approval after quote expiry,
|
||||
overpayment and provider success followed by database failure. Record refused
|
||||
paid events for resolution. Decide who reconciles them and when production must
|
||||
stop or refund. Implement with 1.1 after the checkout/refund policy is approved.
|
||||
|
||||
### `[ ]` 3.8 — Collect a deliverable destination before charging freight
|
||||
|
||||
The quote has a shipping service and CEP but no recipient, street, number,
|
||||
city/state or delivery snapshot. Add and validate these fields with 1.2, then
|
||||
bind the chosen service and final freight amount to the payment intent.
|
||||
|
||||
### `[~]` 3.9 — Make artwork quality and geometry evidence explicit
|
||||
|
||||
Reject or route for review when PDF page count/geometry, image decoding or DPI
|
||||
cannot be established. Do not infer pixels from compressed file size, grade a
|
||||
mixed item from only the readable files, silently shrink oversized artwork, or
|
||||
allow a displayed DPI rejection to proceed through checkout. Use representative
|
||||
real artwork in acceptance checks.
|
||||
|
||||
**Local progress 2026-09-22:** DPI refusal and warning acknowledgement now gate
|
||||
the cart and quote API records the acknowledgement. Oversized loose-art width
|
||||
is rejected in the UI, API production contract, and packer. On 2026-09-23,
|
||||
unreadable loose images are blocked, mixed analyzed/manual sheets receive no
|
||||
automatic grade or discount, and rotated DPI uses the pixel dimension that
|
||||
corresponds to printed width. PDF dimensions now come from the parsed page
|
||||
model, with page count, crop, rotation, and UserUnit checks; multi-page and
|
||||
malformed PDFs block quoting. Isolated browser checks cover those cases and
|
||||
same-origin PDF rendering. Unsupported PDF image operators and representative
|
||||
print-file evidence still need correction before this item can close.
|
||||
|
||||
---
|
||||
|
||||
@@ -403,15 +532,22 @@ charges. Fix as part of 1.1.
|
||||
window of recent finished ones (`BOARD_FINISHED_LIMIT`, default 50) and the true
|
||||
finished total. An operator can never lose a card they could act on; only terminal
|
||||
ones are trimmed. The Kanban column reads "Finalizado · 50 de 213" when truncated,
|
||||
so the count is not mistaken for an all-time total. Pending quotes are capped too.
|
||||
so the count is not mistaken for an all-time total. Pending quotes now have
|
||||
a paginated view; older completed orders still need search in 4.6.
|
||||
- `[ ]` 4.3 — Scan throughput `(F21)`: one `scan_loop` thread, `worker` at
|
||||
`replicas: 1`, ClamAV `MaxThreads 2`, browser gives up after 150s.
|
||||
- `[ ]` 4.4 — Quality grade fallback `(F22)`: when `carregarImagem` fails,
|
||||
`px(f)=Math.sqrt(f.size/1024)*95` stands — a DPI inferred from **file size in
|
||||
bytes** — and it drives up to a 25% discount. Fail closed instead.
|
||||
- `[x]` 4.4 — Failed image decoding no longer infers pixels from compressed
|
||||
file size. Unreadable loose images cannot enter the cart or receive a grade;
|
||||
an isolated browser regression covers the failure path (2026-09-23).
|
||||
- `[x]` 4.5 — Dead config `(F28)`: resolved by deleting `deploy/stack.yaml` in 2.12.
|
||||
`CLAMD_HOST` no longer appears anywhere; `scanning.py` reaching `'scanner'`
|
||||
directly is now simply how it works, not a contradiction.
|
||||
- `[~]` 4.6 — Older unpaid quotes can disappear behind the board limit.
|
||||
On 2026-09-23 the board began showing newest pending and approved unpaid
|
||||
quotes separately, with cursor pagination and counts; a local regression
|
||||
retrieved all 105 pending and 22 approved fixture quotes and cleaned them up.
|
||||
**Still open:** an explicit terminal state for abandoned/expired quotes and
|
||||
search/history for older completed orders.
|
||||
|
||||
---
|
||||
|
||||
@@ -480,6 +616,21 @@ charges. Fix as part of 1.1.
|
||||
days. The copy now states 30 days, says a later order needs the file again, and
|
||||
keeps only the true part: order history remains in the account. Policy unchanged;
|
||||
the promise was corrected to match it.
|
||||
- `[x]` 5.12 — Repair operational entrypoints after the `local/` split.
|
||||
On 2026-09-23, staging and both API images package `ops/`; staging calls
|
||||
`ops.staging_readiness`; local backup calls `ops.storage_backup` through
|
||||
`compose.local.yaml`; documented security and backup commands use the real
|
||||
modules. Verified a network-disabled staging pass with non-secret fixture
|
||||
data, a production API image import, local security status, and a local
|
||||
backup/restore of the database plus 78 clean objects. Production offsite
|
||||
recovery and signature freshness remain separate open items.
|
||||
- `[ ]` 5.13 — Define production recovery: scheduled encrypted offsite database
|
||||
and object backups, a consistent snapshot boundary, Swarm data placement and
|
||||
a restore rehearsal that opens every required live order file.
|
||||
- `[ ]` 5.14 — Promote and verify one immutable release. Scan before publishing
|
||||
mutable tags, make the source preflight validate the active stack, require the
|
||||
browser tests to run, test clean install and upgrade, and check application
|
||||
readiness after Portainer redeploys. Isolate concurrent CI stacks.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user