All checks were successful
Build and deploy / Validate source (push) Successful in 10s
Build and deploy / Integration suite on a real stack (push) Successful in 1m47s
Build and deploy / Secret scan and release gate (push) Successful in 7s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m38s
The previous commit used git add -A and swept in four binaries that were deliberately untracked: the week-1 client report as .docx and .pdf, a duplicate of it under output/documents, and imagem-teste.jpg, an input dropped in to test with. None of them are the repository's to version. They are untracked here, left on disk, and covered by .gitignore so the mistake cannot repeat. Three files had no sensible home. The meeting notes sat at the repository root under a 78-character name with spaces and accents, the roadmap generator lived in tmp/ — a directory otherwise ignored as scratch — and its output in output/, which is otherwise generated evidence. They are now docs/reuniao-2026-09-09- anotacoes.pdf, tools/generate_dtf_report.py and docs/roadmap-cliente.pdf, with CONTEXT.md and ROADMAP.md updated to match and a docs/README.md saying what each document is for. .gitignore no longer needs four rules to keep one generator out of an ignored directory; tmp/ is scratch again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
581 lines
31 KiB
Markdown
581 lines
31 KiB
Markdown
# DTF System — Working Roadmap
|
||
|
||
> Internal engineering tracker. Not a client document, not a promise sheet.
|
||
> The client-facing narrative lives in `docs/roadmap-cliente.pdf` and in the weekly
|
||
> report, which is a deliverable and is not versioned here.
|
||
>
|
||
> Update the **Current step** line and the item status every time something moves.
|
||
> Add new findings at the bottom of the relevant block rather than rewriting history.
|
||
|
||
**Current step:** Block 0 closed; Block 2 closed except 2.9–2.11; 4.1, 4.2, 4.5,
|
||
5.1, 5.3, 5.4 and 5.8 done. Remaining work needs decisions (3.1, 3.2, 3.3) or
|
||
client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
|
||
1.1/1.2.
|
||
|
||
**Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`,
|
||
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
|
||
|
||
| Status | Meaning |
|
||
|---|---|
|
||
| `[ ]` | not started |
|
||
| `[~]` | in progress |
|
||
| `[x]` | done and verified |
|
||
| `[?]` | blocked on a decision (product or client), not on code |
|
||
|
||
---
|
||
|
||
## Block 0 · Broken right now
|
||
|
||
Nothing in this block is optional. Until it is closed, the system cannot be
|
||
demonstrated, and the week-1 claims cannot be defended.
|
||
|
||
### `[x]` 0.1 — API returns 500 on every session, login and registration `(F1)`
|
||
|
||
`local/auth.py:46` calls `os.environ` and the module never imports `os`.
|
||
Reproduced: `NameError: name 'os' is not defined`. `/api/session` calls
|
||
`new_session()` whenever there is no cookie, so the Site checkout bridge, cart
|
||
recovery and the customer portal all fail on first visit. Introduced in `e3e37f6`.
|
||
|
||
- Add `import os` to `local/auth.py`.
|
||
- Move the `COOKIE_SECURE` read to a module constant so it is evaluated once.
|
||
- **Accept:** a fresh browser hits the Site and `/api/session` returns 200 with a
|
||
`cart_scope`; `local/smoke_test.py` and `local/workflow_test.py` pass.
|
||
|
||
### `[x]` 0.2 — "Arquivo por metro" is unreachable in practice
|
||
|
||
Two independent causes, both from 2026-09-18 commits. Verified in a browser.
|
||
|
||
**a. Every entry point hard-routes to loose artwork** (`483a083`)
|
||
|
||
| Control | Currently opens |
|
||
|---|---|
|
||
| Nav "Impressão DTF" | `avulsa` |
|
||
| "DTF Têxtil · 57 cm" | `avulsa` |
|
||
| "DTF UV · 28,5 cm" | `uv` |
|
||
| Hero "Enviar minha arte" | `avulsa` |
|
||
|
||
Only the price card reaches `file`. Revert the three `data-modo-cta` attributes
|
||
so navigation links land on the product chooser, not on a product.
|
||
|
||
**b. Dropping a PNG/JPG in `file` mode silently switches the order** `(F25)`
|
||
|
||
`dtf-site.html` `sel()` — from `abrir('file')`, dropping `imagem-teste.jpg` gives
|
||
`modo: "avulsa"`, header "Artes avulsas", price `R$ 29,90/m`. Meanwhile the same
|
||
screen says *"Arraste suas folhas montadas · PNG, JPG ou PDF"*, marks
|
||
*"PNG, JPG ou PDF · a partir de R$ 14,90"* as the recommended path, and sets
|
||
`input.accept=".png,.jpg,.jpeg,.pdf"`. The page invites the drop and then
|
||
reprices the order 50% higher.
|
||
|
||
The escape hatch `#imagemComoFolha` sits above the drop zone as small text inside
|
||
an informational notice, defaults to unchecked, and must be ticked *before* the
|
||
drop. After the switch fires, `pintaModo()` sets `trocarParaAvulsa.hidden = true`,
|
||
so the checkbox disappears and there is no way back in place.
|
||
|
||
Net effect: the only formats that survive `file` mode are PDF/TIFF/PSD/AI/CDR —
|
||
the path the UI itself marks as the worse option. A mixed drop (JPG + PDF) is
|
||
rejected and accepts nothing.
|
||
|
||
- Make the ready-sheet choice an explicit two-option control **inside** the drop
|
||
area, styled like the existing `.cam` selector — not a checkbox in a notice.
|
||
- Stop advertising PNG/JPG in the by-metre drop zone while rejecting them.
|
||
- When a switch does happen, show the price change and offer one-click undo.
|
||
- **Accept:** a customer can complete a by-metre order with a PNG from any entry
|
||
point, and no product/price change ever happens without a visible confirmation.
|
||
|
||
### `[x]` 0.3 — Ready-sheet declaration is unverified and worth money `(F22 related)`
|
||
|
||
Ticking `#imagemComoFolha` is an honour-system claim that moves the price from
|
||
R$ 29,90/m to R$ 19,90/m (R$ 14,90 with a good grade). `medirFolha` then derives
|
||
sheet height purely from aspect ratio × 57 cm — the original bug, now opt-in.
|
||
|
||
Measured with `imagem-teste.jpg` (466 × 659 px):
|
||
|
||
| Route | System behaviour | Billed |
|
||
|---|---|---|
|
||
| Ticked | treated as a 57 × 80,6 cm mounted sheet | 1 m × R$ 19,90 = **R$ 19,90** |
|
||
| Not ticked | 20 cm wide, packs to 28,3 cm of film | 1 m × R$ 29,90 = **R$ 29,90** |
|
||
|
||
- Validate the claim: declared width must be ≈ film width (57 / 28,5 cm) at a
|
||
plausible DPI before the sheet model is accepted.
|
||
- Re-check server-side in `/api/operator/quotes/{id}/approve` before pricing.
|
||
- **Accept:** a small single artwork declared as a ready sheet is rejected with a
|
||
clear message; a genuine 57 cm sheet passes; the operator sees the verdict.
|
||
|
||
### `[x]` 0.4 — Documented local startup fails `(F2)`
|
||
|
||
`LOCAL_SETUP.md` says `docker compose up --build` with no `.env`.
|
||
`docker compose --env-file .env.example config` exits 1: `R2_ENDPOINT`,
|
||
`R2_ACCESS_KEY_ID`, `SITE_DOMAIN`, `KANBAN_DOMAIN` missing. `docker-compose.yml`
|
||
became a production/R2 stack in `e3e37f6`; `.env.example` is still the MinIO one
|
||
and there is no MinIO service left.
|
||
|
||
- Decide: keep one production compose and add `compose.local.yaml` with MinIO, or
|
||
restore a local default. Recommend the former.
|
||
- **Accept:** a clean clone reaches a working Site + Kanban with the documented
|
||
command, and `LOCAL_SETUP.md` matches what actually runs.
|
||
|
||
### `[x]` 0.5 — App DB role shares the admin password `(F3)`
|
||
|
||
`docker-compose.yml:65` sets `APP_DB_PASSWORD: ${POSTGRES_PASSWORD}` — the same
|
||
value as `dtf_admin`. `bootstrap.py` grants the app role DML-only and then hands
|
||
it a credential that also logs in as the owner. Anyone reading the API container
|
||
env has admin on the database.
|
||
|
||
- **Accept:** distinct secrets; connecting as `dtf_app` with the admin password fails.
|
||
|
||
### `[x]` 0.6 — A paid order showed the customer nothing (found while closing Block 0)
|
||
|
||
`#checkoutStatus` and `#checkoutActions` lived inside `#carr`, and the success path
|
||
in `checkout.js` clears the cart (`pedido=[]; limpaPaineis()`) before writing the
|
||
confirmation — `.carr{display:none}` then hid the panel holding it. Present since
|
||
the first commit; it only surfaced once 0.1 made a payment reachable at all.
|
||
Both elements now sit in their own always-visible `.checkout` container.
|
||
|
||
### How Block 0 was verified
|
||
|
||
A live stack (`compose.local.yaml`), then the full suite:
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| `/api/session` on a cold browser | 200 with `cart_scope` + session cookie |
|
||
| smoke · workflow · security · scanning | pass |
|
||
| retention · runtime security (in-container) | pass |
|
||
| `artwork_browser_test.mjs` | pass, updated to the new declared-product behaviour |
|
||
| `browser_test.mjs` end-to-end | pass — upload → quote → operator approval → paid order → all Kanban states |
|
||
| 4 CI unit tests | pass |
|
||
|
||
Ports 8090/8091/8010 were used; 8080 was held by an unrelated preview server.
|
||
|
||
---
|
||
|
||
## Block 1 · Week 2 — committed to the client
|
||
|
||
From the report already sent. These are dated promises, not backlog.
|
||
|
||
- `[ ]` 1.1 — Mercado Pago transparent checkout, signed and idempotent webhooks.
|
||
Requires production credentials + webhook access. Payment must never be created
|
||
before the freight amount is final.
|
||
- `[ ]` 1.2 — Real freight quotation. **Blocked on client inputs** (see
|
||
`PRODUCTION_INPUTS.md`): source platform, credentials, origin CEP, services,
|
||
packaging weight/dimensions per length, subsidy policy.
|
||
- `[ ]` 1.3 — Idempotent Tiny/Olist order creation with order-number traceability.
|
||
Confirm endpoints, tag behaviour and rate limits first.
|
||
- `[ ]` 1.4 — Final print-file generation (see 3.2 — this is the same problem).
|
||
- `[ ]` 1.5 — Main Kanban production states consolidated.
|
||
- `[ ]` 1.6 — **Block 0.2 + 0.3**, promised as "início da próxima semana".
|
||
|
||
`[!]` The production compose currently blocks `dev_paid` (`ENVIRONMENT != 'local'`)
|
||
and ships only fake adapters, so the deployed system cannot take an order at all.
|
||
1.1 is what unblocks it.
|
||
|
||
---
|
||
|
||
## Block 2 · Security — before any public exposure
|
||
|
||
### `[x]` 2.1 — Rate limiting and audit logs are blind to the client `(F5)`
|
||
|
||
uvicorn runs without trusted proxy headers (`forwarded_allow_ips` defaults to
|
||
`127.0.0.1`; nginx is a different container IP), so `request.client.host` is nginx
|
||
for every request. `rate_limit('auth-source', ...)` at 60/15min becomes a single
|
||
global bucket — **60 failed logins lock out every customer** — and every `audit()`
|
||
record has no attacker IP.
|
||
|
||
- Set `--proxy-headers` with `FORWARDED_ALLOW_IPS` scoped to the nginx service, or
|
||
read `X-Forwarded-For` explicitly at the edge.
|
||
- **Accept:** two clients on different IPs have independent buckets; audit rows
|
||
carry the real IP.
|
||
|
||
### `[x]` 2.2 — The public site throttles itself `(F6)`
|
||
|
||
`local/app.py:115` — `rate_limit('guest-sessions', ENVIRONMENT, 120, 900)` is keyed
|
||
on the environment name: 120 new visitors per 15 minutes **site-wide** (~8/min).
|
||
Normal traffic 429s. Key per source IP (after 2.1) and raise the ceiling.
|
||
|
||
### `[x]` 2.3 — `deploy/stack.yaml` cannot boot `(F7)`
|
||
|
||
It passes `DATABASE_URL_FILE`, `AWS_ACCESS_KEY_ID_FILE`, `OPERATOR_PASSWORD_FILE`,
|
||
`OPERATOR_USER`. The code reads `DATABASE_URL`, `AWS_ACCESS_KEY_ID`,
|
||
`OPERATOR_PASSWORD`, `OPERATOR_EMAIL`, and no `_FILE` loader exists.
|
||
`app.py:58` does `os.environ['OPERATOR_PASSWORD']` → `KeyError` → 500 instead of 503.
|
||
|
||
- Implement `local/secrets.py` (the preflight already expects it) reading `*_FILE`
|
||
with env fallback. Reconcile `OPERATOR_USER` vs `OPERATOR_EMAIL`.
|
||
- **Accept:** the stack renders and boots against Swarm secrets; missing operator
|
||
config yields 503, not 500.
|
||
|
||
### `[x]` 2.4 — The documented release gate does not exist `(F8)`
|
||
|
||
`PORTAINER.md` and `SECURITY_REPORT.md` claim the workflow runs the full isolated
|
||
suite, Trivy HIGH/CRITICAL image gates, secret scanning and the source preflight
|
||
before calling Portainer. `.gitea/workflows/deploy.yml` runs `py_compile` plus four
|
||
unit tests, then builds, pushes `latest` and calls the webhook **unconditionally**.
|
||
`deploy/production_preflight.py` is never invoked — only its unit test runs.
|
||
|
||
- Either implement the gate or correct both documents. Do not leave the gap.
|
||
|
||
### `[x]` 2.5 — The preflight has silently decayed `(F9)`
|
||
|
||
It blocks by string-matching source. **4 of 6 markers are dead** after the R2
|
||
refactor: `'This runtime only supports APP_ENV=local'`,
|
||
`'Only local S3 storage is supported'`, `"allowed_hosts=['localhost', '127.0.0.1']"`,
|
||
`"'environment': 'local'"`. String gates weaken without failing.
|
||
|
||
- Replace marker matching with behavioural assertions (import the module, assert
|
||
the adapter classes in use).
|
||
|
||
### `[x]` 2.12 — Two divergent stack definitions; the docs named the wrong one
|
||
|
||
Found 2026-09-21 by asking which file Portainer deploys. `PORTAINER.md` called
|
||
`deploy/stack.yaml` "the production stack"; the deployed file is the repository's
|
||
`docker-compose.yml`. `deploy/stack.yaml` came from the first commit and was never
|
||
deployed — it supplied credentials as Docker secrets where the deployed file uses
|
||
plain environment variables.
|
||
|
||
**Decision (2026-09-21): keep `docker-compose.yml`, delete `deploy/stack.yaml`.**
|
||
|
||
The gain from Docker secrets here is narrower than it sounds. It keeps values out
|
||
of `docker inspect` and the Portainer UI, but `local/secrets.py` loads them into
|
||
the process environment anyway, and anyone who can read `docker inspect` is
|
||
already root or in the docker group and could read the secret files directly. The
|
||
operator is the only Portainer user, so the main benefit — limiting what a
|
||
lower-privileged console user can see — does not apply. Maintaining two
|
||
definitions that drift was the larger real cost.
|
||
|
||
`local/secrets.py` stays. It is inert against the deployed file and costs nothing,
|
||
and it means a stack can switch to Docker secrets later without a code change.
|
||
|
||
Still open: **rotate the R2 secret key.** Not because of Portainer, but because it
|
||
grants read and write over every customer's artwork and has been readable from the
|
||
stack environment for some time. The operator password is worth rotating with it.
|
||
|
||
### `[x]` 2.6 — Base images are not pinned `(F10)`
|
||
|
||
Dockerfiles default to mutable `python:3.12-slim` / `nginx:1.28-alpine`, the
|
||
workflow passes no digest build-args, and `--pull` makes builds non-reproducible —
|
||
while `PORTAINER.md` documents digest-pinned immutable bases.
|
||
|
||
### `[x]` 2.7 — pdf.js loaded from CDN without integrity `(F11)`
|
||
|
||
Vendored rather than integrity-pinned, so the Site no longer depends on a third
|
||
party being reachable and honest when a customer opens it. Both files are served
|
||
from this origin and their provenance is recorded in `local/static/vendor/README.md`,
|
||
verified against the SRI digests cdnjs publishes for 3.11.174.
|
||
|
||
`cdnjs.cloudflare.com` is gone from `script-src`, `worker-src` and `connect-src` in
|
||
both gateway templates: scripts and workers are now `'self'` plus `blob:` for the
|
||
worker the Site builds itself.
|
||
|
||
Verified in a browser against the running stack: pdf.js loads from `/vendor/`,
|
||
the blob worker starts, and a real 7-page PDF parses with no CSP violation. Both
|
||
browser suites and the full integration suite pass.
|
||
|
||
**Still open: the version.** 3.11.174 is old. GHSA-wgrm-67xf-hhpq is mitigated —
|
||
`dtf-site.html` already passes `isEvalSupported: false`, which is the documented
|
||
workaround — but staying on it indefinitely is not a posture. Upgrading is an API
|
||
change rather than a file swap and needs its own browser testing, so it is
|
||
deliberately not bundled here.
|
||
|
||
### `[x]` 2.8 — Single shared operator credential `(F12)`
|
||
|
||
Accounts now live in `dtf_local.operators`, one per person, with `movements.operator`
|
||
and `order_files.created_by` recording who actually acted. Administered from the API
|
||
container with `python -m local.operators` (list, add, password, disable, enable);
|
||
passwords are read from the terminal so they never reach shell history or the
|
||
process list, and disabling revokes open sessions immediately rather than leaving
|
||
them valid for the rest of the eight-hour window.
|
||
|
||
Migration was the risk, since getting it wrong locks the factory out of the Kanban.
|
||
`OPERATOR_EMAIL`/`OPERATOR_PASSWORD` seed the first account, once: a password
|
||
changed through the CLI is never reverted by a stale environment variable on the
|
||
next deploy. The first attempt did not work — `db-init` was not given those
|
||
variables in either compose file, so no account would have been created and login
|
||
would have failed closed with 503. Both files now pass them to the migration job.
|
||
Verified end to end: the unchanged credential still logs in, a second operator
|
||
authenticates separately, wrong passwords and unknown accounts are rejected, and
|
||
disabling ends access at once.
|
||
|
||
**Roles are deliberately not included.** The meeting described separation of duties
|
||
for rework authorisation (Mayana classifies, Thales or Alexandre authorise), but the
|
||
rework feature does not exist in this system, so there is nothing for a role to
|
||
gate. Building an authorisation model with no consumer would be guesswork. Add roles
|
||
with the feature that needs them.
|
||
|
||
### `[~]` 2.9 — TLS is terminated outside the repository `(F13)`
|
||
|
||
Downgraded 2026-09-21. The stack publishes plain HTTP on 18080/18081 while
|
||
`COOKIE_SECURE: "true"`, and nothing in the repo provisions certificates — but
|
||
`nginx-proxy-manager` on the host owns 80/443 and terminates TLS in front of it,
|
||
so cookies are not being dropped in practice. This is undocumented operational
|
||
knowledge rather than a live defect.
|
||
|
||
What remains: record the proxy in `PORTAINER.md` as part of the deployment
|
||
contract, so nobody moves the stack to a host without one and silently breaks
|
||
every session cookie. `TAREFAS.md` A2 still lists the certificate as pending;
|
||
confirm it is actually issued for the DTF subdomain.
|
||
|
||
### `[ ]` 2.10 — No email verification, no password recovery `(F14)`
|
||
|
||
A locked-out customer has no path back, and registration accepts any CNPJ without
|
||
proving control of the e-mail. Needs a transactional mail provider — **client input**.
|
||
|
||
### `[ ]` 2.11 — LGPD `(F15)`
|
||
|
||
CNPJ, phone and e-mail are kept indefinitely in `accounts.profile` and
|
||
`orders.snapshot`. Artwork has a 30-day policy; personal data has none, and there is
|
||
no privacy notice, consent record or deletion path.
|
||
|
||
---
|
||
|
||
## Block 3 · Architecture — needs a decision before code
|
||
|
||
### `[?]` 3.1 — Manual quote approval contradicts the 24h business case `(F16)`
|
||
|
||
Payment requires `quotes.approved`, set only by an authenticated operator. The
|
||
meeting's premise was that the 17h30 order waiting until 5am is what costs the
|
||
money. As built, a 2am order still waits for a person. `CONTEXT.md` frames this as a
|
||
temporary development trust boundary — the risk is that it silently becomes the
|
||
delivered model.
|
||
|
||
**Decide:** what makes a quote auto-approvable (mode, metre range, grade floor,
|
||
returning customer), and what still routes to a human.
|
||
|
||
### `[?]` 3.2 — Billable metres are computed in the customer's browser `(F17, F18)`
|
||
|
||
For loose artwork, `metros` comes from `desenhaMontagem`/`encaixar` — a canvas
|
||
alpha-mask packer running client-side. The server never recomputes it.
|
||
`passoDe()`/`CELULAS_MAX` coarsen the grid for large sheets and image decoding
|
||
differs by browser, so **the same cart can price differently on different devices**.
|
||
The code comments reference "o motor do servidor"; that engine does not exist.
|
||
|
||
Worse, the layout the customer is quoted on is never produced — operators upload
|
||
final files by hand, so billed metres ≠ printed metres and a designer redoes work
|
||
the site already did.
|
||
|
||
**Decide:** port the packer to the server as the pricing authority and the print-file
|
||
generator, with the browser as preview only. This is the single largest gap between
|
||
what was promised in the meeting and what exists.
|
||
|
||
### `[?]` 3.3 — The 5 GB problem is unsolved `(F19)`
|
||
|
||
Transport accepts 5 GiB; `SCAN_MAX_BYTES` / ClamAV `StreamMaxLength` release only
|
||
≤ 128 MiB. Files above that are quarantined permanently with no path forward. This
|
||
is exactly the risk Jorge raised in the meeting.
|
||
|
||
**Decide:** raise the scan ceiling with a resource/timeout design, or define an
|
||
explicit large-file path (staged scan, sampled scan, operator override with audit).
|
||
|
||
### `[ ]` 3.4 — Upload throughput `(F20)`
|
||
|
||
8 MiB parts, strictly sequential in `local/static/upload.js:21`, one presign
|
||
round-trip per part → ~640 sequential API calls for a 5 GB file, through an nginx
|
||
`limit_req` of 20r/s. Add parallelism (4–6 in flight) and batch presigning.
|
||
|
||
### `[ ]` 3.5 — Payment ordering `(F27)`
|
||
|
||
`dev_paid` charges before persisting the order and passes no idempotency key.
|
||
Harmless with `FakePayment`; with Mercado Pago that ordering is how you get double
|
||
charges. Fix as part of 1.1.
|
||
|
||
---
|
||
|
||
## Block 4 · Scale and performance
|
||
|
||
- `[x]` 4.1 — Ten indexes added, each matched to a query the application issues,
|
||
and no more: every extra index is paid for on each write. The outbox and live
|
||
uploads use partial indexes so they stay the size of the backlog rather than of
|
||
all history. Verified against the running database — the planner chooses
|
||
`outbox_pending` and `orders_owner` for the queries they exist for.
|
||
- `[x]` 4.2 — The board returns every order still in progress, however old, plus a
|
||
window of recent finished ones (`BOARD_FINISHED_LIMIT`, default 50) and the true
|
||
finished total. An operator can never lose a card they could act on; only terminal
|
||
ones are trimmed. The Kanban column reads "Finalizado · 50 de 213" when truncated,
|
||
so the count is not mistaken for an all-time total. Pending quotes are capped too.
|
||
- `[ ]` 4.3 — Scan throughput `(F21)`: one `scan_loop` thread, `worker` at
|
||
`replicas: 1`, ClamAV `MaxThreads 2`, browser gives up after 150s.
|
||
- `[ ]` 4.4 — Quality grade fallback `(F22)`: when `carregarImagem` fails,
|
||
`px(f)=Math.sqrt(f.size/1024)*95` stands — a DPI inferred from **file size in
|
||
bytes** — and it drives up to a 25% discount. Fail closed instead.
|
||
- `[x]` 4.5 — Dead config `(F28)`: resolved by deleting `deploy/stack.yaml` in 2.12.
|
||
`CLAMD_HOST` no longer appears anywhere; `scanning.py` reaching `'scanner'`
|
||
directly is now simply how it works, not a contradiction.
|
||
|
||
---
|
||
|
||
## Block 5 · Hygiene and maintenance
|
||
|
||
- `[x]` 5.1 — CI coverage `(F33)`. An `integration` job now builds the localhost
|
||
stack and runs smoke, workflow, security, scanning, retention, runtime security
|
||
and both browser suites; `publish-and-deploy` depends on it. Verified by
|
||
reintroducing the 0.1 defect: `py_compile` and the unit tests still passed while
|
||
`smoke_test` failed on `/session`, blocking the release. Browser tests skip with
|
||
a warning when the runner has no Chrome — **install `google-chrome-stable` on the
|
||
runner (or set `CHROME_BIN`) to make them gate as well.**
|
||
- `[x]` 5.2 — `portal/`, `kanban/`, `agente/`, the root `schema.sql` and
|
||
`.env.exemplo` removed: 2,283 lines implementing a model this system abandoned,
|
||
referenced by nothing, with several endpoints taking the acting user from the
|
||
request body. The documents describing them are archived under `docs/historico/`
|
||
with a header saying they are background, not instructions.
|
||
- `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed
|
||
packages the system does not use, and sat next to the hash-locked
|
||
`local/requirements.lock` inviting the wrong one to be installed. Only historical
|
||
documentation referred to it.
|
||
- `[x]` 5.4 — `pip` removed from `local/requirements.txt` and from the lock. Nothing
|
||
depended on it; it was pinned only because it was listed directly, and installing
|
||
it put a package manager inside the read-only runtime image. The base image's own
|
||
pip performs the hash-enforced install. Verified: the image builds under
|
||
`--require-hashes` and reports the base pip, 25.0.1.
|
||
- `[ ]` 5.5 — Doc drift `(F34)`. `README.md`, `CONTEXT.md`, `LOCAL_SETUP.md` and
|
||
`SECURITY_REPORT.md` describe a MinIO localhost stack, an API with "no external
|
||
network route", a `operator` / `local-operator-only` login the email-validated
|
||
model rejects, and a release gate — none match the current tree.
|
||
- `[x]` 5.6 — The Site's 1,575-line inline script is now nine files under
|
||
`local/static/`, cut at the author's own section boundaries so no function was
|
||
split: config, modes, upload, sheet analysis, PDF, quality, packing, cart, flow.
|
||
`dtf-site.html` is 1,394 lines of markup and style. The extraction was verified
|
||
byte-identical before the tags were swapped in, and they load as classic scripts
|
||
in the original order, so evaluation semantics are unchanged.
|
||
|
||
A consequence worth having: with no inline script anywhere, the policy needs no
|
||
hash allowlist and is now simply `script-src 'self'`. `site-packing.js` is also
|
||
where a server-side packer (3.2) has to agree, which was the point of splitting.
|
||
- `[ ]` 5.7 — Commercial rules duplicated between `FAIXAS` (JS) and `TIERS` (Python)
|
||
`(F26)`. `test_pricing` guards parity; generate one from the other instead.
|
||
- `[ ]` 5.11 — Nothing tests the schema against an empty database. The 4.1 indexes
|
||
were added next to the existing one, which sits before the tables they name, so
|
||
`CREATE INDEX ... ON dtf_local.order_files` ran before that table existed. Every
|
||
local run passed because the volume already had the tables; CI caught it on its
|
||
clean volume. A migration is only really exercised from nothing, so the
|
||
integration job should run `down -v` before `up` — or a dedicated step should
|
||
apply `schema.sql` twice to a fresh database, proving both a first install and
|
||
a re-run.
|
||
- `[ ]` 5.10 — The browser suites do not run in CI. Chrome runs in the runner
|
||
container and can only reach the stack through ports published on the host, which
|
||
is a different network namespace when the runner is itself a container. The API,
|
||
workflow, security, scanning, retention and runtime suites were moved inside the
|
||
stack's network and do gate. The browser suites are the only coverage for the
|
||
artwork editor and the full customer journey, so they need either Chrome in a
|
||
container on that network, or a runner with host networking. Until then they gate
|
||
locally only, and CI warns when it skips them.
|
||
- `[ ]` 5.9 — `local/browser_test.mjs` failed once and passed on an immediate
|
||
re-run, with no code change in between (2026-09-21). It is a deploy gate when the
|
||
runner has Chrome, so an intermittent failure there blocks releases for no reason.
|
||
Suspect Chrome startup timing or a race against stack readiness. Watch it, and if
|
||
it recurs add an explicit readiness wait rather than a retry.
|
||
- `[x]` 5.8 — The Site claimed 90-day file storage and 12-month history in two
|
||
places, and invited customers to reorder "sem subir de novo". Files are kept 30
|
||
days. The copy now states 30 days, says a later order needs the file again, and
|
||
keeps only the true part: order history remains in the account. Policy unchanged;
|
||
the promise was corrected to match it.
|
||
|
||
---
|
||
|
||
## Done
|
||
|
||
### Week 1 — infrastructure, uploads, service base
|
||
|
||
- `[x]` Compose stack: Site, Kanban, API, PostgreSQL, worker, ClamAV, R2/MinIO storage.
|
||
- `[x]` Gitea + Portainer publication path; web service startup and API rollout fixed.
|
||
- `[x]` Database passwords with special characters handled via discrete libpq fields.
|
||
- `[x]` Kanban e-mail/password login; missing operator config no longer breaks stack boot.
|
||
- `[x]` Direct resumable multipart browser → private object storage.
|
||
- `[x]` Quarantine + ClamAV gate: only `clean` files can be quoted, paid, downloaded or queued.
|
||
- `[x]` Retention worker: incomplete 1d, rejected 3d, originals 7d after approval, finals 30d.
|
||
- `[x]` Server-side pricing authority with parity test against the Site JavaScript (4,444 cases).
|
||
- `[x]` Loose-artwork packing flow: per-file width, copies, rotate, mirror, live 57 cm preview,
|
||
5 mm gap, ruler and watermark preserved; metres follow packed height.
|
||
- `[x]` Rotation/mirror applied to packing masks; stale renders no longer overwrite a newer preview.
|
||
- `[x]` Hash-locked Python dependencies; Trivy reports in `output/security/`.
|
||
|
||
### Block 0 — 2026-09-18
|
||
|
||
- `[x]` `import os` restored in `local/auth.py`; `COOKIE_SECURE` is now a single
|
||
module constant shared with `local/app.py`.
|
||
- `[x]` Navigation links no longer preselect a product (`data-modo-cta` removed).
|
||
- `[x]` Ready sheet vs loose artwork is an explicit, priced, reversible selector
|
||
(`#tipoEnvio`), shown in all four modes and locked once a file is attached.
|
||
The product is the declaration; `sel()` no longer switches anything silently.
|
||
- `[x]` `medirFolha` returns `dpiFolha`; an image that cannot span the film width
|
||
at `DPI_RECUSA` is refused as a sheet, with one click to send it as loose artwork.
|
||
- `[x]` `pintaCaminhos` scoped to `#caminhos .cam` — its global `.cam` selector was
|
||
clobbering the new control.
|
||
- `[x]` `compose.local.yaml` restored (MinIO, fake providers, builds from source).
|
||
- `[x]` `APP_DB_PASSWORD` separated from `POSTGRES_PASSWORD`, with a `bootstrap.py`
|
||
guard that refuses identical credentials in both configuration forms.
|
||
- `[x]` Checkout confirmation moved out of the panel the success path hides.
|
||
|
||
### Block 2 and CI — 2026-09-18
|
||
|
||
- `[x]` The web gateway overwrites `X-Forwarded-For` with the peer address instead
|
||
of appending to it, and `client_ip()` resolves the requester for rate-limit
|
||
buckets and security events. Verified: a forged `203.0.113.99` never reaches the
|
||
audit trail.
|
||
- `[x]` Guest sessions are limited per source. The first attempt used 30/IP, which
|
||
the new regression caught as too tight for shared NAT — recreating the original
|
||
fault in a narrower form — so the ceiling is 240 per 15 minutes, overridable with
|
||
`GUEST_SESSION_LIMIT`.
|
||
- `[x]` Security events now carry the source address (`operator_login_failed`,
|
||
`customer_login_failed`, `cross_origin_rejected`, `http_security_event`).
|
||
- `[x]` CI runs the integration suites against a real stack before publishing.
|
||
|
||
### 2026-09-21
|
||
|
||
- `[x]` 2.3 — `local/secrets.py` resolves every `<NAME>_FILE` into `<NAME>` from the
|
||
API, worker and bootstrap entrypoints, failing closed on an unreadable or empty
|
||
secret and on a value supplied both ways. Verified by booting the API, the worker
|
||
and bootstrap with credentials supplied only as mounted files, including a
|
||
password containing `:/?#[]&=+$ ,%`. `OPERATOR_USER` in the stack became
|
||
`OPERATOR_EMAIL`, which is what the runtime reads.
|
||
- `[x]` 2.5 — The gate now loads `local/secrets.py` and makes it resolve every
|
||
secret `deploy/stack.yaml` declares, plus asserts it fails closed. Verified
|
||
against a no-op loader (11 blockers) and one that swallows a missing file
|
||
(1 blocker); only the real implementation passes. The four marker strings that
|
||
stopped matching when R2 support landed were removed; the two describing real
|
||
blockers stay, so the gate still refuses a release while the payment and
|
||
messaging adapters are fake.
|
||
|
||
- `[x]` 2.4 — A blocking Trivy secret scan was added and verified both ways: a
|
||
planted AWS key pair, GitHub token and private key block the job; the repository
|
||
passes clean. Worth knowing: Trivy allowlists documented example credentials, so
|
||
my first probe passed with AWS's own sample keys — the gate is a backstop, not
|
||
permission to commit secrets. The source preflight now runs and always prints its
|
||
verdict, enforcing only when `ENFORCE_PRODUCTION_PREFLIGHT` is `true`; enforcing
|
||
it today would block every deploy, since it refuses a release while the adapters
|
||
are fake. Image vulnerabilities are reported after each build, not enforced —
|
||
56 HIGH and 3 CRITICAL, only 15 with an upstream fix. `PORTAINER.md` and
|
||
`SECURITY_REPORT.md` now carry a table of what gates and what does not, replacing
|
||
descriptions of checks that never ran.
|
||
|
||
- `[x]` 2.6 — Both bases pinned by digest, OS packages upgraded in the production
|
||
images, and the web image moved off the nginx 1.28 line.
|
||
|
||
| Image | Before | After |
|
||
|---|---|---|
|
||
| API | 56 HIGH, 3 CRITICAL (15 fixable) | 46 HIGH, 0 CRITICAL |
|
||
| Web | 5 HIGH, all unfixable in place | 0 HIGH, 0 CRITICAL |
|
||
|
||
The 1.28 nginx pins `nginx=1.28.3-r1` in `/etc/apk/world`, so `apk upgrade`
|
||
cannot patch it even though Alpine ships `-r7`; `nginx:alpine` (1.31.6) is clean
|
||
while `1.29-alpine` scans worse at 37 HIGH. The two remaining "fixable" API
|
||
findings are `msgpack` and `setuptools`, which I confirmed are absent from the
|
||
built image rather than trusting the earlier report. Local images now share the
|
||
pinned bases, so the integration suite exercises what ships; full suite passes on
|
||
nginx 1.31.6. With both images at zero CRITICAL, the image scan now **gates on
|
||
CRITICAL** and reports HIGH.
|
||
|
||
### 2026-09-21 — from the runner host inventory
|
||
|
||
- `[x]` Fixed a regression in 2.1: the production gateway sits behind
|
||
`nginx-proxy-manager`, so `$remote_addr` there is the proxy, not the customer.
|
||
Overwriting `X-Forwarded-For` with it would have recorded the proxy's address for
|
||
every request in production — the same bug 2.1 set out to fix. The gateway now
|
||
uses `real_ip` to recover the customer's address from the proxy's header, trusting
|
||
only private networks, so a request arriving directly at the published port
|
||
cannot spoof it. Validated with `nginx -t` against the rendered config.
|
||
- `[~]` 2.9 downgraded: TLS is terminated by that proxy, not missing.
|
||
|
||
### Reporting
|
||
|
||
- `[x]` Week-1 client report (`Relatorio-Semana-1-DTF.docx`), corrected 2026-09-18 to
|
||
remove the inaccurate "Arquivo por metro permanece separado, com seleção explícita"
|
||
claim and the internal commit reference.
|