blyzer/dtf-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: move the engineering documents into docs/
All checks were successful
Build and deploy / Validate source (push) Successful in 5s
Details
Build and deploy / Integration suite on a real stack (push) Successful in 1m18s
Details
Build and deploy / Secret scan and release gate (push) Successful in 6s
Details
Build and deploy / Publish images and notify Portainer (push) Successful in 1m26s
Details

Browse Source 
Thirteen files at the repository root, seven of them documents. Only README.md
earns a place there; the rest are now in docs/ beside the meeting notes, the
client roadmap and the historical material.

The compose files stay. docker-compose.yml is the path the dtf-cloud Portainer
stack reads, so moving it would break deployment, and Docker resolves a compose
file's relative build contexts against its own directory, so moving the other
two would silently break every build. Both reasons are now written down where
someone would otherwise try it.

Correcting references turned up a live fault: the Portainer stack creation
instructions still named deploy/stack.yaml as the compose path. That file was
removed, so anyone recreating the stack from these instructions would have
failed. It names docker-compose.yml now, with the reason it stays at the root.

ROADMAP.md keeps the paths its closed findings were written with, and says so at
the top. Those entries record where a fault was when it was found; rewriting
them to match a later layout would make the record less true, not more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Cauê Faleiros
2026-09-21 17:52:20 -03:00
parent b329f76378
commit 7386469404
11 changed files with 26 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

403
docs/CONTEXT.md Normal file
View File

@@ -0,0 +1,403 @@
# DTF System - Project Context for AI Agents
## Purpose of this document
Read this document before changing code, documentation, architecture, or scope.
It is the current English source of truth for the project. It supersedes older
decisions in `README.md`, `docs/historico/ESPECIFICACAO.md`, `schema.sql`, and the existing
prototype code whenever they conflict.
Update this file whenever the team makes a material product, process, or
architecture decision.
## Project goal
Build a DTF ordering and production-flow system for Dropstar/Altus.
The current process is slow and manual: customers send artwork through
WhatsApp, staff forward files, designers discover problems late, and production
status is not visible outside the factory. The business has substantially more
machine capacity than it currently uses. The goal is to let customers place DTF
orders online, pay, send artwork, and follow production without making
WhatsApp, shared folders, or manual handoffs the bottleneck.
The customer-facing value is speed and clarity. The operations value is a
traceable queue and fewer lost or manually handled orders.
## Current MVP scope
The MVP must be delivered in **at most three weeks**. The target is a working
online system, not factory-machine automation.
### Included
- Dedicated DTF site/subdomain.
- Existing Site DTF commercial rules and product experience.
- Direct, resumable multipart file upload to Cloudflare R2.
- Mercado Pago payment integration with signed and idempotent webhooks.
- Freight quotation and charging at checkout.
- Idempotent Tiny/Olist order integration and order traceability.
- Online Kanban for production status and secure file download.
- WhatsApp status notifications.
- Private object storage, 30-day file retention, backups, and basic security.
- Operator guidance for the factory team.
### Explicitly outside the three-week delivery
- Automatic pre-flight validation. It will be validated after delivery using
real customer files and real printed output, then refined as support work.
- Direct FlexiPRINT integration.
- Factory-side agent, hot folder, internal file server automation, VPN, and
heartbeat monitoring.
- Production-room dashboard, printer/machine telemetry, and advanced reports.
- Automated shipping-label purchase, shipping-label printing, and dispatch
automation. The MVP freight scope is quote selection and charging only.
- Personalized cart behaviour based on past orders, day, or time.
After delivery, the team provides support and evaluates requested changes. Do
not turn post-delivery support into a new committed delivery phase without an
explicit decision.
## Agreed production model
```text
Customer browser
-> Site DTF / API on VPS
-> direct multipart upload to private Cloudflare R2
-> Mercado Pago payment + freight selected at checkout
-> Tiny/Olist order record
-> online Kanban
-> factory operator downloads final file and imports it manually into FlexiPRINT
```
WhatsApp is a notification channel, not the artwork-upload channel. It is used
for payment confirmation, correction requests, production status, and order
completion.
The operator works through the browser. The factory does not need an installed
agent or an internal server for the MVP.
If the factory internet connection fails, only the factory team temporarily
loses access to the Kanban and secure downloads. The public portal, payments,
uploads, R2 objects, queue, and cloud services remain online. Factory work
resumes when local access returns.
## Infrastructure and deployment
- **VPS:** runs the website, API, worker, PostgreSQL, ClamAV, Kanban, and
third-party integrations.
- **Cloudflare R2:** private S3-compatible object storage for original uploads
and final files. It does not run the application, database, worker, or
antivirus.
- **Upload path:** the browser uploads directly to R2 using short-lived,
server-issued presigned multipart URLs. Large files must never be proxied
through the VPS.
- **Deployment:** one Portainer-owned `dtf-cloud` Docker Swarm stack. One Gitea
Actions workflow tests/scans, publishes `latest` plus commit-SHA application
images, and calls the Portainer webhook. Activation remains blocked pending
the production work listed below.
- **Kanban access:** configure `OPERATOR_EMAIL` and `OPERATOR_PASSWORD` as
stack environment variables. If the email is absent, only Kanban login is
blocked; the rest of the stack remains available.
- **Recommended VPS baseline:** 4 vCPU, 16 GB RAM, and 200 GB NVMe. Existing
VPS capacity may be used if it safely meets or exceeds this baseline.
- **Backups:** PostgreSQL backups go to R2. Application secrets are never
committed to Git.
## File retention
The business does not want an unlimited artwork library.
| Artifact | Retention |
|---|---:|
| Incomplete multipart upload | 1 day |
| Rejected or infected upload | 3 days |
| Customer original after approval | Up to 7 days |
| Final print artwork | Maximum 30 days from the first upload |
| Order history and metrics | Kept in the database without keeping the image |
The 30-day retention decision overrides older references to 90 days or
12-month artwork reordering.
## R2 planning cost reference
R2 cost is driven primarily by the average number of gigabytes stored during a
month; read/write operations are expected to be a much smaller cost at the
project's scale. There is no egress charge in the current planning model.
The client-facing roadmap uses the following planning examples, based on
standard R2 storage pricing, the included storage allowance, and an exchange
rate reference of USD 1 = BRL 5.13:
| Average storage | Approx. R2/month | Approx. BRL/month |
|---|---:|---:|
| 100 GB | USD 1.35 | BRL 7 |
| 500 GB | USD 7.35 | BRL 38 |
| 1 TB | USD 14.85 | BRL 76 |
| 3 TB | USD 44.85 | BRL 230 |
These are planning estimates only. Confirm Cloudflare pricing, exchange rates,
taxes, and payment-provider fees before presenting a commercial quote.
## Commercial rules
The existing rules in `web/index.html` are approved as the current source of
truth. Do **not** redesign, simplify, or change prices, discounts, minimums,
rounding, or product modes without explicit approval.
The current site contains four product modes, quality-based price tiers,
artwork-ready discounts, separate assembly charges for loose artwork, a
one-metre minimum, and ten-centimetre billing rounding.
The browser may display the calculation, but production payment creation must
recalculate and validate all price, quantity, freight, and discount values on
the server. Client-provided totals are never authoritative.
Normal JPG/PNG uploads are individual artwork by default, including when uploaded
from the by-metre panel. They use the existing loose-artwork editor, printed
width, quantity, and live packing canvas; packed height determines metres.
Customers uploading an already assembled film-width image must explicitly select
the ready-sheet option before uploading. PDF and manual-format ready sheets keep
their existing workflow. These models retain their separate approved price tiers;
image dimensions alone must never classify a normal artwork as a finished sheet.
## Freight
The original visual freight flow in `web/index.html` was a stub with only pickup
functional. The localhost bridge now supports pickup and backend fake freight
quotes; there is still no real carrier quotation or production checkout.
The customer already uses Correios and other shipping platforms. Before freight
can be completed, obtain:
- The platform(s) that should be used as the source of truth.
- API credentials or delegated access for the selected platform.
- Origin postal code/address.
- Available services and carriers to offer.
- Packaging weight and dimensions by DTF length/package.
- Freight business policy: exact pass-through, subsidy, free-shipping rules,
pickup, or other exceptions.
At checkout, the backend must quote freight from the selected source, store the
chosen service and quoted amount, include the amount in the Mercado Pago
payment, and persist it in the order. Do not create a payment before the freight
amount is final.
## Integrations
### Mercado Pago
Required before production payment integration, not for the local mock milestone:
- Production credentials.
- Webhook configuration/access.
Webhook processing must verify authenticity and be idempotent. A duplicate or
late delivery must not create a duplicate payment, order, message, or queue
card.
### Tiny/Olist
Tiny/Olist is already available. The implementation must create/update orders
idempotently and use the order number for traceability. Confirm the actual API
endpoints, marker/tag behaviour, and rate limits before production use.
### WhatsApp
WhatsApp is already available, but the technical provider still needs to be
confirmed (official Meta Cloud API, Z-API, or another provider).
Implement only the required customer events:
1. Payment approved.
2. Artwork correction needed, with a secure link back to the site.
3. Order entered production.
4. Order ready for collection or shipping.
Messages initiated by the business may require approved templates depending on
the provider and conversation state. Use an outbox/job mechanism with retries,
delivery status handling, and idempotency. Do not send artwork through
WhatsApp.
## Factory responsibilities after delivery
The client/factory team is responsible for:
- Opening the Kanban, downloading final files, and importing them manually into
the current FlexiPRINT setup.
- Performing real print tests and reporting mismatches in the final file or
printed result.
- Maintaining PCs, printers, FlexiPRINT licences, printing profiles, internal
folders, and the local factory network.
- Informing the development team about changes to prices, freight, operational
rules, or external platforms that need system changes.
The development team delivers the online system, provides usage guidance, and
supports subsequent corrections or scoped enhancements.
## Three-week roadmap
| Week | Delivery | Result |
|---|---|---|
| 1 | Infrastructure and upload | VPS, domain, database, R2, multipart upload, antivirus, and service foundation. |
| 2 | Payment, freight, and order | Mercado Pago, freight quotation, idempotent Tiny/Olist order creation, final file generation, and main Kanban states. |
| 3 | Kanban and handover | Online Kanban, secure download, WhatsApp status messages, 30-day retention, and operational handover. |
## Known implementation status
### Active localhost milestone (2026-09-15)
The current implementation target is localhost before any production connection.
Use `docker-compose.yml`, `.env.example`, and `LOCAL_SETUP.md`. The runtime is in
`app/`; historical `portal/`, `kanban/`, `agente/`, and `schema.sql` are preserved
as references and are not imported or started by Compose.
- One local `dtf-cloud` Compose project runs seven long-lived services: Site and
Kanban web gateways, FastAPI Portal/API, PostgreSQL, MinIO, ClamAV, and a
PostgreSQL outbox/scanning worker. Separate database and storage initialization
jobs provision restricted runtime identities, for nine Compose services total.
- MinIO implements the S3 storage boundary with direct multipart browser uploads,
resumable parts, private objects, and short-lived signed operator downloads.
- Payment, freight, Tiny/Olist, and WhatsApp use fake adapters only. No production
credentials or integrations are configured. The API and worker have no external
network route; only local web/storage gateways publish loopback ports.
- The original Site commercial calculation and appearance remain the source of
truth. Its existing browser artwork analysis is retained as prototype advice;
no new automatic pre-flight, server artwork analysis, or print-file generation
is implemented or invoked.
- Since client length and grade could be tampered with and automatic pre-flight
is deferred, a local operator manually confirms those commercial inputs before
checkout. This is a development trust-boundary decision, not a new production
promise. The backend calculates all tiers, discounts, assembly-inclusive rates,
one-metre minimum, ten-centimetre rounding, freight, and final totals. Immutable
approved quotes expire after 24 hours. A customer confirms the server total
on the Site to create one idempotent local paid order.
- Paid orders use `rec`, `tra`, `fil`, `imp`, `cor`, `fin`, with allowed transitions,
operator authentication, concurrency checks, and durable movement history.
Local mock integration receipts are visible in the Kanban.
- The customer portal at `/portal.html` supports local registration/login,
stronger scrypt password hashing with legacy-hash upgrade, revocable HttpOnly
sessions, owned order history/status, secure active
file downloads, and correction uploads. Registration/login can claim only the
current guest session's records, never orders found by email or CNPJ. Email
verification and password recovery are not implemented.
- Unfinished carts (including File blobs) recover from IndexedDB for 24 hours in
the same browser, scoped to the guest/account identity. Recovered items can be
removed/replaced or joined by new items; in-place reconstruction of the artwork
editor and cross-device cart synchronization are not implemented. Browser
storage capacity limits apply. Payment clears the saved cart. Logout revokes
the server session and clears local checkout metadata and saved File blobs
across open Site tabs.
- Operators manually upload and approve a complete final-file set, with one or
more files per order item. Queue entry requires active final files for every
item. Corrections invalidate the old set; customers submit corrections through
their portal, and operators reapprove final files. No artwork transformation,
automatic pre-flight, or machine control is performed.
- Every completed upload is quarantined pending a local ClamAV scan. Only `clean`
files can be quoted, commercially approved, paid, downloaded, attached as final
files, or admitted to the print queue. Rejected/error files remain blocked and
expire within three days. The isolated scanner uses signatures bundled in its
pinned image and has no external network route. The transport accepts files up
to 5 GiB, but the local scan/release limit is 128 MiB; larger files remain
blocked. This malware gate is not print pre-flight or artwork validation.
- A retention worker removes expired object bytes but keeps order/file metadata:
incomplete uploads after one day, originals within seven days of manual final
artwork approval, and attached final/correction files within 30 days of the
order's first upload. Storage lifecycle is also a 30-day backstop.
- Structured security events are written to logs and PostgreSQL. The local
`security_status` command summarizes authentication, rate-limit, scan, and
scanner/signature alerts without exposing secrets. Security regression tests,
exact-runtime Python dependency auditing, and Trivy image reports live under
`app/` and `output/security/`; open findings are documented in
`SECURITY_REPORT.md` and are not a production-readiness claim.
- All direct and transitive Python packages are pinned with artifact hashes in
`infra/requirements.lock`; the API image build requires those hashes. The lock
is regenerated in a disposable Python 3.12 container by
`infra/lock_dependencies.sh`. A fresh exact-runtime audit found no known Python
advisories on 2026-09-15; this does not cover OS/container findings.
- `compose.staging.yaml` is a separate, network-disabled readiness gate only. It
validates non-secret staging decisions and rejects placeholders, local endpoints,
fake providers, embedded secret settings, and unsafe secret sources. It does not
deploy the application, contain credentials, or contact any real provider.
- A separate production delivery package now exists under `deploy/`, with
non-root API/web image definitions, a single Portainer Docker Swarm stack,
external secret/volume contracts, health-monitored rolling updates,
commit-SHA rollback images, and one protected Gitea workflow under
`.gitea/workflows/`. The package never contains provider credentials and has
not been deployed. Its fail-closed preflight intentionally rejects the current
source until production adapters, Docker-secret file loading, approved inputs,
restore rehearsal, image scans, and human security approval are complete.
- `python3 -m app.backup create-and-verify` creates a private, Git-ignored bundle
containing a PostgreSQL dump plus every complete, unexpired object already marked
`clean`. SHA-256 manifests protect both parts. Verification restores the database
under a UUID name and the object bytes under a UUID MinIO prefix, hashes the
restored bytes, then removes only those temporary targets. Active data is never
overwritten; pending, rejected, errored, expired, and purged objects are excluded.
Run this local snapshot while uploads and retention are idle. Scheduling, offsite
copies, and production restore operations remain unfinished.
- Real providers, production authentication hardening, email verification/recovery,
automatic final-file generation, production backup operations, and production
activation remain incomplete. Deployment plumbing exists but cannot pass its
release gate yet. No factory automation or pre-flight is added.
See `docs/historico/IMPLEMENTATION_REPORT.md` for verification, `PRODUCTION_INPUTS.md` for the
decisions and evidence required before staging or production connection, and
`PORTAINER.md` for the production delivery contract.
The client roadmap promises are unchanged, so its PDF source is not regenerated
for this local implementation checkpoint.
### Existing assets
- `web/`: the Site (`index.html`), the Kanban and customer portal pages, and
the scripts behind them. `web/site-*.js` holds the Site's behaviour.
- `docs/historico/`: the original specification, endpoint sketch, task plan and
status report. Background only — they describe a model this system does not
implement.
The prototypes they describe (`portal/`, `kanban/`, `agente/`, the root
`schema.sql` and `.env.exemplo`) were removed on 2026-09-21 once nothing
referenced them; recover from Git history if ever needed.
- `tools/generate_dtf_report.py`: generator for the client-facing roadmap PDF.
- `docs/roadmap-cliente.pdf`: the generated roadmap, as last sent.
- `docs/reuniao-2026-09-09-anotacoes.pdf`:
meeting notes that established the business direction. Treat it as context;
the latest decisions in this document define the active scope.
### Important gaps and stale assumptions
- The original Site had no backend payment, freight quote, order persistence,
authentication, purchase history, or real cart persistence. Local order
persistence, reviewed quotes, mock freight and fake payment now work through
`web/checkout.js`. The local customer portal now provides accounts and
order history; cart recovery is browser-local. Production account verification,
recovery, and cross-device cart editing are still absent.
- Without the local bridge, the original freight fallback remains a stub.
Real freight quotation remains unimplemented in all runtimes.
- The existing the removed prototypes duplicated WhatsApp senders and send
direct text messages. They are not production-ready notification modules.
- Older documentation and code describe a Tiny webhook creating an upload link,
a factory agent, local Kanban, FlexiPRINT automation, 90-day retention, and
12-month artwork reuse. Those are not the current MVP model.
## Documentation synchronization
`context.md` is the compact operating context for agents. The current
client-facing narrative, diagrams, and formatting live in
`docs/roadmap-cliente.pdf`, generated from `tools/generate_dtf_report.py`.
When a decision changes, update both the relevant implementation context here
and the roadmap source when it changes what the client-facing plan promises.
## Change-control rules for agents
- Preserve the current Site DTF commercial rules unless explicitly asked to
change them.
- Do not add factory-side automation, a local agent, hot folders, FlexiPRINT
control, machine dashboards, or a new post-MVP delivery commitment by
inference.
- Do not claim that R2 hosts or executes application services.
- Keep all customer-facing and internal documentation in English only when this
file is the requested artifact; otherwise follow the user's requested
language.
- When a requirement is unclear, distinguish between the MVP, a future
enhancement, and an existing prototype assumption before changing code.

370
docs/LOCAL_SETUP.md Normal file
View File

@@ -0,0 +1,370 @@
# DTF local development
Read `CONTEXT.md` first. The service lives in `app/`; older portal,
Kanban, agent, requirements, and SQL files are historical prototypes.
## Start
Install Docker Engine/Desktop with Compose v2 or later. Docker must be running
and your account must have permission to use it. The initial build downloads
public container images and Python packages; running integrations are local.
From this repository directory:
```bash
docker compose -f compose.local.yaml up --build
```
`docker-compose.yml` is the production/R2 stack and will not start locally; the
localhost stack is always `compose.local.yaml`.
No `.env` is required: Compose has the same disposable defaults as `.env.example`.
To customize, copy `.env.example` to `.env` and edit it. Never use real credentials.
For a detached start with readiness verification:
```bash
docker compose -f compose.local.yaml up --build -d --wait
docker compose -f compose.local.yaml ps
```
| Component | Local URL / port | Purpose |
|---|---|---|
| Site DTF | http://localhost:8080 | Existing Site with local checkout bridge |
| Customer portal | http://localhost:8080/portal.html | Local account, order history, corrections and files |
| Kanban | http://localhost:8081 | Quote review, orders, movement, original downloads |
| API health | http://localhost:8000/health | Checks PostgreSQL and MinIO |
| MinIO S3 | http://localhost:9000 | Direct signed multipart uploads and downloads |
| MinIO console | http://localhost:9001 | Inspect private local storage |
| PostgreSQL | `db:5432`, internal only | Shared persistent data |
| ClamAV | `scanner:3310`, internal only | Isolated malware scanner with bundled signatures |
| Worker | `worker:8002/health`, internal only | Mock outbox delivery, scanning and combined health |
Use `localhost` consistently; mixing it with `127.0.0.1` creates a different
browser session. Published ports bind only to `127.0.0.1`.
Kanban default login: `operator@example.test` / `local-operator-only`.
MinIO default login: `dtf_local` / `local-storage-only`.
These are public, disposable development values, not real credentials.
Interactive API documentation is disabled; `/docs` and `/redoc` are not local
operator interfaces.
## Browser test order
1. Open the Site. Choose **Arquivo por metro** and the manual/table-price path
(CDR/AI/PSD/TIFF). Select `tests/fixtures/local-test.cdr`. This is deliberately
harmless text for upload testing, not a printable CDR file.
2. Enter **1.01 metres**. The original calculation bills **1.10 m × R$19.90 =
R$21.89**, with grade 0 and pickup. You can also use your own non-sensitive
artwork with the original product controls, repetitions, and mixed cart.
3. Use test CNPJ `11.222.333/0001-81`, phone `11999999999`, email
`local-test@example.test`, and pickup. For simulated freight, select delivery,
enter an eight-digit CEP, and click quote. The default mock price is R$15.00.
4. Click the Site checkout button. Files upload directly to MinIO, remain
quarantined until ClamAV returns `clean`, and then become eligible for a quote.
The local banner displays a quote ID awaiting commercial review.
5. Open Kanban and log in. In **Cotações**, download the original if needed,
confirm/correct total metres and grade, tick the manual confirmation, and
click **Aprovar cotação**. For the fixture keep 1.01 m and grade 0.
6. Return to the Site and click its checkout action again. Inspect the
authoritative server total, then click **Criar pedido de teste**. No real
payment occurs.
7. Refresh Kanban. The paid order starts in **Arte recebida**. Move it using
the buttons or drag and drop to **Arte tratada**. Open **Arquivos de produção**,
select the manually prepared final files for every item, enter a review note,
tick the confirmation and click **Aprovar arquivos finais**. Use the harmless
fixture again only for this local test; no printable file is generated.
Continue through **Fila de impressão →
Imprimindo → Finalizado**. A move to **Correção** requires a reason; it can
return to **Arte recebida** or **Arte tratada**. Finalizado is terminal locally.
8. Inspect **Histórico** and **Eventos locais de integração**. Worker receipts
should say recorded locally. Download links expire after five minutes;
request another link from Kanban when needed.
The manual quote step is necessary to enforce the backend trust boundary while
automatic pre-flight is deferred. Browser measurements and grade are proposals.
Only an authenticated operator can approve them; the server derives prices and
freight. Payment accepts a quote ID only. Approved quotes are immutable for
24 hours, and repeated/concurrent payment requests return the same order.
This local review step does not settle the future automated production checkout.
Malware scanning is a separate safety gate and does not inspect dimensions,
resolution, colors, printability, or any other pre-flight property.
The existing client-side previews/analysis remain unchanged and advisory. No new
automatic print pre-flight or artwork validation runs on upload; malware scanning
is the separate gate described above. Original downloads and manually approved
final files are separately labeled. Never print the local text fixture.
## Customer accounts, corrections, and cart recovery
Open **Minha conta** on the Site, or `/portal.html`. Create a local account with
a test CNPJ, phone, email and password of at least 12 characters. Passwords use
the current stronger scrypt format; legacy local hashes are verified and upgraded
on successful login. Sessions are stored in PostgreSQL, expire after
seven days, and are revoked on logout. Login/registration attempts are limited.
No email service is used: email verification and password recovery remain absent.
Registration associates only the current guest session's uploads, quotes and
orders with the new account. Signing in from another browser restores access to
that account's orders. Matching an email or CNPJ never grants access to an order.
Guests can still order and track within their current session.
In the portal, **Ver detalhes e arquivos** displays status history, correction
reasons, files and expiry dates. When Kanban requests **Correção**, upload corrected
files against the relevant item and add a note. The order stays in correction
until the operator reviews it. The old final set is invalidated; the operator
must upload/approve another complete set before re-entering the queue. Concurrent
or repeated submissions with a stale order version are rejected.
The browser saves unfinished cart items and their File blobs in IndexedDB for
24 hours. Reloading restores them as cart rows; remove/replace a row to alter its
artwork settings, or add more products. Existing calculations are preserved and
delivery must be requoted. Payment clears the saved cart. Cart recovery is local
to the browser, not a cross-device artwork library; very large files can exceed
browser storage capacity, which is reported visibly. Expired browser entries are
removed the next time the Site opens. Server retention does not erase downloaded
files or copies stored by the browser/operating system.
Logout also clears local checkout keys and IndexedDB File blobs across open Site
tabs. Operator authentication uses an expiring, revocable HttpOnly session;
browser storage never retains the operator password or a Basic-auth credential.
## Automated checks
Pricing parity requires Python 3.10+ and Node 22+ on the host, no package install:
```bash
python3 -m unittest local.test_pricing -v
```
This executes the real pricing constants/functions extracted from `web/site-config.js`
and compares all four modes, 101 grades, and 11 lengths (4,444 cases) to backend
pricing, plus assembly and invalid-input checks.
With the stack healthy, run the integration test (Python standard library only):
```bash
python3 -m tests.smoke_test
python3 -m tests.workflow_test
```
It checks direct two-part upload/resume, missing parts, session isolation,
private downloads and byte identity, customer validation, tampered totals,
operator corrections, all four products, freight, immutable quotes, concurrent
payment retries, state rules, history, and durable fake integration receipts.
Each run leaves one clearly labeled test order and an approximately 8 MiB object.
The workflow test also checks guest-to-account migration, cross-session login,
revoked sessions, ownership isolation, correction submissions, final revision
invalidation, secure downloads, and mandatory final-file approval before queueing.
Security and malware regressions are separate so an authorized harmless EICAR
test is unmistakable:
```bash
python3 -m tests.security_test
python3 -m tests.scanning_test
docker compose exec -T api python3 -m tests.runtime_security_test
docker compose exec -T api python3 -m tests.retention_test
```
`local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV
must reject it, and quote/download gates must remain closed. The rejected fixture
is retained for at most three days, so it temporarily appears in alert summaries.
For an automated real-browser walkthrough, install Chrome and use Node 22+:
```bash
node tests/browser_test.mjs
```
Set `CHROME_BIN` if Chrome is not at `/usr/bin/google-chrome-stable`. The test
uses a separate temporary browser profile, walks through the actual Site and
Kanban controls, and saves screenshots to `output/local/`. It leaves its local
test order for inspection. Both integration scripts read `.env` automatically.
## Configuration and storage
`.env.example` lists local ports, database/MinIO values, operator login, adapter
selection, mock freight amount, maximum file size (5 GiB), and multipart size
(8 MiB by default). The malware scanner releases only files up to 128 MiB by
default (`SCAN_MAX_BYTES`); larger uploads remain blocked even though the
multipart transport supports 5 GiB. `S3_ENDPOINT=http://storage:9000`, database
hostname `db`,
and the internal service ports are fixed Compose wiring. The public S3 endpoint
must resolve from the browser; keep `http://localhost:9000` for this stack.
Parts use 15-minute presigned URLs and uploads must finish within one day.
Only fake integration adapters and `s3-local` storage are accepted. Startup fails
if a production adapter/environment or nonlocal S3 endpoint is selected.
The API, worker, and database run on an internal Docker network; web gateways
and MinIO also join a network that permits loopback port publishing.
Objects are private, use UUID keys rather than filenames, and persist in a named
volume. MinIO lifecycle rules expire objects after 30 days and abandon incomplete
multipart uploads after one day; the API also blocks expired downloads. Order
history remains in PostgreSQL. Completed files start in `pending`; unknown,
scanner-error, over-limit, encrypted/unsafe, and malware results fail closed.
Only `clean` files can cross quote, payment, download, final-approval, and queue
gates. Rejected/error objects expire within three days. The worker deletes
original bytes within seven days of
manual final artwork approval, and final/correction bytes by 30 days from the
order's first upload. Expired files remain visible as metadata but cannot be
downloaded. Retention runs once a minute; MinIO lifecycle is a backstop.
Upload retries resume completed parts. The saved cart retains files when browser
storage permits; otherwise reselect them. Saved quote IDs also survive reloads.
Clearing cookies loses a guest session, while registered customers can sign in
again. The operator can still inspect order records.
## Local backup and restore check
```bash
python3 -m app.backup create-and-verify
```
This writes a private four-file bundle in `backups/` (ignored by Git and Docker
builds): a PostgreSQL custom-format dump, a gzip object archive, a JSON manifest,
and the manifest's SHA-256 sidecar. The object archive includes only complete,
unexpired files that ClamAV has marked `clean`; pending, rejected, errored, expired,
and purged objects remain excluded. Run the command while uploads and retention are
idle so the database dump and subsequent object selection describe the same local
state.
Verification restores the dump into a newly created UUID-named database and the
object bytes under a UUID-named `originals/restore-verification/` MinIO prefix. It
checks database counts, bundle hashes, every archived object's hash, and the bytes
downloaded after restore, then removes only the temporary database and objects. It
never restores over active data. Keep every bundle file private: it contains
customer data, password hashes, and customer artwork. To verify it again, run
`python3 -m app.backup verify` followed by the printed
`backups/...manifest.json` path. Legacy database-only `.dump` backups remain
verifiable. Scheduling, offsite copies, and a production restore runbook remain
unfinished.
After building the current image, test retention with synthetic files:
```bash
docker compose exec -T api python3 -m tests.retention_test
```
This checks that expired bytes are removed while unexpired files survive. It
cleans up its own synthetic object bytes and retains their metadata.
## Operations and troubleshooting
```bash
docker compose logs --tail=100 api worker scanner
docker compose restart api worker
docker compose ps
docker compose down
```
`down` stops the stack and preserves named database/storage volumes. Restart
with `docker compose up -d --wait`. Do not add `--volumes` unless you intend to
permanently erase all local orders and artwork. No reset is required for tests.
Seven long-running services have Docker health checks; the database and storage
initialization jobs must exit successfully. API health queries PostgreSQL and
MinIO; worker health fails if its database-processing loop, scanning thread, or
live ClamAV PING is unavailable. Both web gateways expose `/health`. MinIO
exposes `/minio/health/ready`.
Run the redacted local alert summary inside the API network namespace:
```bash
docker compose exec -T api python3 -m app.security_status
```
Exit status 1 means attention is required. Review blocked artwork, rate limits,
failed logins, scanner availability, and signature age. Immediately after the
security tests, expected synthetic login failures/rate limits and
`SECURITY-EICAR.cdr` rejections will trigger the alert; confirm names and test
timing before classifying them as expected. Unexpected failures, scanner errors,
stale signatures (more than seven days), or non-test rejected files require
investigation. Events are retained for 30 days. See `SECURITY_REPORT.md`.
If a host port is occupied, change `SITE_PORT`, `KANBAN_PORT`, or `API_PORT` in
`.env` and recreate the stack. MinIO currently reserves 9000/9001. If Docker says
permission denied, fix Docker access or use your OS-approved Docker workflow.
If image/package download fails, check development internet access. A warning
about missing Buildx can fall back to Docker's classic builder; install your
platform's Buildx plugin for the supported modern build path.
If API is unhealthy, inspect its logs and check local adapter settings. Database
and MinIO credentials initialize persistent volumes only once; changing values
later also requires updating the existing database/storage account deliberately.
If an upload fails, keep the session and click checkout again. If parts expired,
reselect the file to start another upload. For a stale Kanban move, refresh;
the backend rejects stale versions instead of overwriting another operator.
The Site retains its existing remote visual assets (fonts/logo) and optional
PDF.js 3.11.174 CDN dependency. Its known eval-based advisory is mitigated in
the current call with `isEvalSupported:false`; the old CDN dependency is still
an upgrade/vendor-pinning limitation, not a claim that it is current. The local
API, manual fixture flow, storage, payment,
freight, Kanban, and mock integrations work without these external services.
Offline PDF previews may fall back to the prototype's manual/table-price path.
## Dependency lock
The API, worker, and database initializer install every Python dependency from
`infra/requirements.lock` with `--require-hashes`. `infra/requirements.txt`
remains the human-maintained direct dependency list. After deliberately changing
a direct pin, regenerate the lock in the same Python 3.12 environment and rebuild:
```bash
./infra/lock_dependencies.sh
docker compose up --build -d --wait
```
The generator downloads public package metadata in a disposable container and
does not modify the host Python environment. Review the resolved versions and run
the exact-runtime audit plus regression suite before accepting an updated lock.
## Staging readiness gate
`compose.staging.yaml` is intentionally not a staging deployment. It runs only a
network-disabled validator for non-secret decisions. After completing
`PRODUCTION_INPUTS.md`, copy `staging/staging.env.example` to the ignored
`staging/staging.env`, enter non-secret metadata, and run:
```bash
docker compose -f compose.staging.yaml run --rm readiness
```
A pass does not authorize deployment or prove provider access. The real staging
composition and external secret injection still require approved provider
contracts and owners. See `staging/README.md`.
## Next production connection step
Keep this stack local. Before connecting real services, confirm the production
checkout trust workflow, complete `PRODUCTION_INPUTS.md`, and pass the isolated
staging-readiness gate. Then implement the actual staging composition using the adapter contracts in
`app/adapters.py`: start with private S3 staging storage, CORS, signed multipart
contract tests and scoped credentials injected outside Git. Add provider sandbox
adapters one at a time. Mercado Pago requires authenticated, signed, idempotent
webhook handling before real payment is allowed; Tiny/Olist and WhatsApp need
confirmed contracts and provider idempotency/delivery handling. Do not simply
change the local fake flags or point this Compose file at production.
## Production delivery package
The repository now includes a separate Docker Swarm and Gitea Actions delivery
package in `deploy/` and `.gitea/workflows/`. It is not used by this localhost
Compose stack and does not alter its volumes. Production images run as
unprivileged users and install the hash-locked Python runtime. Gitea publishes
`latest` for the normal Portainer webhook plus the full commit SHA for rollback.
The stack expects pre-provisioned external Swarm secrets and an external
PostgreSQL volume. Only Site and Kanban ports are published for the existing TLS
reverse proxy; API, database, worker, and scanner remain private.
Run the source-only gate without credentials:
```bash
python3 deploy/production_preflight.py --source-only
```
It must remain blocked while `app/` supports only local fake adapters and does
not load Docker secret `*_FILE` settings. Do not bypass or delete this check.
After approved production implementations and inputs exist, follow
`PORTAINER.md` and `deploy/PRODUCTION_CHECKLIST.md`; release and deployment
still require the Gitea scan/test gates and explicit approvals.

156
docs/PORTAINER.md Normal file
View File

@@ -0,0 +1,156 @@
# Portainer deployment
DTF follows the same operating model as Graphs and ComporHUB: Gitea builds
prebuilt images, pushes them to the Gitea registry, and calls one Portainer
webhook. Portainer owns and redeploys one Docker Swarm stack named `dtf-cloud`.
The deployed stack is the repository's `docker-compose.yml`, which the
`dtf-cloud` Portainer stack points at. It was once accompanied by `deploy/stack.yaml`, a more hardened definition
supplying credentials as Docker secrets; that file was removed in favour of one
definition. See `ROADMAP.md` 2.12 for the reasoning and what remains open.
The stack contains Site, Kanban, API,
worker, PostgreSQL, ClamAV, and a one-time database initializer. Production uses
Cloudflare R2, so MinIO is not part of this stack.
## 1. Gitea configuration
The single workflow is `.gitea/workflows/deploy.yml`. Every push and pull request
runs static validation, the integration suite against a real stack, and a secret
scan. A push to `main` then builds the production images, publishes both `latest`
and the full commit SHA, reports their vulnerabilities, and calls Portainer.
What actually gates a deployment:
| Check | Gates? |
|---|---|
| `py_compile` and the unit tests | yes |
| Integration suite on a live stack (smoke, workflow, security, scanning, retention, runtime) | yes |
| Browser suites | only when the runner has Chrome; otherwise warns and continues |
| Trivy secret scan (HIGH/CRITICAL) | yes |
| Source preflight (`deploy/production_preflight.py --source-only`) | only when `ENFORCE_PRODUCTION_PREFLIGHT` is `true` |
| Trivy image vulnerabilities, CRITICAL | yes |
| Trivy image vulnerabilities, HIGH | no — reported after the build |
The source preflight is advisory by default because it refuses a release while
the payment and messaging adapters are fake, which is the deliberate state the
stack runs in today. Enforcing it now would block every deployment. Set the
repository variable `ENFORCE_PRODUCTION_PREFLIGHT` to `true` once real adapters
land, and it becomes a hard gate.
CRITICAL image findings block. Both images carry none: the bases are pinned by
digest, both Dockerfiles upgrade their OS packages, and the web image moved off
the 1.28 nginx line, which pins `nginx=1.28.3-r1` in `/etc/apk/world` and so
cannot be patched in place. HIGH findings are reported rather than enforced
because the remainder have no upstream fix.
`PYTHON_BASE_IMAGE` and `NGINX_BASE_IMAGE` override the digests pinned in the
Dockerfiles. Update the Dockerfile default in the same change, so the repository
still records what a build used.
Repository variables:
- `REGISTRY_HOST` — normally `gitea.blyzer.com.br`.
- `REGISTRY_OWNER` — normally `blyzer`.
- `PYTHON_BASE_IMAGE`, `NGINX_BASE_IMAGE`, `TRIVY_IMAGE` — approved immutable
`@sha256:` image references.
Repository secrets:
- `REGISTRY_USERNAME` and `REGISTRY_TOKEN` — package write credentials.
- `PORTAINER_WEBHOOK` — webhook generated by the `dtf-cloud` Portainer stack.
The webhook is called only after the gating checks in the table above pass.
`ENFORCE_PRODUCTION_PREFLIGHT` and `TRIVY_IMAGE` are optional repository
variables; without them the preflight is advisory and a pinned default scanner
image is used.
## 2. One-time Portainer resources
Use a Docker Swarm environment. Create a dedicated production PostgreSQL volume
and set its name as `POSTGRES_VOLUME`. Label its Swarm node
`dtf_database=true`. Do not reuse the localhost Compose volume.
Create each application credential under **Secrets**. Use versioned names such
as `dtf_prod_database_url_v1`, then place only those names in the corresponding
`*_SECRET` stack variables. The stack expects distinct secrets for:
- runtime and administrator database URLs;
- database administrator and application-role passwords;
- R2 access key and secret key;
- operator password;
- Mercado Pago token and webhook secret;
- Tiny/Olist token;
- WhatsApp token.
Credential values must never be entered into Git, `portainer.env`, or Gitea
workflow variables.
## 3. Create the stack once
In Portainer select **Stacks → Add stack → Git repository**:
- Name: `dtf-cloud`
- Repository: this Gitea repository
- Reference: `main`
- Compose path: `docker-compose.yml` (the repository default, which is what the
existing `dtf-cloud` stack uses; it must stay at the root for this reason)
- Registry: the private `gitea.blyzer.com.br` registry
- Re-pull image: enabled
- Automatic update: webhook
Load a completed copy of `deploy/portainer.env.example` as the stack's
non-secret environment variables. Do not load the example unchanged: `TBD` and
zero digests are deliberate blockers. Keep `IMAGE_TAG=latest` for normal
webhook deployments.
Before entering those values in Portainer, validate the completed ignored file:
```bash
set -a
. deploy/portainer.env
set +a
python3 deploy/production_preflight.py
```
The public reverse proxy should send the Site hostname to `SITE_PORT` and the
Kanban hostname to `KANBAN_PORT`. `/api/` and `/portal.html` are served through
the Site gateway. Preserve the original Host header. Do not expose PostgreSQL,
API, worker, or ClamAV. Restrict the
two published web ports at the host firewall so only the reverse proxy can use
them, and terminate HTTPS at the proxy.
The `db-init` service completing and stopping is expected. The other six
services must be healthy. A failed `db-init` task or an unhealthy service blocks
acceptance.
## 4. Normal deployment
Push to `main`. Gitea validates, tests, scans, publishes these images, and calls
the webhook:
```text
gitea.blyzer.com.br/blyzer/dtf-api:latest
gitea.blyzer.com.br/blyzer/dtf-api:<full-commit-sha>
gitea.blyzer.com.br/blyzer/dtf-web:latest
gitea.blyzer.com.br/blyzer/dtf-web:<full-commit-sha>
```
In Portainer, verify the new image digests, service health, `db-init` result, and
the public `/health` endpoints. Back up PostgreSQL and R2 before changes that
affect stored data or retention.
## 5. Rollback
In the Portainer stack variables, change `IMAGE_TAG` from `latest` to the full
SHA of the last known-good Gitea commit and redeploy. This rolls back Site, API,
Kanban, and worker code; it does not undo database migrations or restore data.
After recovery and a corrected release, return `IMAGE_TAG` to `latest`.
## Current blocker
This is the final deployment shape, but it is not authorized for production
today. Production adapters, Docker-secret file loading, provider sandbox tests,
current clean base images, approved inputs, and a production restore rehearsal
are still required. Do not bypass `deploy/production_preflight.py` or the Gitea
scan gates to make a deployment run.

62
docs/PRODUCTION_INPUTS.md Normal file
View File

@@ -0,0 +1,62 @@
# DTF staging and production input checklist
Status: input worksheet only. Nothing in this document authorizes a real
integration, production deployment, or use of production credentials.
Do not put passwords, tokens, webhook secrets, private keys, customer data, or
provider recovery codes in this repository. Record only the credential owner and
the approved secret-injection location. Test every provider in an isolated staging
or sandbox environment before production activation.
## Decisions required before staging implementation
| Area | Required decision or evidence | Owner | Status/date |
|---|---|---|---|
| Artwork acceptance | Decide whether customer-entered length and quality grade are accepted directly, manually approved, or verified by another defined process. Name the commercial authority and rejection/correction flow. | | |
| Checkout | Approve the exact transition from quote to payment, including quote expiry, customer confirmation, cancellation, refund, and correction rules. | | |
| Infrastructure | Confirm VPS capacity, operating system, domain/subdomain, DNS owner, TLS termination, Portainer access, Gitea Actions path, and deployment/rollback owner. | | |
| Cloudflare R2 | Provide a staging bucket and endpoint, CORS policy, retention/lifecycle rules, narrowly scoped credential owner, storage budget, and restore/rollback acceptance test. | | |
| PostgreSQL | Define the managed/self-hosted choice, encryption and access policy, backup destination, schedule, retention, restore objective, and named restore-test owner. | | |
| Malware signatures | Approve how ClamAV signatures are refreshed without unrestricted scanner egress, plus stale-signature and scanner-outage response. | | |
| Freight | Select the source-of-truth platform, staging access, origin address/postal code, services/carriers, packaging weight/dimensions by DTF length, pickup rules, and pass-through/subsidy/free-shipping policy. | | |
| Mercado Pago | Provide sandbox account ownership, webhook administration, approved event/status mapping, refund/cancellation policy, reconciliation owner, and secret-injection location. | | |
| Tiny/Olist | Confirm API version/endpoints, staging access, order/product/customer mappings, tag/marker behavior, idempotency key, rate limits, retry rules, and reconciliation owner. | | |
| WhatsApp | Select the provider, sending number owner, opt-in/legal basis, approved templates for the four agreed events, staging access, retry/delivery-failure policy, and support owner. | | |
| Customer accounts | Select email verification and password-recovery provider/flow, session policy, privacy/contact requirements, and customer-support ownership. | | |
| Operators | Define named operator provisioning/removal, roles, MFA expectation, emergency access, and periodic access review. Shared production credentials are not acceptable. | | |
| Monitoring | Name alert recipients and escalation windows for authentication abuse, malware detections, stale signatures, queue failures, provider failures, backup failures, capacity, and downtime. | | |
| Security findings | Record remediation or explicit risk acceptance for the current MinIO, PostgreSQL, Nginx, Debian, and PDF.js findings before a production-readiness review. | | |
## Evidence required before production activation
- A separate staging composition/deployment with no production secrets and no
route from local fake integrations to real provider accounts.
- Server-authoritative pricing regression results, including all four product
modes, discounts, assembly charges, minimum length, rounding, and freight.
- Direct multipart R2 tests for resume, exact part sizes, private access, CORS,
expiry, abort, malware quarantine, secure download, and retention.
- Signed and idempotent Mercado Pago webhook tests, including duplicate, delayed,
invalid, cancelled, and refunded events. No payment may be created before freight
is final.
- Idempotent Tiny/Olist and WhatsApp outbox tests covering retry, duplicate delivery,
rate limiting, permanent failure, and operator reconciliation.
- Account verification/recovery, operator access, TLS, cookie, Host/origin, audit,
alert, and incident-response checks in the target architecture.
- A scheduled, encrypted, offsite PostgreSQL/object backup and a documented restore
rehearsal. The verified localhost bundle is useful evidence, not this production
control.
- A recorded go/no-go review signed by the business owner, operations owner, and
technical owner, with rollback contacts and a support window.
## Safe implementation order after inputs are approved
1. Complete the non-secret metadata and pass the network-disabled readiness gate.
2. Create the actual separate staging composition and external secret-injection path.
3. Validate private R2 multipart storage, malware release gates, downloads,
retention, backup, and restore.
4. Add freight quotation because its final value is required before payment.
5. Add Mercado Pago sandbox checkout and authenticated idempotent webhooks.
6. Add Tiny/Olist through the existing outbox/idempotency boundary.
7. Add only the four agreed WhatsApp notification events.
8. Run the complete functional, security, dependency, image, recovery, and manual
acceptance suite in staging before any production activation.

585
docs/ROADMAP.md Normal file
View File

@@ -0,0 +1,585 @@
# DTF System — Working Roadmap
> Internal engineering tracker. Not a client document, not a promise sheet.
> The client-facing narrative lives in `docs/roadmap-cliente.pdf` and in the weekly
> report, which is a deliverable and is not versioned here.
>
> Update the **Current step** line and the item status every time something moves.
> Add new findings at the bottom of the relevant block rather than rewriting history.
**Current step:** Block 0 closed; Block 2 closed except 2.9–2.11; 4.1, 4.2, 4.5,
5.1, 5.3, 5.4 and 5.8 done. Remaining work needs decisions (3.1, 3.2, 3.3) or
client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
1.1/1.2.
> Paths in closed items are written as they were when the finding was made.
> The repository was laid out by role on 2026-09-21 (`local/` became `app/`,
> with `tests/`, `ops/`, `infra/` and `web/` beside it); the history is left
> as recorded rather than rewritten.
**Last audit:** 2026-09-18, full read of `app/`, `dtf-site.html`, `deploy/`,
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
| Status | Meaning |
|---|---|
| `[ ]` | not started |
| `[~]` | in progress |
| `[x]` | done and verified |
| `[?]` | blocked on a decision (product or client), not on code |
---
## Block 0 · Broken right now
Nothing in this block is optional. Until it is closed, the system cannot be
demonstrated, and the week-1 claims cannot be defended.
### `[x]` 0.1 — API returns 500 on every session, login and registration `(F1)`
`local/auth.py:46` calls `os.environ` and the module never imports `os`.
Reproduced: `NameError: name 'os' is not defined`. `/api/session` calls
`new_session()` whenever there is no cookie, so the Site checkout bridge, cart
recovery and the customer portal all fail on first visit. Introduced in `e3e37f6`.
- Add `import os` to `local/auth.py`.
- Move the `COOKIE_SECURE` read to a module constant so it is evaluated once.
- **Accept:** a fresh browser hits the Site and `/api/session` returns 200 with a
`cart_scope`; `local/smoke_test.py` and `local/workflow_test.py` pass.
### `[x]` 0.2 — "Arquivo por metro" is unreachable in practice
Two independent causes, both from 2026-09-18 commits. Verified in a browser.
**a. Every entry point hard-routes to loose artwork** (`483a083`)
| Control | Currently opens |
|---|---|
| Nav "Impressão DTF" | `avulsa` |
| "DTF Têxtil · 57 cm" | `avulsa` |
| "DTF UV · 28,5 cm" | `uv` |
| Hero "Enviar minha arte" | `avulsa` |
Only the price card reaches `file`. Revert the three `data-modo-cta` attributes
so navigation links land on the product chooser, not on a product.
**b. Dropping a PNG/JPG in `file` mode silently switches the order** `(F25)`
`dtf-site.html` `sel()` — from `abrir('file')`, dropping `imagem-teste.jpg` gives
`modo: "avulsa"`, header "Artes avulsas", price `R$ 29,90/m`. Meanwhile the same
screen says *"Arraste suas folhas montadas · PNG, JPG ou PDF"*, marks
*"PNG, JPG ou PDF · a partir de R$ 14,90"* as the recommended path, and sets
`input.accept=".png,.jpg,.jpeg,.pdf"`. The page invites the drop and then
reprices the order 50% higher.
The escape hatch `#imagemComoFolha` sits above the drop zone as small text inside
an informational notice, defaults to unchecked, and must be ticked *before* the
drop. After the switch fires, `pintaModo()` sets `trocarParaAvulsa.hidden = true`,
so the checkbox disappears and there is no way back in place.
Net effect: the only formats that survive `file` mode are PDF/TIFF/PSD/AI/CDR —
the path the UI itself marks as the worse option. A mixed drop (JPG + PDF) is
rejected and accepts nothing.
- Make the ready-sheet choice an explicit two-option control **inside** the drop
area, styled like the existing `.cam` selector — not a checkbox in a notice.
- Stop advertising PNG/JPG in the by-metre drop zone while rejecting them.
- When a switch does happen, show the price change and offer one-click undo.
- **Accept:** a customer can complete a by-metre order with a PNG from any entry
point, and no product/price change ever happens without a visible confirmation.
### `[x]` 0.3 — Ready-sheet declaration is unverified and worth money `(F22 related)`
Ticking `#imagemComoFolha` is an honour-system claim that moves the price from
R$ 29,90/m to R$ 19,90/m (R$ 14,90 with a good grade). `medirFolha` then derives
sheet height purely from aspect ratio × 57 cm — the original bug, now opt-in.
Measured with `imagem-teste.jpg` (466 × 659 px):
| Route | System behaviour | Billed |
|---|---|---|
| Ticked | treated as a 57 × 80,6 cm mounted sheet | 1 m × R$ 19,90 = **R$ 19,90** |
| Not ticked | 20 cm wide, packs to 28,3 cm of film | 1 m × R$ 29,90 = **R$ 29,90** |
- Validate the claim: declared width must be ≈ film width (57 / 28,5 cm) at a
plausible DPI before the sheet model is accepted.
- Re-check server-side in `/api/operator/quotes/{id}/approve` before pricing.
- **Accept:** a small single artwork declared as a ready sheet is rejected with a
clear message; a genuine 57 cm sheet passes; the operator sees the verdict.
### `[x]` 0.4 — Documented local startup fails `(F2)`
`LOCAL_SETUP.md` says `docker compose up --build` with no `.env`.
`docker compose --env-file .env.example config` exits 1: `R2_ENDPOINT`,
`R2_ACCESS_KEY_ID`, `SITE_DOMAIN`, `KANBAN_DOMAIN` missing. `docker-compose.yml`
became a production/R2 stack in `e3e37f6`; `.env.example` is still the MinIO one
and there is no MinIO service left.
- Decide: keep one production compose and add `compose.local.yaml` with MinIO, or
restore a local default. Recommend the former.
- **Accept:** a clean clone reaches a working Site + Kanban with the documented
command, and `LOCAL_SETUP.md` matches what actually runs.
### `[x]` 0.5 — App DB role shares the admin password `(F3)`
`docker-compose.yml:65` sets `APP_DB_PASSWORD: ${POSTGRES_PASSWORD}` — the same
value as `dtf_admin`. `bootstrap.py` grants the app role DML-only and then hands
it a credential that also logs in as the owner. Anyone reading the API container
env has admin on the database.
- **Accept:** distinct secrets; connecting as `dtf_app` with the admin password fails.
### `[x]` 0.6 — A paid order showed the customer nothing (found while closing Block 0)
`#checkoutStatus` and `#checkoutActions` lived inside `#carr`, and the success path
in `checkout.js` clears the cart (`pedido=[]; limpaPaineis()`) before writing the
confirmation — `.carr{display:none}` then hid the panel holding it. Present since
the first commit; it only surfaced once 0.1 made a payment reachable at all.
Both elements now sit in their own always-visible `.checkout` container.
### How Block 0 was verified
A live stack (`compose.local.yaml`), then the full suite:
| Check | Result |
|---|---|
| `/api/session` on a cold browser | 200 with `cart_scope` + session cookie |
| smoke · workflow · security · scanning | pass |
| retention · runtime security (in-container) | pass |
| `artwork_browser_test.mjs` | pass, updated to the new declared-product behaviour |
| `browser_test.mjs` end-to-end | pass — upload → quote → operator approval → paid order → all Kanban states |
| 4 CI unit tests | pass |
Ports 8090/8091/8010 were used; 8080 was held by an unrelated preview server.
---
## Block 1 · Week 2 — committed to the client
From the report already sent. These are dated promises, not backlog.
- `[ ]` 1.1 — Mercado Pago transparent checkout, signed and idempotent webhooks.
Requires production credentials + webhook access. Payment must never be created
before the freight amount is final.
- `[ ]` 1.2 — Real freight quotation. **Blocked on client inputs** (see
`PRODUCTION_INPUTS.md`): source platform, credentials, origin CEP, services,
packaging weight/dimensions per length, subsidy policy.
- `[ ]` 1.3 — Idempotent Tiny/Olist order creation with order-number traceability.
Confirm endpoints, tag behaviour and rate limits first.
- `[ ]` 1.4 — Final print-file generation (see 3.2 — this is the same problem).
- `[ ]` 1.5 — Main Kanban production states consolidated.
- `[ ]` 1.6 — **Block 0.2 + 0.3**, promised as "início da próxima semana".
`[!]` The production compose currently blocks `dev_paid` (`ENVIRONMENT != 'local'`)
and ships only fake adapters, so the deployed system cannot take an order at all.
1.1 is what unblocks it.
---
## Block 2 · Security — before any public exposure
### `[x]` 2.1 — Rate limiting and audit logs are blind to the client `(F5)`
uvicorn runs without trusted proxy headers (`forwarded_allow_ips` defaults to
`127.0.0.1`; nginx is a different container IP), so `request.client.host` is nginx
for every request. `rate_limit('auth-source', ...)` at 60/15min becomes a single
global bucket — **60 failed logins lock out every customer** — and every `audit()`
record has no attacker IP.
- Set `--proxy-headers` with `FORWARDED_ALLOW_IPS` scoped to the nginx service, or
read `X-Forwarded-For` explicitly at the edge.
- **Accept:** two clients on different IPs have independent buckets; audit rows
carry the real IP.
### `[x]` 2.2 — The public site throttles itself `(F6)`
`local/app.py:115` — `rate_limit('guest-sessions', ENVIRONMENT, 120, 900)` is keyed
on the environment name: 120 new visitors per 15 minutes **site-wide** (~8/min).
Normal traffic 429s. Key per source IP (after 2.1) and raise the ceiling.
### `[x]` 2.3 — `deploy/stack.yaml` cannot boot `(F7)`
It passes `DATABASE_URL_FILE`, `AWS_ACCESS_KEY_ID_FILE`, `OPERATOR_PASSWORD_FILE`,
`OPERATOR_USER`. The code reads `DATABASE_URL`, `AWS_ACCESS_KEY_ID`,
`OPERATOR_PASSWORD`, `OPERATOR_EMAIL`, and no `_FILE` loader exists.
`app.py:58` does `os.environ['OPERATOR_PASSWORD']` → `KeyError` → 500 instead of 503.
- Implement `local/secrets.py` (the preflight already expects it) reading `*_FILE`
with env fallback. Reconcile `OPERATOR_USER` vs `OPERATOR_EMAIL`.
- **Accept:** the stack renders and boots against Swarm secrets; missing operator
config yields 503, not 500.
### `[x]` 2.4 — The documented release gate does not exist `(F8)`
`PORTAINER.md` and `SECURITY_REPORT.md` claim the workflow runs the full isolated
suite, Trivy HIGH/CRITICAL image gates, secret scanning and the source preflight
before calling Portainer. `.gitea/workflows/deploy.yml` runs `py_compile` plus four
unit tests, then builds, pushes `latest` and calls the webhook **unconditionally**.
`deploy/production_preflight.py` is never invoked — only its unit test runs.
- Either implement the gate or correct both documents. Do not leave the gap.
### `[x]` 2.5 — The preflight has silently decayed `(F9)`
It blocks by string-matching source. **4 of 6 markers are dead** after the R2
refactor: `'This runtime only supports APP_ENV=local'`,
`'Only local S3 storage is supported'`, `"allowed_hosts=['localhost', '127.0.0.1']"`,
`"'environment': 'local'"`. String gates weaken without failing.
- Replace marker matching with behavioural assertions (import the module, assert
the adapter classes in use).
### `[x]` 2.12 — Two divergent stack definitions; the docs named the wrong one
Found 2026-09-21 by asking which file Portainer deploys. `PORTAINER.md` called
`deploy/stack.yaml` "the production stack"; the deployed file is the repository's
`docker-compose.yml`. `deploy/stack.yaml` came from the first commit and was never
deployed — it supplied credentials as Docker secrets where the deployed file uses
plain environment variables.
**Decision (2026-09-21): keep `docker-compose.yml`, delete `deploy/stack.yaml`.**
The gain from Docker secrets here is narrower than it sounds. It keeps values out
of `docker inspect` and the Portainer UI, but `local/secrets.py` loads them into
the process environment anyway, and anyone who can read `docker inspect` is
already root or in the docker group and could read the secret files directly. The
operator is the only Portainer user, so the main benefit — limiting what a
lower-privileged console user can see — does not apply. Maintaining two
definitions that drift was the larger real cost.
`local/secrets.py` stays. It is inert against the deployed file and costs nothing,
and it means a stack can switch to Docker secrets later without a code change.
Still open: **rotate the R2 secret key.** Not because of Portainer, but because it
grants read and write over every customer's artwork and has been readable from the
stack environment for some time. The operator password is worth rotating with it.
### `[x]` 2.6 — Base images are not pinned `(F10)`
Dockerfiles default to mutable `python:3.12-slim` / `nginx:1.28-alpine`, the
workflow passes no digest build-args, and `--pull` makes builds non-reproducible —
while `PORTAINER.md` documents digest-pinned immutable bases.
### `[x]` 2.7 — pdf.js loaded from CDN without integrity `(F11)`
Vendored rather than integrity-pinned, so the Site no longer depends on a third
party being reachable and honest when a customer opens it. Both files are served
from this origin and their provenance is recorded in `local/static/vendor/README.md`,
verified against the SRI digests cdnjs publishes for 3.11.174.
`cdnjs.cloudflare.com` is gone from `script-src`, `worker-src` and `connect-src` in
both gateway templates: scripts and workers are now `'self'` plus `blob:` for the
worker the Site builds itself.
Verified in a browser against the running stack: pdf.js loads from `/vendor/`,
the blob worker starts, and a real 7-page PDF parses with no CSP violation. Both
browser suites and the full integration suite pass.
**Still open: the version.** 3.11.174 is old. GHSA-wgrm-67xf-hhpq is mitigated —
`dtf-site.html` already passes `isEvalSupported: false`, which is the documented
workaround — but staying on it indefinitely is not a posture. Upgrading is an API
change rather than a file swap and needs its own browser testing, so it is
deliberately not bundled here.
### `[x]` 2.8 — Single shared operator credential `(F12)`
Accounts now live in `dtf_local.operators`, one per person, with `movements.operator`
and `order_files.created_by` recording who actually acted. Administered from the API
container with `python3 -m app.operators` (list, add, password, disable, enable);
passwords are read from the terminal so they never reach shell history or the
process list, and disabling revokes open sessions immediately rather than leaving
them valid for the rest of the eight-hour window.
Migration was the risk, since getting it wrong locks the factory out of the Kanban.
`OPERATOR_EMAIL`/`OPERATOR_PASSWORD` seed the first account, once: a password
changed through the CLI is never reverted by a stale environment variable on the
next deploy. The first attempt did not work — `db-init` was not given those
variables in either compose file, so no account would have been created and login
would have failed closed with 503. Both files now pass them to the migration job.
Verified end to end: the unchanged credential still logs in, a second operator
authenticates separately, wrong passwords and unknown accounts are rejected, and
disabling ends access at once.
**Roles are deliberately not included.** The meeting described separation of duties
for rework authorisation (Mayana classifies, Thales or Alexandre authorise), but the
rework feature does not exist in this system, so there is nothing for a role to
gate. Building an authorisation model with no consumer would be guesswork. Add roles
with the feature that needs them.
### `[~]` 2.9 — TLS is terminated outside the repository `(F13)`
Downgraded 2026-09-21. The stack publishes plain HTTP on 18080/18081 while
`COOKIE_SECURE: "true"`, and nothing in the repo provisions certificates — but
`nginx-proxy-manager` on the host owns 80/443 and terminates TLS in front of it,
so cookies are not being dropped in practice. This is undocumented operational
knowledge rather than a live defect.
What remains: record the proxy in `PORTAINER.md` as part of the deployment
contract, so nobody moves the stack to a host without one and silently breaks
every session cookie. `TAREFAS.md` A2 still lists the certificate as pending;
confirm it is actually issued for the DTF subdomain.
### `[ ]` 2.10 — No email verification, no password recovery `(F14)`
A locked-out customer has no path back, and registration accepts any CNPJ without
proving control of the e-mail. Needs a transactional mail provider — **client input**.
### `[ ]` 2.11 — LGPD `(F15)`
CNPJ, phone and e-mail are kept indefinitely in `accounts.profile` and
`orders.snapshot`. Artwork has a 30-day policy; personal data has none, and there is
no privacy notice, consent record or deletion path.
---
## Block 3 · Architecture — needs a decision before code
### `[?]` 3.1 — Manual quote approval contradicts the 24h business case `(F16)`
Payment requires `quotes.approved`, set only by an authenticated operator. The
meeting's premise was that the 17h30 order waiting until 5am is what costs the
money. As built, a 2am order still waits for a person. `CONTEXT.md` frames this as a
temporary development trust boundary — the risk is that it silently becomes the
delivered model.
**Decide:** what makes a quote auto-approvable (mode, metre range, grade floor,
returning customer), and what still routes to a human.
### `[?]` 3.2 — Billable metres are computed in the customer's browser `(F17, F18)`
For loose artwork, `metros` comes from `desenhaMontagem`/`encaixar` — a canvas
alpha-mask packer running client-side. The server never recomputes it.
`passoDe()`/`CELULAS_MAX` coarsen the grid for large sheets and image decoding
differs by browser, so **the same cart can price differently on different devices**.
The code comments reference "o motor do servidor"; that engine does not exist.
Worse, the layout the customer is quoted on is never produced — operators upload
final files by hand, so billed metres ≠ printed metres and a designer redoes work
the site already did.
**Decide:** port the packer to the server as the pricing authority and the print-file
generator, with the browser as preview only. This is the single largest gap between
what was promised in the meeting and what exists.
### `[?]` 3.3 — The 5 GB problem is unsolved `(F19)`
Transport accepts 5 GiB; `SCAN_MAX_BYTES` / ClamAV `StreamMaxLength` release only
≤ 128 MiB. Files above that are quarantined permanently with no path forward. This
is exactly the risk Jorge raised in the meeting.
**Decide:** raise the scan ceiling with a resource/timeout design, or define an
explicit large-file path (staged scan, sampled scan, operator override with audit).
### `[ ]` 3.4 — Upload throughput `(F20)`
8 MiB parts, strictly sequential in `local/static/upload.js:21`, one presign
round-trip per part → ~640 sequential API calls for a 5 GB file, through an nginx
`limit_req` of 20r/s. Add parallelism (4–6 in flight) and batch presigning.
### `[ ]` 3.5 — Payment ordering `(F27)`
`dev_paid` charges before persisting the order and passes no idempotency key.
Harmless with `FakePayment`; with Mercado Pago that ordering is how you get double
charges. Fix as part of 1.1.
---
## Block 4 · Scale and performance
- `[x]` 4.1 — Ten indexes added, each matched to a query the application issues,
and no more: every extra index is paid for on each write. The outbox and live
uploads use partial indexes so they stay the size of the backlog rather than of
all history. Verified against the running database — the planner chooses
`outbox_pending` and `orders_owner` for the queries they exist for.
- `[x]` 4.2 — The board returns every order still in progress, however old, plus a
window of recent finished ones (`BOARD_FINISHED_LIMIT`, default 50) and the true
finished total. An operator can never lose a card they could act on; only terminal
ones are trimmed. The Kanban column reads "Finalizado · 50 de 213" when truncated,
so the count is not mistaken for an all-time total. Pending quotes are capped too.
- `[ ]` 4.3 — Scan throughput `(F21)`: one `scan_loop` thread, `worker` at
`replicas: 1`, ClamAV `MaxThreads 2`, browser gives up after 150s.
- `[ ]` 4.4 — Quality grade fallback `(F22)`: when `carregarImagem` fails,
`px(f)=Math.sqrt(f.size/1024)*95` stands — a DPI inferred from **file size in
bytes** — and it drives up to a 25% discount. Fail closed instead.
- `[x]` 4.5 — Dead config `(F28)`: resolved by deleting `deploy/stack.yaml` in 2.12.
`CLAMD_HOST` no longer appears anywhere; `scanning.py` reaching `'scanner'`
directly is now simply how it works, not a contradiction.
---
## Block 5 · Hygiene and maintenance
- `[x]` 5.1 — CI coverage `(F33)`. An `integration` job now builds the localhost
stack and runs smoke, workflow, security, scanning, retention, runtime security
and both browser suites; `publish-and-deploy` depends on it. Verified by
reintroducing the 0.1 defect: `py_compile` and the unit tests still passed while
`smoke_test` failed on `/session`, blocking the release. Browser tests skip with
a warning when the runner has no Chrome — **install `google-chrome-stable` on the
runner (or set `CHROME_BIN`) to make them gate as well.**
- `[x]` 5.2 — `portal/`, `kanban/`, `agente/`, the root `schema.sql` and
`.env.exemplo` removed: 2,283 lines implementing a model this system abandoned,
referenced by nothing, with several endpoints taking the acting user from the
request body. The documents describing them are archived under `docs/historico/`
with a header saying they are background, not instructions.
- `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed
packages the system does not use, and sat next to the hash-locked
`infra/requirements.lock` inviting the wrong one to be installed. Only historical
documentation referred to it.
- `[x]` 5.4 — `pip` removed from `infra/requirements.txt` and from the lock. Nothing
depended on it; it was pinned only because it was listed directly, and installing
it put a package manager inside the read-only runtime image. The base image's own
pip performs the hash-enforced install. Verified: the image builds under
`--require-hashes` and reports the base pip, 25.0.1.
- `[ ]` 5.5 — Doc drift `(F34)`. `README.md`, `CONTEXT.md`, `LOCAL_SETUP.md` and
`SECURITY_REPORT.md` describe a MinIO localhost stack, an API with "no external
network route", a `operator` / `local-operator-only` login the email-validated
model rejects, and a release gate — none match the current tree.
- `[x]` 5.6 — The Site's 1,575-line inline script is now nine files under
`local/static/`, cut at the author's own section boundaries so no function was
split: config, modes, upload, sheet analysis, PDF, quality, packing, cart, flow.
`dtf-site.html` is 1,394 lines of markup and style. The extraction was verified
byte-identical before the tags were swapped in, and they load as classic scripts
in the original order, so evaluation semantics are unchanged.
A consequence worth having: with no inline script anywhere, the policy needs no
hash allowlist and is now simply `script-src 'self'`. `site-packing.js` is also
where a server-side packer (3.2) has to agree, which was the point of splitting.
- `[ ]` 5.7 — Commercial rules duplicated between `FAIXAS` (JS) and `TIERS` (Python)
`(F26)`. `test_pricing` guards parity; generate one from the other instead.
- `[ ]` 5.11 — Nothing tests the schema against an empty database. The 4.1 indexes
were added next to the existing one, which sits before the tables they name, so
`CREATE INDEX ... ON dtf_local.order_files` ran before that table existed. Every
local run passed because the volume already had the tables; CI caught it on its
clean volume. A migration is only really exercised from nothing, so the
integration job should run `down -v` before `up` — or a dedicated step should
apply `schema.sql` twice to a fresh database, proving both a first install and
a re-run.
- `[ ]` 5.10 — The browser suites do not run in CI. Chrome runs in the runner
container and can only reach the stack through ports published on the host, which
is a different network namespace when the runner is itself a container. The API,
workflow, security, scanning, retention and runtime suites were moved inside the
stack's network and do gate. The browser suites are the only coverage for the
artwork editor and the full customer journey, so they need either Chrome in a
container on that network, or a runner with host networking. Until then they gate
locally only, and CI warns when it skips them.
- `[ ]` 5.9 — `local/browser_test.mjs` failed once and passed on an immediate
re-run, with no code change in between (2026-09-21). It is a deploy gate when the
runner has Chrome, so an intermittent failure there blocks releases for no reason.
Suspect Chrome startup timing or a race against stack readiness. Watch it, and if
it recurs add an explicit readiness wait rather than a retry.
- `[x]` 5.8 — The Site claimed 90-day file storage and 12-month history in two
places, and invited customers to reorder "sem subir de novo". Files are kept 30
days. The copy now states 30 days, says a later order needs the file again, and
keeps only the true part: order history remains in the account. Policy unchanged;
the promise was corrected to match it.
---
## Done
### Week 1 — infrastructure, uploads, service base
- `[x]` Compose stack: Site, Kanban, API, PostgreSQL, worker, ClamAV, R2/MinIO storage.
- `[x]` Gitea + Portainer publication path; web service startup and API rollout fixed.
- `[x]` Database passwords with special characters handled via discrete libpq fields.
- `[x]` Kanban e-mail/password login; missing operator config no longer breaks stack boot.
- `[x]` Direct resumable multipart browser → private object storage.
- `[x]` Quarantine + ClamAV gate: only `clean` files can be quoted, paid, downloaded or queued.
- `[x]` Retention worker: incomplete 1d, rejected 3d, originals 7d after approval, finals 30d.
- `[x]` Server-side pricing authority with parity test against the Site JavaScript (4,444 cases).
- `[x]` Loose-artwork packing flow: per-file width, copies, rotate, mirror, live 57 cm preview,
5 mm gap, ruler and watermark preserved; metres follow packed height.
- `[x]` Rotation/mirror applied to packing masks; stale renders no longer overwrite a newer preview.
- `[x]` Hash-locked Python dependencies; Trivy reports in `output/security/`.
### Block 0 — 2026-09-18
- `[x]` `import os` restored in `local/auth.py`; `COOKIE_SECURE` is now a single
module constant shared with `local/app.py`.
- `[x]` Navigation links no longer preselect a product (`data-modo-cta` removed).
- `[x]` Ready sheet vs loose artwork is an explicit, priced, reversible selector
(`#tipoEnvio`), shown in all four modes and locked once a file is attached.
The product is the declaration; `sel()` no longer switches anything silently.
- `[x]` `medirFolha` returns `dpiFolha`; an image that cannot span the film width
at `DPI_RECUSA` is refused as a sheet, with one click to send it as loose artwork.
- `[x]` `pintaCaminhos` scoped to `#caminhos .cam` — its global `.cam` selector was
clobbering the new control.
- `[x]` `compose.local.yaml` restored (MinIO, fake providers, builds from source).
- `[x]` `APP_DB_PASSWORD` separated from `POSTGRES_PASSWORD`, with a `bootstrap.py`
guard that refuses identical credentials in both configuration forms.
- `[x]` Checkout confirmation moved out of the panel the success path hides.
### Block 2 and CI — 2026-09-18
- `[x]` The web gateway overwrites `X-Forwarded-For` with the peer address instead
of appending to it, and `client_ip()` resolves the requester for rate-limit
buckets and security events. Verified: a forged `203.0.113.99` never reaches the
audit trail.
- `[x]` Guest sessions are limited per source. The first attempt used 30/IP, which
the new regression caught as too tight for shared NAT — recreating the original
fault in a narrower form — so the ceiling is 240 per 15 minutes, overridable with
`GUEST_SESSION_LIMIT`.
- `[x]` Security events now carry the source address (`operator_login_failed`,
`customer_login_failed`, `cross_origin_rejected`, `http_security_event`).
- `[x]` CI runs the integration suites against a real stack before publishing.
### 2026-09-21
- `[x]` 2.3 — `local/secrets.py` resolves every `<NAME>_FILE` into `<NAME>` from the
API, worker and bootstrap entrypoints, failing closed on an unreadable or empty
secret and on a value supplied both ways. Verified by booting the API, the worker
and bootstrap with credentials supplied only as mounted files, including a
password containing `:/?#[]&=+$ ,%`. `OPERATOR_USER` in the stack became
`OPERATOR_EMAIL`, which is what the runtime reads.
- `[x]` 2.5 — The gate now loads `local/secrets.py` and makes it resolve every
secret `deploy/stack.yaml` declares, plus asserts it fails closed. Verified
against a no-op loader (11 blockers) and one that swallows a missing file
(1 blocker); only the real implementation passes. The four marker strings that
stopped matching when R2 support landed were removed; the two describing real
blockers stay, so the gate still refuses a release while the payment and
messaging adapters are fake.
- `[x]` 2.4 — A blocking Trivy secret scan was added and verified both ways: a
planted AWS key pair, GitHub token and private key block the job; the repository
passes clean. Worth knowing: Trivy allowlists documented example credentials, so
my first probe passed with AWS's own sample keys — the gate is a backstop, not
permission to commit secrets. The source preflight now runs and always prints its
verdict, enforcing only when `ENFORCE_PRODUCTION_PREFLIGHT` is `true`; enforcing
it today would block every deploy, since it refuses a release while the adapters
are fake. Image vulnerabilities are reported after each build, not enforced —
56 HIGH and 3 CRITICAL, only 15 with an upstream fix. `PORTAINER.md` and
`SECURITY_REPORT.md` now carry a table of what gates and what does not, replacing
descriptions of checks that never ran.
- `[x]` 2.6 — Both bases pinned by digest, OS packages upgraded in the production
images, and the web image moved off the nginx 1.28 line.
| Image | Before | After |
|---|---|---|
| API | 56 HIGH, 3 CRITICAL (15 fixable) | 46 HIGH, 0 CRITICAL |
| Web | 5 HIGH, all unfixable in place | 0 HIGH, 0 CRITICAL |
The 1.28 nginx pins `nginx=1.28.3-r1` in `/etc/apk/world`, so `apk upgrade`
cannot patch it even though Alpine ships `-r7`; `nginx:alpine` (1.31.6) is clean
while `1.29-alpine` scans worse at 37 HIGH. The two remaining "fixable" API
findings are `msgpack` and `setuptools`, which I confirmed are absent from the
built image rather than trusting the earlier report. Local images now share the
pinned bases, so the integration suite exercises what ships; full suite passes on
nginx 1.31.6. With both images at zero CRITICAL, the image scan now **gates on
CRITICAL** and reports HIGH.
### 2026-09-21 — from the runner host inventory
- `[x]` Fixed a regression in 2.1: the production gateway sits behind
`nginx-proxy-manager`, so `$remote_addr` there is the proxy, not the customer.
Overwriting `X-Forwarded-For` with it would have recorded the proxy's address for
every request in production — the same bug 2.1 set out to fix. The gateway now
uses `real_ip` to recover the customer's address from the proxy's header, trusting
only private networks, so a request arriving directly at the published port
cannot spoof it. Validated with `nginx -t` against the rendered config.
- `[~]` 2.9 downgraded: TLS is terminated by that proxy, not missing.
### Reporting
- `[x]` Week-1 client report (`Relatorio-Semana-1-DTF.docx`), corrected 2026-09-18 to
remove the inaccurate "Arquivo por metro permanece separado, com seleção explícita"
claim and the internal commit reference.

182
docs/SECURITY_REPORT.md Normal file
View File

@@ -0,0 +1,182 @@
# Local security closeout
Date: 2026-09-15
## Scope and conclusion
This report covers the localhost-only Site, Portal/API, Kanban, PostgreSQL,
MinIO, ClamAV, and fake integration worker. It does not approve production use
or real Mercado Pago, R2, Tiny/Olist, WhatsApp, freight, or other providers.
Malware scanning is a quarantine control; it is not artwork or print pre-flight.
The local stack is healthy and its security regressions pass. Existing orders
and the named PostgreSQL/MinIO volumes were preserved. Open dependency and image
findings remain and are recorded below; there is no claim of zero vulnerabilities,
complete OWASP compliance, or production readiness.
## Implemented controls
- Loopback-only published ports, local-only adapter guards, internal API/worker/
database/scanner network, Host checks, cross-origin write rejection, CSP,
frame denial, MIME sniffing protection, and no interactive API documentation.
- Escaped untrusted filenames and browser regression coverage with CSP bypassed.
- Expiring, revocable, hashed HttpOnly operator sessions. No Basic credentials or
operator password are retained in browser storage.
- Seven-day revocable customer sessions, stronger scrypt password hashes, legacy
verification and upgrade, comparable missing-account password work, and login
throttling.
- Upload extension/size/count quotas, storage quotas, private UUID object keys,
exact signed multipart `Content-Length`, completion/ownership checks, and
five-minute signed downloads.
- Files start `pending`. Only `clean` files may be quoted, approved, paid,
downloaded, attached as final files, or moved into queue/printing states.
Unknown, scanner error, unsafe/encrypted, over-limit, and malware outcomes fail
closed. Rejected/error files expire within three days.
- Restricted PostgreSQL and MinIO runtime identities provisioned separately from
administrator credentials. Containers use read-only filesystems/capability
drops where compatible.
- Structured redacted security logs plus a 30-day PostgreSQL event table. Worker
health includes outbox progress, scan-thread liveness, and a live ClamAV PING.
The alert summary also reports signature age.
- Logout revokes server sessions and clears checkout keys and IndexedDB File blobs
across open Site tabs.
- Production delivery definitions use unprivileged application/web users,
read-only filesystems, dropped capabilities, external Swarm secrets, private
service networking, immutable base images, commit-SHA rollback tags,
health-monitored rolling updates, and automatic application rollback. A
fail-closed source/configuration gate prevents the current local-only runtime
from being released as production.
## Verified checks
All results below were observed against the final application source. Smoke and
workflow were repeated after the refreshed PostgreSQL 17 image was activated.
| Check | Result |
|---|---|
| Pricing parity against Site JavaScript | Pass: all 4,444 comparisons plus invalid inputs |
| Security regression | Pass: headers, Host/origin, sessions, quotas, multipart signing, throttling |
| Real ClamAV EICAR regression | Pass: rejected and blocked from download/quote; clean control released |
| Smoke workflow | Pass: multipart, ownership, all modes, pricing/freight, payment idempotency, states/history |
| Customer/final-file workflow | Pass: identity, revocation, corrections, secure downloads, invalidation and queue gate |
| Runtime security | Pass: PostgreSQL/MinIO least privilege, hashes, quarantine and scanner failure behavior |
| Retention | Pass: expired bytes removed, live bytes preserved, metadata retained |
| Browser workflow | Pass: end-to-end flow, filename XSS probe, no stored operator password, logout blob cleanup |
| Dependency lock | Pass: 26 exact packages, SHA-256 hashes on every entry, direct-pin parity, hash-enforced image build, `pip check` |
| Staging readiness gate | Pass with synthetic non-secret metadata in a network-disabled container; unsafe/incomplete regression cases rejected |
| Production delivery definitions | Pass: unit/syntax/YAML checks, Swarm render, immutable-base image builds, non-root read-only web runtime and Host rejection |
| Production source preflight | Expected block: local-only/fake adapters and Docker secret-file loading are not production implementations |
| Production ClamAV runtime | Pass: UID 100:101, read-only filesystem, no capabilities/no-new-privileges, live PONG |
| Repository secret scan | Pass: no HIGH/CRITICAL Trivy secret findings; release workflow enforces the same gate |
## Malware-scanner operations
The scanner image is digest-pinned and has no external network route. It uses
the signatures bundled into that image. On closeout it reported ClamAV
`1.5.4/28122/Sun Sep 13 06:26:25 2026`; the summary calculated 31.8 hours of age,
below the seven-day alert threshold.
The multipart transport supports uploads up to 5 GiB, but `SCAN_MAX_BYTES` and
ClamAV stream limits release at most 128 MiB by default. Larger files remain
blocked. Supporting larger files requires a deliberate resource/timeout design,
not simply increasing the upload limit.
Run:
```bash
docker compose exec -T api python3 -m app.security_status
```
An exit status of 1 requires review. At closeout, attention was expected because
the regression suite produced two rate-limit alerts, 20 synthetic failed operator
logins, and two rejected `SECURITY-EICAR.cdr` fixtures. Both rejected fixtures
expire on 2026-09-17. ClamAV was available and its signatures were not stale.
Do not automatically dismiss later alerts: verify filename, timestamp, test run,
scanner availability, and signature age. Treat non-test rejected files, scanner
errors, unexplained authentication bursts, or stale signatures as incidents.
## Dependency and image audit
`pip-audit` inspected the exact packages installed in the hash-enforced rebuilt API
and found no known Python advisories on 2026-09-15. All 26 direct and transitive
runtime packages are pinned with artifact hashes in `infra/requirements.lock`.
`local/lock_dependencies.sh` regenerates it in a disposable Python 3.12 container.
This is a point-in-time package-database result, not proof that the dependencies
or image are vulnerability-free. Scheduled lock refresh and audit automation remain
unfinished.
Trivy JSON reports are in `output/security/`. HIGH/CRITICAL results after the
available custom-image package upgrades were:
| Image | Findings | Qualification |
|---|---:|---|
| API | 46 HIGH | 44 Debian records had no fix; Trivy's two Python records named `msgpack` and `setuptools`, which `pip list` confirmed are absent from the runtime. Trivy warned that the third-party SBOM may be inaccurate. |
| Site | 5 HIGH | Nginx package records with fixes listed by Trivy, but the current official `nginx:1.28-alpine` image/repository did not supply them. |
| PostgreSQL 17 | 30 HIGH, 1 CRITICAL | Nine Alpine library records and 22 records in `/usr/local/bin/gosu`; presence is confirmed, reachability through this local stack is not. |
| MinIO | 102 HIGH, 6 CRITICAL | Findings span the MinIO and `mc` binaries and two OS packages. A verified local clean-object archive now exists, but the old pinned release was not changed without a tested version-migration and rollback plan. |
| ClamAV | 0 HIGH/CRITICAL | This scan result is not a guarantee that no vulnerability exists. |
| Production API validation image | 55 HIGH, 3 CRITICAL | 44 records had no fix. Fourteen listed fixes, but the `msgpack` and `setuptools` records came from a third-party SBOM and both packages were confirmed absent with `pip list`; the remaining fixed records are Debian packages. |
| Production web validation image | 52 HIGH, 2 CRITICAL | All 54 records listed fixed Alpine versions. A current approved Nginx base digest must replace the locally available validation base before release. |
Scanner presence is evidence, while exploitability/reachability requires separate
analysis. MinIO and the remaining base-image findings block any production-readiness
claim even though ports are loopback-only here. The production validation reports
are `production-api-container-audit.json` and
`production-web-container-audit.json`; both make the release workflow's
HIGH/CRITICAL gate fail. Counts reflect the 2026-09-15 Trivy database and can
change when the advisory database or selected base digest changes.
## Production delivery security boundary
The files in `deploy/` and `.gitea/workflows/` are a guarded delivery mechanism,
not an approval to operate the current application on the public internet.
Corrected 2026-09-21: an earlier version of this section described gates the
workflow did not contain. The workflow now runs static validation, the
integration suite against a live stack, and a blocking Trivy secret scan before
publishing. The source preflight is advisory unless
`ENFORCE_PRODUCTION_PREFLIGHT` is set, and image vulnerabilities are reported
rather than enforced, because the current bases carry HIGH/CRITICAL findings
with no upstream fix. Base images are still mutable tags, not digests.
`PORTAINER.md` holds the authoritative table of what gates and what does not.
It publishes both `latest` and the full commit SHA, then calls the Portainer
webhook. Application/provider secrets are created directly as versioned external
Swarm secrets and never cross the workflow. Rollback selects the prior commit SHA
in Portainer and does not roll back the database.
The gate currently identifies real blockers: production adapters and authenticated
payment webhooks are absent, the runtime intentionally rejects production/R2,
Docker secret-file configuration is not implemented, approvals are unset, and
the existing image findings are unresolved. These checks must be satisfied by
implementation and review, not by replacing the checks with permissive values.
## PDF.js review
The Site loads version 3.11.174 from a versioned CDN URL. That version is affected
by [GHSA-wgrm-67xf-hhpq](https://github.com/mozilla/pdf.js/security/advisories/GHSA-wgrm-67xf-hhpq),
whose documented workaround is `isEvalSupported:false`; the existing Site call
already sets that value, so the known eval path is mitigated and is not reported
as unmitigated. The newer
[GHSA-hq66-cqwq-w95j](https://github.com/mozilla/pdf.js/security/advisories/GHSA-hq66-cqwq-w95j)
affects versions from 5.6.83 up to the patched 6.2.108 and therefore does not
include 3.11.174.
Version 3.11.174 is nevertheless old, remotely loaded, and not a satisfactory
long-term dependency posture. Plan a compatibility-tested upgrade and preferably
vendor or integrity-pin the asset. Keep script execution disabled and do not
weaken CSP merely to support a preview.
## Before any staging or production work
- Resolve or formally accept current MinIO, PostgreSQL, Nginx, and Debian image
findings. Use the verified local clean-object bundle to rehearse a MinIO
migration and rollback; it is not a scheduled, offsite, or production backup.
- Schedule controlled dependency-lock refreshes and exact-runtime audits; review
and test every resulting version change.
- Establish a signature update/rebuild process that works without giving the
scanner an unrestricted external network route.
- Replace disposable local secrets; add TLS, production session/cookie settings,
account verification/recovery, centralized monitoring, and complete backup/
restore procedures.
- Re-run every security, malware, workflow, browser, retention, dependency, and
image check in the target staging architecture.