# DTF local development Read `CONTEXT.md` first. The service lives in `app/`; older portal, Kanban, agent, requirements, and SQL files are historical prototypes. ## Start Install Docker Engine/Desktop with Compose v2 or later. Docker must be running and your account must have permission to use it. The initial build downloads public container images and Python packages; running integrations are local. From this repository directory: ```bash docker compose -f compose.local.yaml up --build ``` `docker-compose.yml` is the production/R2 stack and will not start locally; the localhost stack is always `compose.local.yaml`. No `.env` is required: Compose has the same disposable defaults as `.env.example`. To customize, copy `.env.example` to `.env` and edit it. Never use real credentials. For a detached start with readiness verification: ```bash docker compose -f compose.local.yaml up --build -d --wait docker compose -f compose.local.yaml ps ``` | Component | Local URL / port | Purpose | |---|---|---| | Site DTF | http://localhost:8080 | Existing Site with local checkout bridge | | Customer portal | http://localhost:8080/portal.html | Local account, order history, corrections and files | | Kanban | http://localhost:8081 | Quote review, orders, movement, original downloads | | API health | http://localhost:8000/health | Checks PostgreSQL and MinIO | | MinIO S3 | http://localhost:9000 | Direct signed multipart uploads and downloads | | MinIO console | http://localhost:9001 | Inspect private local storage | | PostgreSQL | `db:5432`, internal only | Shared persistent data | | ClamAV | `scanner:3310`, internal only | Isolated malware scanner with bundled signatures | | Worker | `worker:8002/health`, internal only | Mock outbox delivery, scanning and combined health | Use `localhost` consistently; mixing it with `127.0.0.1` creates a different browser session. Published ports bind only to `127.0.0.1`. Kanban default login: `operator@example.test` / `local-operator-only`. MinIO default login: `dtf_local` / `local-storage-only`. These are public, disposable development values, not real credentials. Interactive API documentation is disabled; `/docs` and `/redoc` are not local operator interfaces. ## Browser test order 1. Open the Site. Choose **Arquivo por metro** and the manual/table-price path (CDR/AI/PSD/TIFF). Select `tests/fixtures/local-test.cdr`. This is deliberately harmless text for upload testing, not a printable CDR file. 2. Enter **1.01 metres**. The original calculation bills **1.10 m × R$19.90 = R$21.89**, with grade 0 and pickup. You can also use your own non-sensitive artwork with the original product controls, repetitions, and mixed cart. 3. Use test CNPJ `11.222.333/0001-81`, phone `11999999999`, email `local-test@example.test`, and pickup. For simulated freight, select delivery, enter an eight-digit CEP, and click quote. The default mock price is R$15.00. 4. Click the Site checkout button. Files upload directly to MinIO, remain quarantined until ClamAV returns `clean`, and then become eligible for a quote. The local banner displays a quote ID awaiting commercial review. 5. Open Kanban and log in. In **Cotações**, download the original if needed, confirm/correct total metres and grade, tick the manual confirmation, and click **Aprovar cotação**. For the fixture keep 1.01 m and grade 0. 6. Return to the Site and click its checkout action again. Inspect the authoritative server total, then click **Criar pedido de teste**. No real payment occurs. 7. Refresh Kanban. The paid order starts in **Arte recebida**. Move it using the buttons or drag and drop to **Arte tratada**. Open **Arquivos de produção**, select the manually prepared final files for every item, enter a review note, tick the confirmation and click **Aprovar arquivos finais**. Use the harmless fixture again only for this local test. The text fixture is not an image, so its print file shows **preparar à mão**; with a PNG or JPEG artwork the generated PDF is preselected instead (see "Print files"). Continue through **Fila de impressão → Imprimindo → Finalizado**. A move to **Correção** requires a reason; it can return to **Arte recebida** or **Arte tratada**. Finalizado is terminal locally. 8. Inspect **Histórico** and **Eventos locais de integração**. Worker receipts should say recorded locally. Download links expire after five minutes; request another link from Kanban when needed. The manual quote step is necessary to enforce the backend trust boundary while automatic pre-flight is deferred. Browser measurements and grade are proposals. Only an authenticated operator can approve them; the server derives prices and freight. Payment accepts a quote ID only. Approved quotes are immutable for 24 hours, and repeated/concurrent payment requests return the same order. This local review step does not settle the future automated production checkout. Malware scanning is a separate safety gate and does not inspect dimensions, resolution, colors, printability, or any other pre-flight property. The existing client-side previews/analysis remain unchanged and advisory. No new automatic print pre-flight or artwork validation runs on upload; malware scanning is the separate gate described above. Original downloads and manually approved final files are separately labeled. Never print the local text fixture. ## Customer accounts, corrections, and cart recovery Open **Minha conta** on the Site, or `/portal.html`. Create a local account with a test CNPJ, phone, email and password of at least 12 characters. Passwords use the current stronger scrypt format; legacy local hashes are verified and upgraded on successful login. Sessions are stored in PostgreSQL, expire after seven days, and are revoked on logout. Login/registration attempts are limited. No email service is used: email verification and password recovery remain absent. Registration associates only the current guest session's uploads, quotes and orders with the new account. Signing in from another browser restores access to that account's orders. Matching an email or CNPJ never grants access to an order. Guests can still order and track within their current session. In the portal, **Ver detalhes e arquivos** displays status history, correction reasons, files and expiry dates. When Kanban requests **Correção**, upload corrected files against the relevant item and add a note. The order stays in correction until the operator reviews it. The old final set is invalidated; the operator must upload/approve another complete set before re-entering the queue. Concurrent or repeated submissions with a stale order version are rejected. The browser saves unfinished cart items and their File blobs in IndexedDB for 24 hours. Reloading restores them as cart rows; remove/replace a row to alter its artwork settings, or add more products. Existing calculations are preserved and delivery must be requoted. Payment clears the saved cart. Cart recovery is local to the browser, not a cross-device artwork library; very large files can exceed browser storage capacity, which is reported visibly. Expired browser entries are removed the next time the Site opens. Server retention does not erase downloaded files or copies stored by the browser/operating system. Logout also clears local checkout keys and IndexedDB File blobs across open Site tabs. Operator authentication uses an expiring, revocable HttpOnly session; browser storage never retains the operator password or a Basic-auth credential. ## Automated checks Pricing parity requires Python 3.10+ and Node 22+ on the host, no package install: ```bash python3 -m unittest local.test_pricing -v ``` This executes the real pricing constants/functions extracted from `web/site-config.js` and compares all four modes, 101 grades, and 11 lengths (4,444 cases) to backend pricing, plus assembly and invalid-input checks. With the stack healthy, run the integration test (Python standard library only): ```bash python3 -m tests.smoke_test python3 -m tests.workflow_test ``` It checks direct two-part upload/resume, missing parts, session isolation, private downloads and byte identity, customer validation, tampered totals, operator corrections, all four products, freight, immutable quotes, concurrent payment retries, state rules, history, and durable fake integration receipts. Each run leaves one clearly labeled test order and an approximately 8 MiB object. The workflow test also checks guest-to-account migration, cross-session login, revoked sessions, ownership isolation, correction submissions, final revision invalidation, secure downloads, and mandatory final-file approval before queueing. Security and malware regressions are separate so an authorized harmless EICAR test is unmistakable: ```bash python3 -m tests.security_test python3 -m tests.scanning_test docker compose -f compose.local.yaml exec -T api python3 -m tests.runtime_security_test docker compose -f compose.local.yaml exec -T api python3 -m tests.retention_test ``` `local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV must reject it, and quote/download gates must remain closed. The rejected fixture is retained for at most three days, so it temporarily appears in alert summaries. For an automated real-browser walkthrough, install Chrome and use Node 22+: ```bash node tests/browser_test.mjs ``` Set `CHROME_BIN` if Chrome is not at `/usr/bin/google-chrome-stable`. The test uses a separate temporary browser profile, walks through the actual Site and Kanban controls, and saves screenshots to `output/local/`. It leaves its local test order for inspection. Both integration scripts read `.env` automatically. ## Configuration and storage `.env.example` lists local ports, database/MinIO values, operator login, adapter selection, mock freight amount, transport ceiling (5 GiB), and multipart size (8 MiB by default). The API and customer picker admit only files within the malware scanner's effective 128 MiB limit by default (`SCAN_MAX_BYTES`); the larger-file path remains a Week 2 decision. `S3_ENDPOINT=http://storage:9000`, database hostname `db`, and the internal service ports are fixed Compose wiring. The public S3 endpoint must resolve from the browser; keep `http://localhost:9000` for this stack. Parts use 15-minute presigned URLs and unfinished reservations expire after one hour. Clients can cancel a reservation through the upload DELETE endpoint. Fake integration adapters and `s3-local` storage are the default. Mercado Pago and Tiny can be selected for sandbox testing only (see "Provider sandboxes"); startup fails if their credentials are missing, if any other adapter is selected, or if a production environment or nonlocal S3 endpoint is used. The API, worker, and database run on an internal Docker network; web gateways and MinIO also join a network that permits loopback port publishing. Objects are private, use UUID keys rather than filenames, and persist in a named volume. MinIO lifecycle rules expire objects after 30 days and abandon incomplete multipart uploads after one day as a backstop; the API lease and worker release unfinished reservations after one hour. The API also blocks expired downloads. Order history remains in PostgreSQL. Completed files start in `pending`; unknown, scanner-error, over-limit, encrypted/unsafe, and malware results fail closed. Only `clean` files can cross quote, payment, download, final-approval, and queue gates. Rejected/error objects expire within three days. The worker deletes original bytes within seven days of manual final artwork approval, and final/correction bytes by 30 days from the order's first upload. Expired files remain visible as metadata but cannot be downloaded. Retention runs once a minute; MinIO lifecycle is a backstop. Upload retries resume completed parts. The saved cart retains files when browser storage permits; otherwise reselect them. Saved quote IDs also survive reloads. Clearing cookies loses a guest session, while registered customers can sign in again. The operator can still inspect order records. ## Print files Every paid order queues one print-file job per item. The worker renders a PDF exactly as wide as the film and as long as the approved layout, placing each copy at the position, rotation and mirror the customer reviewed. Each source image is embedded once at its original resolution (JPEG bytes unchanged, PNG transparency kept as a soft mask), so nothing is resampled. The Kanban card shows the status per item, the page size and the lowest DPI, and offers **Baixar PDF**. In **Arquivos de produção** the generated file is preselected as the final file; untick it to upload one by hand instead. Only JPEG, PNG, WebP and TIFF are generated. PDF, PSD, AI and CDR artwork, a file whose proportions do not match the quoted size, a layout longer than was billed, or an image above `PRINT_MAX_PIXELS` (250 Mpx by default) goes to **preparar à mão** with the reason, and the operator prepares it as before. **Gerar arquivos de impressão** retries those items and generates files for orders paid before the generator existed. After a customer correction the generated file is no longer offered: it reproduces the replaced artwork. Layouts longer than about 5 m use the PDF `UserUnit` page scale instead of being split into pages. Confirm on the factory's FlexiPRINT that such a file imports at full length before relying on it for long orders. ```bash docker compose -f compose.local.yaml exec -T api python -m unittest tests.test_printfile -v docker compose -f compose.local.yaml exec -T api python -m tests.print_file_test ``` With PyMuPDF installed locally (`pip install pymupdf`), the unit suite also draws each page and checks where every quadrant of the artwork lands. ## Provider sandboxes `app/mercadopago.py` and `app/tiny.py` follow the providers' public API documentation and pass their unit suites against a fake transport. They are not verified integrations until they pass with the client's sandbox accounts. Put only **test** credentials in `.env`: ```bash PAYMENT_ADAPTER=mercadopago MP_ACCESS_TOKEN=TEST-... MP_WEBHOOK_SECRET=... # "Assinatura secreta" in the webhook settings MP_NOTIFICATION_URL=https:///api/payments/webhook TINY_ADAPTER=tiny TINY_TOKEN=... TINY_TAG=Site DTF # optional marker on created orders ``` With Mercado Pago selected, an approved quote shows **Pagar com PIX** on the Site instead of the local test button. The order is created only by the signed notification, after the payment is fetched from the Mercado Pago API and its BRL amount matches the approved total. Mercado Pago must be able to reach the webhook, so a local run needs a public HTTPS tunnel to the Site port. A paid notification that cannot become an order, or a refund on an existing order, appears under **Pagamentos que precisam de atenção** on the Kanban until an operator records the resolution. Card payment needs the Mercado Pago public key and its card form on the Site; that part is not built yet. ## Local backup and restore check ```bash python3 -m ops.backup create-and-verify ``` This writes a private four-file bundle in `backups/` (ignored by Git and Docker builds): a PostgreSQL custom-format dump, a gzip object archive, a JSON manifest, and the manifest's SHA-256 sidecar. The object archive includes only complete, unexpired files that ClamAV has marked `clean`; pending, rejected, errored, expired, and purged objects remain excluded. Run the command while uploads and retention are idle so the database dump and subsequent object selection describe the same local state. Verification restores the dump into a newly created UUID-named database and the object bytes under a UUID-named `originals/restore-verification/` MinIO prefix. It checks database counts, bundle hashes, every archived object's hash, and the bytes downloaded after restore, then removes only the temporary database and objects. It never restores over active data. Keep every bundle file private: it contains customer data, password hashes, and customer artwork. To verify it again, run `python3 -m ops.backup verify` followed by the printed `backups/...manifest.json` path. Legacy database-only `.dump` backups remain verifiable. Scheduling, offsite copies, and a production restore runbook remain unfinished. After building the current image, test retention with synthetic files: ```bash docker compose -f compose.local.yaml exec -T api python3 -m tests.retention_test ``` This checks that expired bytes are removed while unexpired files survive. It cleans up its own synthetic object bytes and retains their metadata. ## Operations and troubleshooting ```bash docker compose -f compose.local.yaml logs --tail=100 api worker scanner docker compose -f compose.local.yaml restart api worker docker compose -f compose.local.yaml ps docker compose -f compose.local.yaml down ``` `down` stops the stack and preserves named database/storage volumes. Restart with `docker compose -f compose.local.yaml up -d --wait`. Do not add `--volumes` unless you intend to permanently erase all local orders and artwork. No reset is required for tests. Seven long-running services have Docker health checks; the database and storage initialization jobs must exit successfully. API health queries PostgreSQL and MinIO; worker health fails if its database-processing loop, scanning thread, or live ClamAV PING is unavailable. Both web gateways expose `/health`. MinIO exposes `/minio/health/ready`. Run the redacted local alert summary inside the API network namespace: ```bash docker compose -f compose.local.yaml exec -T api python3 -m ops.security_status ``` Exit status 1 means attention is required. Review blocked artwork, rate limits, failed logins, scanner availability, and signature age. Immediately after the security tests, expected synthetic login failures/rate limits and `SECURITY-EICAR.cdr` rejections will trigger the alert; confirm names and test timing before classifying them as expected. Unexpected failures, scanner errors, stale signatures (more than seven days), or non-test rejected files require investigation. Events are retained for 30 days. See `SECURITY_REPORT.md`. If a host port is occupied, change `SITE_PORT`, `KANBAN_PORT`, or `API_PORT` in `.env` and recreate the stack. MinIO currently reserves 9000/9001. If Docker says permission denied, fix Docker access or use your OS-approved Docker workflow. If image/package download fails, check development internet access. A warning about missing Buildx can fall back to Docker's classic builder; install your platform's Buildx plugin for the supported modern build path. If API is unhealthy, inspect its logs and check local adapter settings. Database and MinIO credentials initialize persistent volumes only once; changing values later also requires updating the existing database/storage account deliberately. If an upload fails, keep the session and click checkout again. If parts expired, reselect the file to start another upload. For a stale Kanban move, refresh; the backend rejects stale versions instead of overwriting another operator. The Site retains its existing remote visual assets (fonts/logo) and optional PDF.js 3.11.174 CDN dependency. Its known eval-based advisory is mitigated in the current call with `isEvalSupported:false`; the old CDN dependency is still an upgrade/vendor-pinning limitation, not a claim that it is current. The local API, manual fixture flow, storage, payment, freight, Kanban, and mock integrations work without these external services. Offline PDF previews may fall back to the prototype's manual/table-price path. ## Dependency lock The API, worker, and database initializer install every Python dependency from `infra/requirements.lock` with `--require-hashes`. `infra/requirements.txt` remains the human-maintained direct dependency list. After deliberately changing a direct pin, regenerate the lock in the same Python 3.12 environment and rebuild: ```bash ./infra/lock_dependencies.sh docker compose -f compose.local.yaml up --build -d --wait ``` The generator downloads public package metadata in a disposable container and does not modify the host Python environment. Review the resolved versions and run the exact-runtime audit plus regression suite before accepting an updated lock. ## Staging readiness gate `compose.staging.yaml` is intentionally not a staging deployment. It runs only a network-disabled validator for non-secret decisions. After completing `PRODUCTION_INPUTS.md`, copy `staging/staging.env.example` to the ignored `staging/staging.env`, enter non-secret metadata, and run: ```bash docker compose -f compose.staging.yaml run --rm readiness ``` A pass does not authorize deployment or prove provider access. The real staging composition and external secret injection still require approved provider contracts and owners. See `staging/README.md`. ## Next production connection step Keep this stack local. Before connecting real services, confirm the production checkout trust workflow, complete `PRODUCTION_INPUTS.md`, and pass the isolated staging-readiness gate. Then implement the actual staging composition using the adapter contracts in `app/adapters.py`: start with private S3 staging storage, CORS, signed multipart contract tests and scoped credentials injected outside Git. Add provider sandbox adapters one at a time. Mercado Pago requires authenticated, signed, idempotent webhook handling before real payment is allowed; Tiny/Olist and WhatsApp need confirmed contracts and provider idempotency/delivery handling. Do not simply change the local fake flags or point this Compose file at production. ## Production delivery package The repository now includes a separate Docker Swarm and Gitea Actions delivery package in `deploy/` and `.gitea/workflows/`. It is not used by this localhost Compose stack and does not alter its volumes. Production images run as unprivileged users and install the hash-locked Python runtime. Gitea publishes `latest` for the normal Portainer webhook plus the full commit SHA for rollback. The stack expects pre-provisioned external Swarm secrets and an external PostgreSQL volume. Only Site and Kanban ports are published for the existing TLS reverse proxy; API, database, worker, and scanner remain private. Run the source-only gate without credentials: ```bash python3 deploy/production_preflight.py --source-only ``` It must remain blocked while `app/` supports only local fake adapters and does not load Docker secret `*_FILE` settings. Do not bypass or delete this check. After approved production implementations and inputs exist, follow `PORTAINER.md` and `deploy/PRODUCTION_CHECKLIST.md`; release and deployment still require the Gitea scan/test gates and explicit approvals.