dtf-system/IMPLEMENTATION_REPORT.md

# Local milestone implementation report

## Implemented

- A single `dtf-cloud` Compose project: Site, Kanban, FastAPI Portal/API,
  PostgreSQL 17, private MinIO S3 storage, isolated ClamAV, and a mock integration
  outbox/scanning worker. Restricted database and MinIO runtime identities are
  provisioned by separate one-shot initialization jobs.
- Existing Site product UI and commercial functions retained, with a separate
  local checkout bridge. Server pricing mirrors the four price ladders, all
  grade discounts, assembly-inclusive rates, minimum, and rounding.
- Direct multipart browser uploads, part resumption, owned upload completion,
  size verification, private five-minute operator downloads, persistent volumes,
  and 30-day object/one-day incomplete-upload lifecycle rules.
- Authenticated manual quote review supplies trusted commercial quantities and
  grades without adding pre-flight. Immutable server quotes include mock freight;
  the customer confirms the total and creates an idempotent local paid order.
- Shared PostgreSQL orders appear in Kanban. Validated transitions, correction
  reasons, version checks and movement history persist. Tiny/WhatsApp events use
  a transactional outbox, retry scheduling, unique event keys, and fake receipts.
- Loopback-only published ports; API/worker/database on an isolated network;
  guards reject production mode, real adapters and nonlocal storage endpoints.
  All long-running services have health checks. `.env.example` contains disposable local
  defaults only. No new production credentials or provider endpoints.
- Local customer registration/login, scrypt password hashes, revocable database
  sessions and attempt throttling. Customers see owned orders, timelines, final
  files and correction requests, and can upload corrections through the portal.
- Browser-local unfinished-cart recovery for 24 hours, including file blobs,
  with storage errors surfaced. Recovered rows can be removed/replaced and new
  items added. Customer navigation now points to local pages instead of the old
  external account/cart destinations.
- Manual final-file sets cover every order item, support multiple parts, and
  are required before queue entry. Corrections invalidate old final approvals.
  Customer and operator file views distinguish originals, corrections and finals.
- Worker retention cleanup: one-day unfinished uploads, seven-day originals
  after final artwork approval, and final/correction files no later than 30 days
  from the first upload. Combined database/clean-object backup plus isolated,
  hash-checked restore verification.
- Upload extension/size/count quotas, exact signed multipart `Content-Length`,
  Host/cross-origin rejection, CSP and security headers, escaped filenames, and
  logout cleanup of browser cart blobs and checkout metadata.
- Expiring, revocable HttpOnly operator sessions replace browser-stored Basic
  credentials. Customer passwords use stronger scrypt parameters, with legacy
  verification and upgrade on login.
- Completed artwork is quarantined until local ClamAV marks it `clean`. Quote,
  payment, download, final-file approval, queue, and printing gates fail closed.
  Scanner errors/rejections remain blocked and expire within three days. This is
  malware scanning only, not print pre-flight.
- Structured redacted security logging, a 30-day `security_events` table, live
  scanner/signature alert summaries, security regressions, exact-runtime Python
  auditing, and JSON image-scan reports under `output/security/`.
- Reproducible Python 3.12 dependency resolution: all 26 direct/transitive runtime
  packages are pinned with artifact hashes, and image builds enforce those hashes.
- A separate network-disabled staging-readiness gate validates non-secret inputs;
  it is deliberately not an application deployment or provider connection.
- A separate production delivery package defines non-root, hash-locked API/web
  images, one external-secret Portainer Docker Swarm stack, health-monitored
  rolling updates, commit-SHA rollback, and one Gitea test/scan/publish/webhook
  workflow matching the established Graphs/ComporHUB operating model. Its
  preflight deliberately blocks the current local-only source and placeholder
  inputs; no registry push, Swarm deployment, or real provider call occurred.

## URLs

| Component | URL |
|---|---|
| Site | http://localhost:8080 |
| Customer portal | http://localhost:8080/portal.html |
| Kanban | http://localhost:8081 |
| API health | http://localhost:8000/health |
| Local object API and console | http://localhost:9000 · http://localhost:9001 |

Kanban local login: `operator` / `local-operator-only`.
Setup and the complete browser test are in `LOCAL_SETUP.md`.
Interactive `/docs` and `/redoc` are disabled.

## Verification completed

- `docker compose up --build -d --wait`: all seven long-running services healthy;
  both initialization jobs exited successfully and published ports are loopback-only.
- `python3 -m unittest local.test_pricing -v`: all tests passed, including
  4,444 comparisons with the actual Site JavaScript calculator.
- `python3 -m local.smoke_test`: passed multipart resume/incomplete completion,
  download byte identity, private bucket and session ownership, input/tamper
  rejection, four-mode pricing, reviewed corrections, mock freight, concurrent
  payment idempotency, valid/invalid transitions, history and eight mock receipts.
- `node local/browser_test.mjs`: passed actual browser upload, manual review,
  local payment, cart recovery, final-file approval, customer registration,
  customer tracking, Kanban state changes and persisted board reload, with no
  JavaScript exceptions. Screenshots inspected in `output/local/`.
- Local test orders are retained in **Finalizado** for inspection, including
  the browser fixture at **R$21.89** and the four-mode smoke order at **R$537.25**.
- Restarts of all long-running containers preserved order snapshots, states, mock
  receipts and original file bytes; all services returned to healthy.
- Explicit guard checks rejected production mode, every real integration adapter,
  R2 selection and a nonlocal S3 endpoint without contacting external services.
- `python3 -m local.workflow_test`: passed account/session isolation, guest
  migration, revoked-cookie rejection, customer corrections, final revision
  invalidation, secure file downloads and final-file gating.
- `python3 -m local.backup create-and-verify`: the refreshed 2026-09-15 bundle
  archived 61 clean MinIO objects (58,723,323 bytes) alongside the local database,
  verified SHA-256 manifests,
  restored and rehashed every object under an isolated MinIO prefix, restored the
  database with 16 orders and 107 upload records, then removed all temporary restore
  targets. The prior database-only backup also passed the legacy verifier.
- `docker compose exec -T api python -m local.retention_test`: verified expired
  object deletion, preservation of unexpired bytes, and retention of upload
  metadata. Only synthetic retention-test object bytes were cleaned up.
- `python3 -m local.security_test`: passed CSP/frame/Host/origin defenses,
  rejection of Basic credentials, HttpOnly operator-session revocation, extension
  and multipart-size signing, upload quotas, and login throttling.
- `python3 -m local.scanning_test`: a real harmless EICAR fixture was rejected by
  ClamAV and could not be downloaded or quoted; a clean control was released.
- `docker compose exec -T api python -m local.runtime_security_test`: passed
  database/MinIO least privilege, legacy/current password hashes, quarantine
  states, live scanner commands, scan-size rejection, and fail-closed offline behavior.
- The hash-enforced rebuild completed successfully; `pip check`, four staging-gate
  regressions, security, smoke, and runtime-security tests passed. A fresh
  exact-runtime `pip-audit` found no known Python advisories on 2026-09-15.
- `node local/browser_test.mjs`: additionally passed filename XSS probes with CSP
  bypassed, absence of stored operator credentials, and logout removal of File blobs.
- The final rebuilt stack has seven healthy long-running services; Site/API routing and a fresh
  guest portal session were checked after the rebuild.
- Production-package validation passed Python/unit and shell syntax checks, Gitea
  workflow YAML parsing, and `docker stack config` rendering. The API production
  image built from an immutable Python base. The web production image built from
  immutable Python/Nginx bases and ran as UID 101 with a read-only filesystem,
  returning 200 for its configured Host and 400 for an unexpected Host.
- The pinned ClamAV validation image started as UID 100:101 with a read-only
  filesystem, all capabilities dropped, and `no-new-privileges`, then returned
  `PONG` through the production health command; those restrictions are encoded in
  the Swarm stack.
- `python3 deploy/production_preflight.py --source-only` returned the expected
  blocked result for local-only adapters/hosts/fake providers and missing Docker
  secret-file loading. This is verified fail-closed behavior, not production
  acceptance.

The final closeout smoke test retained local order #14 in **Finalizado** with
eight durable fake receipts. The final workflow test again passed identity,
correction, download, invalidation, and final-file gates after the PostgreSQL 17
image refresh. Existing database and MinIO named volumes were preserved.

## Reuse and replacement decisions

| Existing asset | Decision |
|---|---|
| `dtf-site.html` | Reused directly. Preserved layout, modes, pricing, client previews, minimums and rounding; added file references and hooks for local checkout/freight. |
| `portal/main.py` | Inspected and preserved. Reused FastAPI and direct signed S3 upload design; replaced runtime with `local/app.py` because the prototype starts from Tiny and invokes pre-flight/agent delivery. |
| `portal/preflight.py` | Inspected, untouched, never imported by local runtime. |
| `kanban/main.py` | Inspected and preserved. Reused workflow state names, movement-history concept and outbox approach; replaced SQLite/folder/machine runtime with PostgreSQL endpoints. |
| `kanban/static/kanban.html` | Inspected and preserved. Reused dark palette, cards, columns and drag/drop pattern in `local/static/kanban.html`. Legacy machine controls and conflicting local/server handlers are not loaded. |
| Duplicated Tiny/WhatsApp modules | Preserved but excluded from build/runtime. New explicit fake adapters cannot invoke them. |
| `agente/` | Inspected, untouched, excluded from build and Compose. Factory automation remains out of scope. |
| Root `schema.sql`, `requirements.txt`, `.env.exemplo` | Historical only; local runtime uses a separate `dtf_local` PostgreSQL schema, dependencies, and `.env.example`. No migration of prototype data. |
| README/API docs | Legacy content preserved under explicit notices pointing to active local instructions. |

## Deferred and practical limits

This is a working local development milestone, not the completed three-week production
MVP. Real payments/webhooks, freight providers, Tiny/Olist, WhatsApp, production
R2, production account verification/recovery/security, automatic final print-file
generation, scheduled/offsite backups and a production restore runbook, and production
activation remain undone. Deployment definitions and automation now exist, but
their gate correctly prevents use with the local-only application. No factory agent, hot folders, FlexiPRINT, VPN,
machine dashboards, label automation, or new automatic pre-flight were added.

Browser artwork analysis already in the Site is retained as advisory prototype
behavior. The backend never adopts its prices, lengths or grades as approved.
The temporary manual quote step resolves that trust boundary locally; its role
in production needs an explicit product decision. Final files are manually
prepared and approved by operators; the test `.cdr` fixture is text, not printable
artwork. Cart recovery is browser-local, subject to quota, and does not reconstruct
the active artwork editor in place. Fake delivery receipts
mean only that the local adapter recorded an event, never that a person received
a message or an ERP order was created.

The multipart transport supports up to 5 GiB, but this local scanner can release
only files up to 128 MiB. Larger uploads remain blocked. The scanner has no
external network route and uses signatures bundled in its pinned image; rebuild
or replace that image before signatures exceed the seven-day alert threshold.

The 2026-09-15 exact-runtime Python audit reported no known dependency findings.
All 26 installed packages are transitively pinned with artifact hashes; scheduled
lock refresh and audit automation are still pending.
Trivy HIGH/CRITICAL image reports are retained in `output/security/`: API 46
(44 Debian records without fixes plus two third-party-SBOM package records absent
from the runtime), Site 5, refreshed PostgreSQL 31, pinned MinIO 108, and pinned
ClamAV 0. Counts are scanner observations, not exploitability determinations.
MinIO was not upgraded over the preserved object volume; its old pinned release
is a material localhost-only limitation. See `SECURITY_REPORT.md`.

The Site retains its pre-existing external fonts/logo/PDF.js 3.11.174 references.
The known eval advisory is mitigated by the existing `isEvalSupported:false` call;
the old CDN dependency still needs a planned upgrade or vendored/pinned
replacement. No new production service references were added. The `.git` directory is unavailable
as a working Git repository in this workspace, so changes are delivered as local
files without a commit or Git diff.

## Exact next step

The localhost security closeout is complete; continue local product work without
treating these controls as production approval. Before any staging or production
connection, complete `PRODUCTION_INPUTS.md` and pass the network-disabled readiness
gate, including the acceptance flow,
production authority for length/grade, and freight platform, origin, packaging and
policy. The verified local database/clean-object bundle addresses the local
recovery gap but is neither scheduled nor offsite and should be created while local
writes are idle. Remaining foundations include production backup/restore design,
container-image remediation/upgrade decisions, and account verification/recovery
design. The current `compose.staging.yaml` validates inputs only and cannot deploy
the application or contact providers.
Use the checked-in `deploy/` package as the target deployment contract rather
than creating another production stack. First add the actual separate staging application runtime, beginning with S3
adapter contract tests for direct multipart upload, private downloads, CORS and
retention using injected staging credentials. Add real provider adapters only
after their contracts are confirmed; payment activation requires signed,
idempotent webhooks. Implement Docker-secret file loading, resolve or formally
accept image findings, rehearse production restore, and make the release preflight
pass without weakening it. The local runtime intentionally refuses production values.