diff --git a/README.md b/README.md index 0c32fa4..57885ff 100644 --- a/README.md +++ b/README.md @@ -418,7 +418,7 @@ depende de a pasta de rede funcionar. ```bash apt install -y python3.12-venv libvips-tools postgresql nginx certbot python3 -m venv /opt/dtf/venv && source /opt/dtf/venv/bin/activate -pip install -r requirements.txt +pip install fastapi uvicorn sqlmodel 'psycopg[binary]' boto3 httpx # prototype only; no longer tracked cp .env.exemplo .env # preencher cd portal && uvicorn main:app --host 0.0.0.0 --port 8000 ``` diff --git a/ROADMAP.md b/ROADMAP.md index 6fcd7d7..2e132a7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -7,8 +7,9 @@ > Update the **Current step** line and the item status every time something moves. > Add new findings at the bottom of the relevant block rather than rewriting history. -**Current step:** Block 0 closed, plus 2.1–2.6 and 5.1. Next: 2.7 (vendor or -integrity-pin pdf.js), then 2.8–2.11. Block 1 still waits on client inputs for +**Current step:** Block 0 closed; Block 2 closed except 2.9–2.11; 4.1, 4.2, 4.5, +5.1, 5.3, 5.4 and 5.8 done. Remaining work needs decisions (3.1, 3.2, 3.3) or +client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for 1.1/1.2. **Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`, @@ -379,13 +380,16 @@ charges. Fix as part of 1.1. ## Block 4 · Scale and performance -- `[ ]` 4.1 — Missing indexes `(F23)`. `local/schema.sql` indexes only - `uploads(owner)`. Add `orders(owner)`, `quotes(owner)`, `order_files(order_id)`, - `movements(order_id)`, and a partial index on - `outbox(available_at) WHERE delivered_at IS NULL` — the worker polls that table - every second and it only grows. -- `[ ]` 4.2 — `/api/operator/board` is unpaginated `(F24)`: every order ever, plus a - per-quote subquery each. Fine at 10 orders, not at 200/day. +- `[x]` 4.1 — Ten indexes added, each matched to a query the application issues, + and no more: every extra index is paid for on each write. The outbox and live + uploads use partial indexes so they stay the size of the backlog rather than of + all history. Verified against the running database — the planner chooses + `outbox_pending` and `orders_owner` for the queries they exist for. +- `[x]` 4.2 — The board returns every order still in progress, however old, plus a + window of recent finished ones (`BOARD_FINISHED_LIMIT`, default 50) and the true + finished total. An operator can never lose a card they could act on; only terminal + ones are trimmed. The Kanban column reads "Finalizado · 50 de 213" when truncated, + so the count is not mistaken for an all-time total. Pending quotes are capped too. - `[ ]` 4.3 — Scan throughput `(F21)`: one `scan_loop` thread, `worker` at `replicas: 1`, ClamAV `MaxThreads 2`, browser gives up after 150s. - `[ ]` 4.4 — Quality grade fallback `(F22)`: when `carregarImagem` fails, @@ -410,11 +414,15 @@ charges. Fix as part of 1.1. `agente/`, root `schema.sql` (~1,500 lines describing an abandoned model). Several expose unauthenticated endpoints taking the acting user from the request body (`/api/puxar`, `/api/devolver`). -- `[ ]` 5.3 — Root `requirements.txt` is the prototype's `(F31)`: wildcard pins, - unused `sqlmodel`/`pyvips`/`qrcode`/`pillow`, next to the hash-locked - `local/requirements.lock`. -- `[ ]` 5.4 — `pip==26.2.1` pinned as a runtime dependency `(F32)` — pip ships inside - the read-only production image. +- `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed + packages the system does not use, and sat next to the hash-locked + `local/requirements.lock` inviting the wrong one to be installed. Only historical + documentation referred to it. +- `[x]` 5.4 — `pip` removed from `local/requirements.txt` and from the lock. Nothing + depended on it; it was pinned only because it was listed directly, and installing + it put a package manager inside the read-only runtime image. The base image's own + pip performs the hash-enforced install. Verified: the image builds under + `--require-hashes` and reports the base pip, 25.0.1. - `[ ]` 5.5 — Doc drift `(F34)`. `README.md`, `CONTEXT.md`, `LOCAL_SETUP.md` and `SECURITY_REPORT.md` describe a MinIO localhost stack, an API with "no external network route", a `operator` / `local-operator-only` login the email-validated @@ -437,11 +445,11 @@ charges. Fix as part of 1.1. runner has Chrome, so an intermittent failure there blocks releases for no reason. Suspect Chrome startup timing or a race against stack readiness. Watch it, and if it recurs add an explicit readiness wait rather than a retry. -- `[ ]` 5.8 — The Site promises retention the system does not honour. The cart aside - still reads *"O arquivo fica guardado por 90 dias e o histórico do pedido por 12 - meses"*, while `CONTEXT.md` and the implemented retention are 30 days maximum. - This is a customer-facing commercial promise, so correct the copy or the policy — - do not leave them disagreeing. Found 2026-09-18 while closing Block 0. +- `[x]` 5.8 — The Site claimed 90-day file storage and 12-month history in two + places, and invited customers to reorder "sem subir de novo". Files are kept 30 + days. The copy now states 30 days, says a later order needs the file again, and + keeps only the true part: order history remains in the account. Policy unchanged; + the promise was corrected to match it. --- diff --git a/dtf-site.html b/dtf-site.html index 77563b4..10836d0 100644 --- a/dtf-site.html +++ b/dtf-site.html @@ -1243,8 +1243,9 @@ footer a:hover{color:var(--laranja2)} -

A nota fiscal sai no CNPJ informado. O arquivo fica guardado por 90 dias e o histórico - do pedido por 12 meses, para você repetir sem subir de novo.

+

A nota fiscal sai no CNPJ informado. O arquivo fica guardado por 30 dias; + depois disso, um novo pedido precisa do arquivo de novo. O histórico do pedido + continua disponível na sua conta.

@@ -1336,7 +1337,7 @@ footer a:hover{color:var(--laranja2)}
  • Sem cadastro para subir e ver o preço
  • Envio em até 24 horas
  • WhatsApp a cada etapa do pedido
  • -
  • Arquivo guardado 90 dias · histórico 12 meses
  • +
  • Arquivo guardado 30 dias · histórico do pedido na conta
  • diff --git a/local/app.py b/local/app.py index 3059513..13009a4 100644 --- a/local/app.py +++ b/local/app.py @@ -34,6 +34,11 @@ ALLOWED_ORIGINS = [origin for origin in os.environ.get('ALLOWED_ORIGINS', PUBLIC # offices and mobile carriers put many real customers behind one address, so a # tight per-IP ceiling would lock out the same people the old global one did. GUEST_SESSION_LIMIT = int(os.environ.get('GUEST_SESSION_LIMIT', '240')) +# The board returned every order ever created. Finished ones are terminal, so +# they were pure growth: at a few hundred a day the response and the page both +# degrade with no operator benefit. +BOARD_FINISHED_LIMIT = int(os.environ.get('BOARD_FINISHED_LIMIT', '50')) +BOARD_QUOTE_LIMIT = int(os.environ.get('BOARD_QUOTE_LIMIT', '100')) PART_BYTES = int(os.environ.get('UPLOAD_PART_BYTES', '8388608')) if not 5242880 <= PART_BYTES <= 67108864: raise RuntimeError('UPLOAD_PART_BYTES must be between 5 and 64 MiB') @@ -284,9 +289,19 @@ def dev_paid(body: Pay, session_id=Depends(owner)): @app.get('/api/operator/board') def board(user=Depends(operator)): with db.connect() as c: - orders = c.execute('SELECT * FROM dtf_local.orders ORDER BY created_at').fetchall() - quotes = c.execute('SELECT q.* FROM dtf_local.quotes q LEFT JOIN dtf_local.orders o ON o.quote_id=q.id WHERE o.id IS NULL ORDER BY q.created_at').fetchall() - return {'states': STATES, 'transitions': TRANSITIONS, 'orders': orders, + # Everything still in progress, however old: an operator must never lose a + # card they can act on. Finished orders are terminal and only accumulate, + # so the board carries a recent window of them and reports the true total. + active = c.execute("SELECT * FROM dtf_local.orders WHERE state<>'fin' ORDER BY created_at").fetchall() + finished = c.execute("SELECT * FROM dtf_local.orders WHERE state='fin' ORDER BY created_at DESC LIMIT %s", + (BOARD_FINISHED_LIMIT,)).fetchall() + finished_total = c.execute("SELECT count(*) AS n FROM dtf_local.orders WHERE state='fin'").fetchone()['n'] + quotes = c.execute('''SELECT q.* FROM dtf_local.quotes q + LEFT JOIN dtf_local.orders o ON o.quote_id=q.id WHERE o.id IS NULL + ORDER BY q.created_at LIMIT %s''', (BOARD_QUOTE_LIMIT,)).fetchall() + return {'states': STATES, 'transitions': TRANSITIONS, + 'orders': active + list(reversed(finished)), + 'finished_shown': len(finished), 'finished_total': finished_total, 'quotes': [quote_view(c, q) for q in quotes], 'events': c.execute('SELECT * FROM dtf_local.outbox ORDER BY id DESC LIMIT 100').fetchall()} diff --git a/local/requirements.lock b/local/requirements.lock index 76007d0..9c2d3ad 100644 --- a/local/requirements.lock +++ b/local/requirements.lock @@ -310,7 +310,3 @@ uvicorn==0.34.2 \ # via -r local/requirements.txt # The following packages are considered to be unsafe in a requirements file: -pip==26.2.1 \ - --hash=sha256:71138adf1f4ca900cdb7d289c21b7494329f2332b6d85f0e1c42108c0384ed3e \ - --hash=sha256:f6ad667e89a1fe78046c8f13232b247200f5258d7828f3f7883d660878e0813f - # via -r local/requirements.txt diff --git a/local/requirements.txt b/local/requirements.txt index 9cdac78..4cb2adf 100644 --- a/local/requirements.txt +++ b/local/requirements.txt @@ -1,6 +1,5 @@ fastapi==0.141.1 starlette==1.6.0 -pip==26.2.1 uvicorn==0.34.2 psycopg[binary]==3.2.9 boto3==1.38.23 diff --git a/local/schema.sql b/local/schema.sql index 7f46b34..6ef21c3 100644 --- a/local/schema.sql +++ b/local/schema.sql @@ -69,6 +69,39 @@ CREATE TABLE IF NOT EXISTS dtf_local.security_events ( event text NOT NULL, details jsonb NOT NULL, created_at timestamptz NOT NULL DEFAULT now() ); CREATE INDEX IF NOT EXISTS uploads_owner ON dtf_local.uploads(owner); + +-- Indexes follow the queries the application actually issues. Only these; every +-- extra index is paid for on each write. + +-- The customer portal lists a person's orders and open quotes, newest first. +CREATE INDEX IF NOT EXISTS orders_owner ON dtf_local.orders(owner, created_at DESC); +CREATE INDEX IF NOT EXISTS quotes_owner ON dtf_local.quotes(owner, created_at DESC); + +-- Order detail and the movement history read by order. +CREATE INDEX IF NOT EXISTS movements_order ON dtf_local.movements(order_id, id); +CREATE INDEX IF NOT EXISTS order_files_order ON dtf_local.order_files(order_id); +-- submit_files checks whether an upload is already attached, by upload. +CREATE INDEX IF NOT EXISTS order_files_upload ON dtf_local.order_files(upload_id); + +-- The worker polls this once a second, and the table only grows. Partial, so the +-- index stays the size of the backlog rather than of all history. +CREATE INDEX IF NOT EXISTS outbox_pending ON dtf_local.outbox(available_at, id) + WHERE delivered_at IS NULL; + +-- The scanner and the retention sweep both walk live uploads in arrival order. +-- now() cannot appear in a partial index predicate, so the time comparisons stay +-- in the query and the index narrows to rows still worth looking at. +CREATE INDEX IF NOT EXISTS uploads_live ON dtf_local.uploads(created_at) + WHERE purged_at IS NULL; + +-- Sign-in transfers and logout delete a person's sessions by owner. +CREATE INDEX IF NOT EXISTS sessions_owner ON dtf_local.sessions(owner); + +-- Disabling an operator revokes their open sessions by name. +CREATE INDEX IF NOT EXISTS operator_sessions_username ON dtf_local.operator_sessions(username); + +-- security_status reads recent events; the worker prunes old ones by age. +CREATE INDEX IF NOT EXISTS security_events_created ON dtf_local.security_events(created_at); ALTER TABLE dtf_local.uploads ADD COLUMN IF NOT EXISTS expires_at timestamptz; ALTER TABLE dtf_local.uploads ADD COLUMN IF NOT EXISTS purged_at timestamptz; ALTER TABLE dtf_local.uploads ADD COLUMN IF NOT EXISTS scan_state text NOT NULL DEFAULT 'pending'; diff --git a/local/static/kanban.js b/local/static/kanban.js index cde8e17..e1340c2 100644 --- a/local/static/kanban.js +++ b/local/static/kanban.js @@ -51,7 +51,12 @@ function render(){ const colors=['#00b8da','#e0357c','#efb700','#edebe6','#e0642a','#48b072']; Object.entries(board.states).forEach(([state,title],index)=>{ const column=node('section',undefined,'col');column.dataset.state=state;column.style.setProperty('--cc',colors[index]); - const orders=board.orders.filter(o=>o.state===state);column.append(node('h2',title+' · '+orders.length)); + const orders=board.orders.filter(o=>o.state===state); + // Finished orders are a recent window, not the whole history: say so rather + // than let the count read as an all-time total. + const count=state==='fin'&&board.finished_total>orders.length + ? orders.length+' de '+board.finished_total : String(orders.length); + column.append(node('h2',title+' · '+count)); for(const order of orders){ const card=node('article',undefined,'cd');card.draggable=true;card.dataset.order=order.id; card.append(node('b','#'+order.number+' · Pago local'),node('p',order.snapshot.customer.mail,'meta'),node('p',money(order.snapshot.total_cents))); diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 329e014..0000000 --- a/requirements.txt +++ /dev/null @@ -1,10 +0,0 @@ -fastapi==0.115.* -uvicorn[standard]==0.32.* -sqlmodel==0.0.22 -psycopg[binary]==3.2.* -boto3==1.35.* -httpx==0.27.* -pyvips==2.2.* -qrcode==7.4.* -pillow==10.4.* -python-multipart==0.0.12