feat: accept sheets of up to 5 GB end to end
Some checks failed
Build and deploy / Validate source (push) Successful in 12s
Build and deploy / Integration suite on a real stack (push) Failing after 2m36s
Build and deploy / Secret scan and release gate (push) Successful in 7s
Build and deploy / Publish images (push) Has been skipped

Sheets of several GB are the normal order. The upload limit is now 5 GB.
ClamAV scans files up to 2 GB; a larger file is released only when its
first bytes match the format its name claims, and a disguised file is
refused. The Site grades a sheet over 150 MB from the pixel size in its
PNG, JPEG or WebP header without decoding it, and reads large PDFs in
ranges. The worker never opens a source over 300 MB: a finished sheet
placed whole becomes its own print file, which the Kanban offers to approve
as the final, and anything else goes to hand preparation. Files start
uploading as they enter the cart, with progress in the summary, and each
part renews the reservation so slow uploads do not expire. Quotas grow to
50 GB per customer and 500 GB in total; the Swarm config for ClamAV is
renamed because a deployed config cannot change in place.

Verified locally with a 386 MB and a 1.8 GB PNG (scanned, paid, original
as print file), a 2.3 GB PNG (format check) and a disguised 2.3 GB file
(refused).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cauê Faleiros
2026-09-29 13:18:18 -03:00
parent 4437232d27
commit 5f2be7ea20
21 changed files with 329 additions and 54 deletions

View File

@@ -627,7 +627,7 @@ the site already did.
generator, with the browser as preview only. This is the single largest gap between
what was promised in the meeting and what exists.
### `[?]` 3.3 — The 5 GB problem is unsolved `(F19)`
### `[~]` 3.3 — The 5 GB problem is unsolved `(F19)`
Transport accepts 5 GiB; `SCAN_MAX_BYTES` / ClamAV `StreamMaxLength` release only
≤ 128 MiB. As of 2026-09-23, customer selection and API reservation reject files
@@ -638,6 +638,23 @@ This is exactly the risk Jorge raised in the meeting.
**Decide:** raise the scan ceiling with a resource/timeout design, or define an
explicit large-file path (staged scan, sampled scan, operator override with audit).
**Built 2026-09-29 (sheets of several GB are the normal order, not the
exception):** files up to 5 GB. ClamAV scans up to 2 GB (`StreamMaxLength
2000M`); above that a file is released only when its first bytes match the
format its name claims (option A, the user's choice). The Site grades a sheet
over 150 MB from the pixel size in the PNG/JPEG/WebP header without decoding
it, and measures large PDFs through ranged reads; the worker never opens a
source over 300 MB: a single finished sheet placed whole becomes its own print
file, anything else goes to hand preparation. Uploads start as items enter the
cart and the lease renews with each part. Verified on the local stack: 386 MB
(ClamAV 78 s), 1.8 GB (ClamAV 6 min 18 s, scanner under 430 MB of memory, the
original as print file), 2.3 GB PNG released by the format check and a
disguised 2.3 GB file refused, and the header grade of a 200 MB file in 0.5 s.
**Open:** large PDFs get no automatic grade (the DPI of images inside is not
read without rendering); the scanner takes one file at a time, so several
multi-GB uploads queue; pieces, residue and background of a large sheet are
not checked automatically.
### `[ ]` 3.4 — Upload throughput `(F20)`
8 MiB parts, strictly sequential in `local/static/upload.js:21`, one presign