blyzer/dtf-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: harden week-two ordering, artwork and operations

Browse Source
This commit is contained in:
Cauê Faleiros
2026-09-23 10:40:18 -03:00
parent ccc25a2d5d
commit 24013458c9
43 changed files with 1064 additions and 228 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/LOCAL_SETUP.md
View File

@@ -161,8 +161,8 @@ test is unmistakable:
```bash
python3 -m tests.security_test
python3 -m tests.scanning_test
docker compose exec -T api python3 -m tests.runtime_security_test
docker compose exec -T api python3 -m tests.retention_test
docker compose -f compose.local.yaml exec -T api python3 -m tests.runtime_security_test
docker compose -f compose.local.yaml exec -T api python3 -m tests.retention_test
```
`local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV
@@ -183,14 +183,15 @@ test order for inspection. Both integration scripts read `.env` automatically.
## Configuration and storage
`.env.example` lists local ports, database/MinIO values, operator login, adapter
selection, mock freight amount, maximum file size (5 GiB), and multipart size
(8 MiB by default). The malware scanner releases only files up to 128 MiB by
default (`SCAN_MAX_BYTES`); larger uploads remain blocked even though the
multipart transport supports 5 GiB. `S3_ENDPOINT=http://storage:9000`, database
selection, mock freight amount, transport ceiling (5 GiB), and multipart size
(8 MiB by default). The API and customer picker admit only files within the
malware scanner's effective 128 MiB limit by default (`SCAN_MAX_BYTES`); the
larger-file path remains a Week 2 decision. `S3_ENDPOINT=http://storage:9000`, database
hostname `db`,
and the internal service ports are fixed Compose wiring. The public S3 endpoint
must resolve from the browser; keep `http://localhost:9000` for this stack.
Parts use 15-minute presigned URLs and uploads must finish within one day.
Parts use 15-minute presigned URLs and unfinished reservations expire after
one hour. Clients can cancel a reservation through the upload DELETE endpoint.
Only fake integration adapters and `s3-local` storage are accepted. Startup fails
if a production adapter/environment or nonlocal S3 endpoint is selected.
@@ -199,7 +200,8 @@ and MinIO also join a network that permits loopback port publishing.
Objects are private, use UUID keys rather than filenames, and persist in a named
volume. MinIO lifecycle rules expire objects after 30 days and abandon incomplete
multipart uploads after one day; the API also blocks expired downloads. Order
multipart uploads after one day as a backstop; the API lease and worker release
unfinished reservations after one hour. The API also blocks expired downloads. Order
history remains in PostgreSQL. Completed files start in `pending`; unknown,
scanner-error, over-limit, encrypted/unsafe, and malware results fail closed.
Only `clean` files can cross quote, payment, download, final-approval, and queue
@@ -217,7 +219,7 @@ again. The operator can still inspect order records.
## Local backup and restore check
```bash
python3 -m app.backup create-and-verify
python3 -m ops.backup create-and-verify
```
This writes a private four-file bundle in `backups/` (ignored by Git and Docker
@@ -234,7 +236,7 @@ checks database counts, bundle hashes, every archived object's hash, and the byt
downloaded after restore, then removes only the temporary database and objects. It
never restores over active data. Keep every bundle file private: it contains
customer data, password hashes, and customer artwork. To verify it again, run
`python3 -m app.backup verify` followed by the printed
`python3 -m ops.backup verify` followed by the printed
`backups/...manifest.json` path. Legacy database-only `.dump` backups remain
verifiable. Scheduling, offsite copies, and a production restore runbook remain
unfinished.
@@ -242,7 +244,7 @@ unfinished.
After building the current image, test retention with synthetic files:
```bash
docker compose exec -T api python3 -m tests.retention_test
docker compose -f compose.local.yaml exec -T api python3 -m tests.retention_test
```
This checks that expired bytes are removed while unexpired files survive. It
@@ -251,14 +253,14 @@ cleans up its own synthetic object bytes and retains their metadata.
## Operations and troubleshooting
```bash
docker compose logs --tail=100 api worker scanner
docker compose restart api worker
docker compose ps
docker compose down
docker compose -f compose.local.yaml logs --tail=100 api worker scanner
docker compose -f compose.local.yaml restart api worker
docker compose -f compose.local.yaml ps
docker compose -f compose.local.yaml down
```
`down` stops the stack and preserves named database/storage volumes. Restart
with `docker compose up -d --wait`. Do not add `--volumes` unless you intend to
with `docker compose -f compose.local.yaml up -d --wait`. Do not add `--volumes` unless you intend to
permanently erase all local orders and artwork. No reset is required for tests.
Seven long-running services have Docker health checks; the database and storage
@@ -270,7 +272,7 @@ exposes `/minio/health/ready`.
Run the redacted local alert summary inside the API network namespace:
```bash
docker compose exec -T api python3 -m app.security_status
docker compose -f compose.local.yaml exec -T api python3 -m ops.security_status
```
Exit status 1 means attention is required. Review blocked artwork, rate limits,
@@ -312,7 +314,7 @@ a direct pin, regenerate the lock in the same Python 3.12 environment and rebuil
```bash
./infra/lock_dependencies.sh
docker compose up --build -d --wait
docker compose -f compose.local.yaml up --build -d --wait
```
The generator downloads public package metadata in a disposable container and