refactor: lay the repository out by role
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Integration suite on a real stack (push) Successful in 1m25s
Build and deploy / Secret scan and release gate (push) Successful in 5s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m31s

local/ held six unrelated things under a name that stopped being true once it
became the production runtime: the service, the frontend, the tests, the ops
commands, the container definitions and the dependency lock, 65 files with
nothing to tell them apart.

  app/      the service: api/ routers, core/ for identity, database, models,
            prices and secret loading, and the worker, bootstrap and schema
  tests/    the twelve suites, no longer inside the shipped package
  ops/      backup, readiness, dependency audit, security summary
  infra/    Dockerfiles, gateway templates, ClamAV and storage configuration,
            the requirements and their hash lock
  web/      the Site, Kanban and portal pages with their scripts

deploy/Dockerfile.api now copies app/ alone, so the tests stop shipping to
production; the local image still carries them, because the suites run inside
the stack's network.

Five kinds of reference had to follow, and each was found by something different
rather than by reading. Imports of the form "from . import db" survived a rewrite
that only matched "from .db import". Tests kept relative imports of modules that
had left the package. A mock.patch target names its module in a string, where no
import rewriting can see it. The browser test resolves a fixture by path. And the
release gate's markers pointed at local/runtime.py and local/worker.py, which is
the decay its new marker test exists to catch — it caught it.

Verified from docker compose down -v: the stack starts, all six integration
suites, both browser suites and the twenty-nine unit tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Cauê Faleiros
2026-09-21 17:40:57 -03:00
parent c9f8122600
commit b329f76378
69 changed files with 146 additions and 139 deletions

View File

@@ -16,14 +16,14 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Run fast regression checks - name: Run fast regression checks
run: | run: |
python3 -m py_compile local/*.py deploy/*.py python3 -m py_compile app/*.py app/**/*.py ops/*.py deploy/*.py
python3 -m unittest \ python3 -m unittest \
local.test_dependency_lock \ tests.test_dependency_lock \
local.test_staging_readiness \ tests.test_staging_readiness \
deploy.test_production_preflight \ deploy.test_production_preflight \
local.test_pricing \ tests.test_pricing \
local.test_secrets -v tests.test_secrets -v
sh -n local/lock_dependencies.sh sh -n infra/lock_dependencies.sh
integration: integration:
name: Integration suite on a real stack name: Integration suite on a real stack
@@ -75,13 +75,13 @@ jobs:
$COMPOSE exec -T \ $COMPOSE exec -T \
-e SITE_BASE_URL=http://site \ -e SITE_BASE_URL=http://site \
-e SITE_HOST_HEADER=localhost \ -e SITE_HOST_HEADER=localhost \
api python -m "local.$suite" api python -m "tests.$suite"
done done
- name: Runtime and retention regressions - name: Runtime and retention regressions
run: | run: |
$COMPOSE exec -T api python -m local.retention_test $COMPOSE exec -T api python -m tests.retention_test
$COMPOSE exec -T api python -m local.runtime_security_test $COMPOSE exec -T api python -m tests.runtime_security_test
# These need a real Chrome. They are the only coverage for the artwork # These need a real Chrome. They are the only coverage for the artwork
# editor and the full customer journey, so install google-chrome-stable # editor and the full customer journey, so install google-chrome-stable
@@ -110,8 +110,8 @@ jobs:
exit 0 exit 0
fi fi
echo "Using $CHROME_BIN" echo "Using $CHROME_BIN"
node local/artwork_browser_test.mjs node tests/artwork_browser_test.mjs
node local/browser_test.mjs node tests/browser_test.mjs
- name: Diagnostics on failure - name: Diagnostics on failure
if: failure() if: failure()

View File

@@ -249,7 +249,7 @@ supports subsequent corrections or scoped enhancements.
The current implementation target is localhost before any production connection. The current implementation target is localhost before any production connection.
Use `docker-compose.yml`, `.env.example`, and `LOCAL_SETUP.md`. The runtime is in Use `docker-compose.yml`, `.env.example`, and `LOCAL_SETUP.md`. The runtime is in
`local/`; historical `portal/`, `kanban/`, `agente/`, and `schema.sql` are preserved `app/`; historical `portal/`, `kanban/`, `agente/`, and `schema.sql` are preserved
as references and are not imported or started by Compose. as references and are not imported or started by Compose.
- One local `dtf-cloud` Compose project runs seven long-lived services: Site and - One local `dtf-cloud` Compose project runs seven long-lived services: Site and
@@ -308,10 +308,10 @@ as references and are not imported or started by Compose.
`security_status` command summarizes authentication, rate-limit, scan, and `security_status` command summarizes authentication, rate-limit, scan, and
scanner/signature alerts without exposing secrets. Security regression tests, scanner/signature alerts without exposing secrets. Security regression tests,
exact-runtime Python dependency auditing, and Trivy image reports live under exact-runtime Python dependency auditing, and Trivy image reports live under
`local/` and `output/security/`; open findings are documented in `app/` and `output/security/`; open findings are documented in
`SECURITY_REPORT.md` and are not a production-readiness claim. `SECURITY_REPORT.md` and are not a production-readiness claim.
- All direct and transitive Python packages are pinned with artifact hashes in - All direct and transitive Python packages are pinned with artifact hashes in
`local/requirements.lock`; the API image build requires those hashes. The lock `infra/requirements.lock`; the API image build requires those hashes. The lock
is regenerated in a disposable Python 3.12 container by is regenerated in a disposable Python 3.12 container by
`local/lock_dependencies.sh`. A fresh exact-runtime audit found no known Python `local/lock_dependencies.sh`. A fresh exact-runtime audit found no known Python
advisories on 2026-09-15; this does not cover OS/container findings. advisories on 2026-09-15; this does not cover OS/container findings.
@@ -327,7 +327,7 @@ as references and are not imported or started by Compose.
not been deployed. Its fail-closed preflight intentionally rejects the current not been deployed. Its fail-closed preflight intentionally rejects the current
source until production adapters, Docker-secret file loading, approved inputs, source until production adapters, Docker-secret file loading, approved inputs,
restore rehearsal, image scans, and human security approval are complete. restore rehearsal, image scans, and human security approval are complete.
- `python3 -m local.backup create-and-verify` creates a private, Git-ignored bundle - `python3 -m app.backup create-and-verify` creates a private, Git-ignored bundle
containing a PostgreSQL dump plus every complete, unexpired object already marked containing a PostgreSQL dump plus every complete, unexpired object already marked
`clean`. SHA-256 manifests protect both parts. Verification restores the database `clean`. SHA-256 manifests protect both parts. Verification restores the database
under a UUID name and the object bytes under a UUID MinIO prefix, hashes the under a UUID name and the object bytes under a UUID MinIO prefix, hashes the

View File

@@ -1,6 +1,6 @@
# DTF local development # DTF local development
Read `CONTEXT.md` first. The current runtime lives in `local/`; older portal, Read `CONTEXT.md` first. The service lives in `app/`; older portal,
Kanban, agent, requirements, and SQL files are historical prototypes. Kanban, agent, requirements, and SQL files are historical prototypes.
## Start ## Start
@@ -51,7 +51,7 @@ operator interfaces.
## Browser test order ## Browser test order
1. Open the Site. Choose **Arquivo por metro** and the manual/table-price path 1. Open the Site. Choose **Arquivo por metro** and the manual/table-price path
(CDR/AI/PSD/TIFF). Select `local/fixtures/local-test.cdr`. This is deliberately (CDR/AI/PSD/TIFF). Select `tests/fixtures/local-test.cdr`. This is deliberately
harmless text for upload testing, not a printable CDR file. harmless text for upload testing, not a printable CDR file.
2. Enter **1.01 metres**. The original calculation bills **1.10 m × R$19.90 = 2. Enter **1.01 metres**. The original calculation bills **1.10 m × R$19.90 =
R$21.89**, with grade 0 and pickup. You can also use your own non-sensitive R$21.89**, with grade 0 and pickup. You can also use your own non-sensitive
@@ -142,8 +142,8 @@ pricing, plus assembly and invalid-input checks.
With the stack healthy, run the integration test (Python standard library only): With the stack healthy, run the integration test (Python standard library only):
```bash ```bash
python3 -m local.smoke_test python3 -m tests.smoke_test
python3 -m local.workflow_test python3 -m tests.workflow_test
``` ```
It checks direct two-part upload/resume, missing parts, session isolation, It checks direct two-part upload/resume, missing parts, session isolation,
@@ -159,10 +159,10 @@ Security and malware regressions are separate so an authorized harmless EICAR
test is unmistakable: test is unmistakable:
```bash ```bash
python3 -m local.security_test python3 -m tests.security_test
python3 -m local.scanning_test python3 -m tests.scanning_test
docker compose exec -T api python -m local.runtime_security_test docker compose exec -T api python3 -m tests.runtime_security_test
docker compose exec -T api python -m local.retention_test docker compose exec -T api python3 -m tests.retention_test
``` ```
`local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV `local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV
@@ -172,7 +172,7 @@ is retained for at most three days, so it temporarily appears in alert summaries
For an automated real-browser walkthrough, install Chrome and use Node 22+: For an automated real-browser walkthrough, install Chrome and use Node 22+:
```bash ```bash
node local/browser_test.mjs node tests/browser_test.mjs
``` ```
Set `CHROME_BIN` if Chrome is not at `/usr/bin/google-chrome-stable`. The test Set `CHROME_BIN` if Chrome is not at `/usr/bin/google-chrome-stable`. The test
@@ -217,7 +217,7 @@ again. The operator can still inspect order records.
## Local backup and restore check ## Local backup and restore check
```bash ```bash
python3 -m local.backup create-and-verify python3 -m app.backup create-and-verify
``` ```
This writes a private four-file bundle in `backups/` (ignored by Git and Docker This writes a private four-file bundle in `backups/` (ignored by Git and Docker
@@ -234,7 +234,7 @@ checks database counts, bundle hashes, every archived object's hash, and the byt
downloaded after restore, then removes only the temporary database and objects. It downloaded after restore, then removes only the temporary database and objects. It
never restores over active data. Keep every bundle file private: it contains never restores over active data. Keep every bundle file private: it contains
customer data, password hashes, and customer artwork. To verify it again, run customer data, password hashes, and customer artwork. To verify it again, run
`python3 -m local.backup verify` followed by the printed `python3 -m app.backup verify` followed by the printed
`backups/...manifest.json` path. Legacy database-only `.dump` backups remain `backups/...manifest.json` path. Legacy database-only `.dump` backups remain
verifiable. Scheduling, offsite copies, and a production restore runbook remain verifiable. Scheduling, offsite copies, and a production restore runbook remain
unfinished. unfinished.
@@ -242,7 +242,7 @@ unfinished.
After building the current image, test retention with synthetic files: After building the current image, test retention with synthetic files:
```bash ```bash
docker compose exec -T api python -m local.retention_test docker compose exec -T api python3 -m tests.retention_test
``` ```
This checks that expired bytes are removed while unexpired files survive. It This checks that expired bytes are removed while unexpired files survive. It
@@ -270,7 +270,7 @@ exposes `/minio/health/ready`.
Run the redacted local alert summary inside the API network namespace: Run the redacted local alert summary inside the API network namespace:
```bash ```bash
docker compose exec -T api python -m local.security_status docker compose exec -T api python3 -m app.security_status
``` ```
Exit status 1 means attention is required. Review blocked artwork, rate limits, Exit status 1 means attention is required. Review blocked artwork, rate limits,
@@ -306,12 +306,12 @@ Offline PDF previews may fall back to the prototype's manual/table-price path.
## Dependency lock ## Dependency lock
The API, worker, and database initializer install every Python dependency from The API, worker, and database initializer install every Python dependency from
`local/requirements.lock` with `--require-hashes`. `local/requirements.txt` `infra/requirements.lock` with `--require-hashes`. `infra/requirements.txt`
remains the human-maintained direct dependency list. After deliberately changing remains the human-maintained direct dependency list. After deliberately changing
a direct pin, regenerate the lock in the same Python 3.12 environment and rebuild: a direct pin, regenerate the lock in the same Python 3.12 environment and rebuild:
```bash ```bash
./local/lock_dependencies.sh ./infra/lock_dependencies.sh
docker compose up --build -d --wait docker compose up --build -d --wait
``` ```
@@ -339,7 +339,7 @@ contracts and owners. See `staging/README.md`.
Keep this stack local. Before connecting real services, confirm the production Keep this stack local. Before connecting real services, confirm the production
checkout trust workflow, complete `PRODUCTION_INPUTS.md`, and pass the isolated checkout trust workflow, complete `PRODUCTION_INPUTS.md`, and pass the isolated
staging-readiness gate. Then implement the actual staging composition using the adapter contracts in staging-readiness gate. Then implement the actual staging composition using the adapter contracts in
`local/adapters.py`: start with private S3 staging storage, CORS, signed multipart `app/adapters.py`: start with private S3 staging storage, CORS, signed multipart
contract tests and scoped credentials injected outside Git. Add provider sandbox contract tests and scoped credentials injected outside Git. Add provider sandbox
adapters one at a time. Mercado Pago requires authenticated, signed, idempotent adapters one at a time. Mercado Pago requires authenticated, signed, idempotent
webhook handling before real payment is allowed; Tiny/Olist and WhatsApp need webhook handling before real payment is allowed; Tiny/Olist and WhatsApp need
@@ -363,7 +363,7 @@ Run the source-only gate without credentials:
python3 deploy/production_preflight.py --source-only python3 deploy/production_preflight.py --source-only
``` ```
It must remain blocked while `local/` supports only local fake adapters and does It must remain blocked while `app/` supports only local fake adapters and does
not load Docker secret `*_FILE` settings. Do not bypass or delete this check. not load Docker secret `*_FILE` settings. Do not bypass or delete this check.
After approved production implementations and inputs exist, follow After approved production implementations and inputs exist, follow
`PORTAINER.md` and `deploy/PRODUCTION_CHECKLIST.md`; release and deployment `PORTAINER.md` and `deploy/PRODUCTION_CHECKLIST.md`; release and deployment

View File

@@ -12,7 +12,7 @@
client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
1.1/1.2. 1.1/1.2.
**Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`, **Last audit:** 2026-09-18, full read of `app/`, `dtf-site.html`, `deploy/`,
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs. `.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
| Status | Meaning | | Status | Meaning |
@@ -279,7 +279,7 @@ deliberately not bundled here.
Accounts now live in `dtf_local.operators`, one per person, with `movements.operator` Accounts now live in `dtf_local.operators`, one per person, with `movements.operator`
and `order_files.created_by` recording who actually acted. Administered from the API and `order_files.created_by` recording who actually acted. Administered from the API
container with `python -m local.operators` (list, add, password, disable, enable); container with `python3 -m app.operators` (list, add, password, disable, enable);
passwords are read from the terminal so they never reach shell history or the passwords are read from the terminal so they never reach shell history or the
process list, and disabling revokes open sessions immediately rather than leaving process list, and disabling revokes open sessions immediately rather than leaving
them valid for the rest of the eight-hour window. them valid for the rest of the eight-hour window.
@@ -417,9 +417,9 @@ charges. Fix as part of 1.1.
with a header saying they are background, not instructions. with a header saying they are background, not instructions.
- `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed - `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed
packages the system does not use, and sat next to the hash-locked packages the system does not use, and sat next to the hash-locked
`local/requirements.lock` inviting the wrong one to be installed. Only historical `infra/requirements.lock` inviting the wrong one to be installed. Only historical
documentation referred to it. documentation referred to it.
- `[x]` 5.4 — `pip` removed from `local/requirements.txt` and from the lock. Nothing - `[x]` 5.4 — `pip` removed from `infra/requirements.txt` and from the lock. Nothing
depended on it; it was pinned only because it was listed directly, and installing depended on it; it was pinned only because it was listed directly, and installing
it put a package manager inside the read-only runtime image. The base image's own it put a package manager inside the read-only runtime image. The base image's own
pip performs the hash-enforced install. Verified: the image builds under pip performs the hash-enforced install. Verified: the image builds under

View File

@@ -84,7 +84,7 @@ not simply increasing the upload limit.
Run: Run:
```bash ```bash
docker compose exec -T api python -m local.security_status docker compose exec -T api python3 -m app.security_status
``` ```
An exit status of 1 requires review. At closeout, attention was expected because An exit status of 1 requires review. At closeout, attention was expected because
@@ -99,7 +99,7 @@ errors, unexplained authentication bursts, or stale signatures as incidents.
`pip-audit` inspected the exact packages installed in the hash-enforced rebuilt API `pip-audit` inspected the exact packages installed in the hash-enforced rebuilt API
and found no known Python advisories on 2026-09-15. All 26 direct and transitive and found no known Python advisories on 2026-09-15. All 26 direct and transitive
runtime packages are pinned with artifact hashes in `local/requirements.lock`. runtime packages are pinned with artifact hashes in `infra/requirements.lock`.
`local/lock_dependencies.sh` regenerates it in a disposable Python 3.12 container. `local/lock_dependencies.sh` regenerates it in a disposable Python 3.12 container.
This is a point-in-time package-database result, not proof that the dependencies This is a point-in-time package-database result, not proof that the dependencies
or image are vulnerability-free. Scheduled lock refresh and audit automation remain or image are vulnerability-free. Scheduled lock refresh and audit automation remain

View File

@@ -7,10 +7,10 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException from fastapi import APIRouter, Depends, HTTPException
from .. import db from ..core import db
from ..artwork import submit_files from ..artwork import submit_files
from ..auth import operator from ..core.auth import operator
from ..models import ArtworkSubmission, UploadStart from ..core.models import ArtworkSubmission, UploadStart
from ..runtime import file_rows, operator_identity from ..runtime import file_rows, operator_identity
from .uploads import begin_upload, complete_upload, part_url, upload_status from .uploads import begin_upload, complete_upload, part_url, upload_status

View File

@@ -6,11 +6,11 @@ from fastapi import APIRouter, Depends, HTTPException, Request, Response
from psycopg.errors import UniqueViolation from psycopg.errors import UniqueViolation
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .. import db from ..core import db
from ..artwork import submit_files from ..artwork import submit_files
from ..auth import (DUMMY_PASSWORD_HASH, audit, client_ip, new_session, owner, from ..core.auth import (DUMMY_PASSWORD_HASH, audit, client_ip, new_session, owner,
password_hash, password_matches, session_row, throttle, transfer_guest) password_hash, password_matches, session_row, throttle, transfer_guest)
from ..models import ArtworkSubmission, Login, Register from ..core.models import ArtworkSubmission, Login, Register
from ..runtime import STATES, file_rows, owned_order, storage from ..runtime import STATES, file_rows, owned_order, storage
from ..scanning import require_clean from ..scanning import require_clean

View File

@@ -3,9 +3,9 @@ import os
from fastapi import APIRouter, HTTPException, Request, Response from fastapi import APIRouter, HTTPException, Request, Response
from .. import db from ..core import db
from ..auth import client_ip, owner, new_session, rate_limit from ..core.auth import client_ip, owner, new_session, rate_limit
from ..models import Freight from ..core.models import Freight
from ..runtime import ENVIRONMENT, GUEST_SESSION_LIMIT, PART_BYTES, freight, storage from ..runtime import ENVIRONMENT, GUEST_SESSION_LIMIT, PART_BYTES, freight, storage
router = APIRouter() router = APIRouter()

View File

@@ -8,11 +8,11 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Request, Response from fastapi import APIRouter, Depends, HTTPException, Request, Response
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .. import db from ..core import db
from ..auth import (COOKIE_SECURE, DUMMY_PASSWORD_HASH, audit, client_ip, operator, from ..core.auth import (COOKIE_SECURE, DUMMY_PASSWORD_HASH, audit, client_ip, operator,
password_matches, throttle) password_matches, throttle)
from ..models import Move, OperatorLogin, Review from ..core.models import Move, OperatorLogin, Review
from ..pricing import price from ..core.pricing import price
from ..runtime import (BOARD_FINISHED_LIMIT, BOARD_QUOTE_LIMIT, STATES, TRANSITIONS, from ..runtime import (BOARD_FINISHED_LIMIT, BOARD_QUOTE_LIMIT, STATES, TRANSITIONS,
enqueue, freight, quote_view, storage, upload_row) enqueue, freight, quote_view, storage, upload_row)
from ..scanning import require_clean from ..scanning import require_clean

View File

@@ -5,9 +5,9 @@ from uuid import UUID, uuid4
from fastapi import APIRouter, Depends, HTTPException from fastapi import APIRouter, Depends, HTTPException
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .. import db from ..core import db
from ..auth import owner from ..core.auth import owner
from ..models import Pay from ..core.models import Pay
from ..runtime import ENVIRONMENT, enqueue, payment, upload_row from ..runtime import ENVIRONMENT, enqueue, payment, upload_row
from ..scanning import require_clean from ..scanning import require_clean

View File

@@ -6,9 +6,9 @@ from uuid import UUID, uuid4
from fastapi import APIRouter, Depends, HTTPException from fastapi import APIRouter, Depends, HTTPException
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .. import db from ..core import db
from ..auth import owner from ..core.auth import owner
from ..models import QuoteRequest from ..core.models import QuoteRequest
from ..runtime import freight, quote_view, upload_row from ..runtime import freight, quote_view, upload_row
from ..scanning import require_clean from ..scanning import require_clean

View File

@@ -10,9 +10,9 @@ from uuid import UUID, uuid4
from botocore.exceptions import ClientError from botocore.exceptions import ClientError
from fastapi import APIRouter, Depends, HTTPException from fastapi import APIRouter, Depends, HTTPException
from .. import db from ..core import db
from ..auth import audit, owner, rate_limit from ..core.auth import audit, owner, rate_limit
from ..models import UploadStart from ..core.models import UploadStart
from ..runtime import PART_BYTES, storage, upload_row from ..runtime import PART_BYTES, storage, upload_row
router = APIRouter() router = APIRouter()

View File

@@ -10,8 +10,8 @@ from fastapi import FastAPI
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from starlette.middleware.trustedhost import TrustedHostMiddleware from starlette.middleware.trustedhost import TrustedHostMiddleware
from . import db from .core import db
from .auth import audit, client_ip from .core.auth import audit, client_ip
from .runtime import ALLOWED_HOSTS, ALLOWED_ORIGINS, storage from .runtime import ALLOWED_HOSTS, ALLOWED_ORIGINS, storage
from .api import artwork, customer, health, operator, orders, quotes, uploads from .api import artwork, customer, health, operator, orders, quotes, uploads

View File

@@ -4,7 +4,7 @@ from pathlib import Path
from urllib.parse import urlparse from urllib.parse import urlparse
import psycopg import psycopg
from psycopg import sql from psycopg import sql
from .secrets import load as load_secret_files from .core.secrets import load as load_secret_files
from .operators import seed_from_environment from .operators import seed_from_environment

1
app/core/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Shared foundations: identity, database, models, prices, secret loading."""

View File

@@ -6,7 +6,7 @@ import logging
import json import json
from uuid import UUID, uuid4 from uuid import UUID, uuid4
from fastapi import HTTPException, Request from fastapi import HTTPException, Request
from .db import connect from ..core.db import connect
COOKIE_SECURE = os.environ.get('COOKIE_SECURE', 'false').lower() == 'true' COOKIE_SECURE = os.environ.get('COOKIE_SECURE', 'false').lower() == 'true'

View File

@@ -19,4 +19,5 @@ def connect():
def initialize(): def initialize():
with connect() as c: with connect() as c:
c.execute(Path(__file__).with_name('schema.sql').read_text()) # The schema lives at the package root, one level up from core/.
c.execute((Path(__file__).resolve().parent.parent / 'schema.sql').read_text())

View File

@@ -21,8 +21,8 @@ import os
import sys import sys
from uuid import uuid4 from uuid import uuid4
from .auth import password_hash from .core.auth import password_hash
from .db import connect from .core.db import connect
MIN_PASSWORD = 12 MIN_PASSWORD = 12

View File

@@ -13,7 +13,7 @@ from fastapi import HTTPException
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .adapters import FakeFreight, FakePayment, LocalS3Storage, require_runtime from .adapters import FakeFreight, FakePayment, LocalS3Storage, require_runtime
from .secrets import load as load_secret_files from .core.secrets import load as load_secret_files
# Secret files must resolve before any configuration below is read. # Secret files must resolve before any configuration below is read.
load_secret_files() load_secret_files()

View File

@@ -4,8 +4,8 @@ import socket
import struct import struct
import time import time
from fastapi import HTTPException from fastapi import HTTPException
from .auth import audit from .core.auth import audit
from .db import connect from .core.db import connect
def require_clean(row): def require_clean(row):
if not row['complete'] or row['scan_state'] != 'clean': if not row['complete'] or row['scan_state'] != 'clean':

View File

@@ -6,8 +6,8 @@ import time
from http.server import BaseHTTPRequestHandler, HTTPServer from http.server import BaseHTTPRequestHandler, HTTPServer
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from .adapters import FakeTiny, FakeWhatsApp, LocalS3Storage, require_runtime from .adapters import FakeTiny, FakeWhatsApp, LocalS3Storage, require_runtime
from .secrets import load as load_secret_files from .core.secrets import load as load_secret_files
from .db import connect from .core.db import connect
from .scanning import ClamAV, scan_loop from .scanning import ClamAV, scan_loop
load_secret_files() load_secret_files()

View File

@@ -9,7 +9,7 @@
x-app: &app x-app: &app
build: build:
context: . context: .
dockerfile: local/Dockerfile dockerfile: infra/Dockerfile
environment: &environment environment: &environment
APP_ENV: local APP_ENV: local
DATABASE_URL: postgresql://${APP_DB_USER:-dtf_app}:${APP_DB_PASSWORD:-local-app-database-only}@db:5432/${POSTGRES_DB:-dtf_local} DATABASE_URL: postgresql://${APP_DB_USER:-dtf_app}:${APP_DB_PASSWORD:-local-app-database-only}@db:5432/${POSTGRES_DB:-dtf_local}
@@ -88,8 +88,8 @@ services:
db-init: db-init:
build: build:
context: . context: .
dockerfile: local/Dockerfile dockerfile: infra/Dockerfile
command: python -m local.bootstrap command: python -m app.bootstrap
environment: environment:
# Local only: the app role keeps a password distinct from the administrator. # Local only: the app role keeps a password distinct from the administrator.
DATABASE_ADMIN_URL: postgresql://${POSTGRES_USER:-dtf_local}:${POSTGRES_PASSWORD:-local-database-only}@db:5432/${POSTGRES_DB:-dtf_local} DATABASE_ADMIN_URL: postgresql://${POSTGRES_USER:-dtf_local}:${POSTGRES_PASSWORD:-local-database-only}@db:5432/${POSTGRES_DB:-dtf_local}
@@ -109,7 +109,7 @@ services:
storage-init: storage-init:
build: build:
context: . context: .
dockerfile: local/Dockerfile.storage-init dockerfile: infra/Dockerfile.storage-init
args: args:
MINIO_IMAGE: ${MINIO_IMAGE:-quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z} MINIO_IMAGE: ${MINIO_IMAGE:-quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z}
entrypoint: [/bin/sh, /init.sh] entrypoint: [/bin/sh, /init.sh]
@@ -128,7 +128,7 @@ services:
# Built, not bind-mounted: see local/Dockerfile.scanner. # Built, not bind-mounted: see local/Dockerfile.scanner.
build: build:
context: . context: .
dockerfile: local/Dockerfile.scanner dockerfile: infra/Dockerfile.scanner
args: args:
CLAMAV_IMAGE: ${CLAMAV_IMAGE:-clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4} CLAMAV_IMAGE: ${CLAMAV_IMAGE:-clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4}
entrypoint: [clamd, --foreground=true, --config-file=/etc/clamav/clamd.conf] entrypoint: [clamd, --foreground=true, --config-file=/etc/clamav/clamd.conf]
@@ -146,7 +146,7 @@ services:
api: api:
<<: *app <<: *app
command: uvicorn local.app:app --host 0.0.0.0 --port 8000 --no-access-log command: uvicorn app.app:app --host 0.0.0.0 --port 8000 --no-access-log
depends_on: depends_on:
db-init: {condition: service_completed_successfully} db-init: {condition: service_completed_successfully}
storage-init: {condition: service_completed_successfully} storage-init: {condition: service_completed_successfully}
@@ -158,7 +158,7 @@ services:
worker: worker:
<<: *app <<: *app
command: python -m local.worker command: python -m app.worker
depends_on: depends_on:
api: {condition: service_healthy} api: {condition: service_healthy}
scanner: {condition: service_healthy} scanner: {condition: service_healthy}
@@ -171,7 +171,7 @@ services:
site: site:
build: build:
context: . context: .
dockerfile: local/Dockerfile.web dockerfile: infra/Dockerfile.web
environment: environment:
S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}} S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}}
ports: ports:
@@ -193,7 +193,7 @@ services:
kanban: kanban:
build: build:
context: . context: .
dockerfile: local/Dockerfile.web dockerfile: infra/Dockerfile.web
environment: environment:
WEB_INDEX: kanban.html WEB_INDEX: kanban.html
S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}} S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}}

View File

@@ -7,8 +7,8 @@ services:
readiness: readiness:
build: build:
context: . context: .
dockerfile: local/Dockerfile dockerfile: infra/Dockerfile
command: python -m local.staging_readiness /config/staging.env command: python -m app.staging_readiness /config/staging.env
volumes: volumes:
- ./staging/staging.env:/config/staging.env:ro - ./staging/staging.env:/config/staging.env:ro
network_mode: none network_mode: none

View File

@@ -18,11 +18,11 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/ COPY infra/requirements.txt infra/requirements.lock /app/infra/
RUN python -m pip install --no-cache-dir --require-hashes -r local/requirements.lock RUN python -m pip install --no-cache-dir --require-hashes -r infra/requirements.lock
COPY local /app/local COPY app /app/app
RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin dtf RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin dtf
USER 10001:10001 USER 10001:10001
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PYTHONPATH=/app ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PYTHONPATH=/app
EXPOSE 8000 EXPOSE 8000
CMD ["uvicorn", "local.app:app", "--host", "0.0.0.0", "--port", "8000", "--no-access-log"] CMD ["uvicorn", "app.app:app", "--host", "0.0.0.0", "--port", "8000", "--no-access-log"]

View File

@@ -8,10 +8,10 @@ ARG NGINX_BASE_IMAGE=nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b
FROM ${PYTHON_BASE_IMAGE} AS policy FROM ${PYTHON_BASE_IMAGE} AS policy
WORKDIR /build WORKDIR /build
COPY web /build/web COPY web /build/web
COPY local /build/local COPY infra /build/infra
COPY deploy /build/deploy COPY deploy /build/deploy
ENV NGINX_TEMPLATE=/build/deploy/nginx.conf.template ENV NGINX_TEMPLATE=/build/deploy/nginx.conf.template
RUN python local/compile_web.py RUN python infra/compile_web.py
FROM ${NGINX_BASE_IMAGE} FROM ${NGINX_BASE_IMAGE}
# Same reason as the API image: a pinned base freezes its packages. # Same reason as the API image: a pinned base freezes its packages.

View File

@@ -38,10 +38,10 @@ SOURCE_BLOCKERS = {
# matching when R2 support landed; they were removed rather than left to rot. # matching when R2 support landed; they were removed rather than left to rot.
# What remains is the real blocker: no production payment or messaging adapter # What remains is the real blocker: no production payment or messaging adapter
# exists, so these lines must change before a release can be meaningful. # exists, so these lines must change before a release can be meaningful.
'local/runtime.py': ( 'app/runtime.py': (
'payment = FakePayment()', 'payment = FakePayment()',
), ),
'local/worker.py': ( 'app/worker.py': (
"adapters = {'tiny': FakeTiny(), 'whatsapp': FakeWhatsApp()}", "adapters = {'tiny': FakeTiny(), 'whatsapp': FakeWhatsApp()}",
), ),
} }
@@ -68,7 +68,7 @@ def secret_loading_errors(root=ROOT):
import importlib.util import importlib.util
import tempfile import tempfile
module_path = root / 'local' / 'secrets.py' module_path = root / 'app' / 'core' / 'secrets.py'
if not module_path.exists(): if not module_path.exists():
return ['local runtime does not load the production Docker secret *_FILE settings'] return ['local runtime does not load the production Docker secret *_FILE settings']
try: try:
@@ -76,7 +76,7 @@ def secret_loading_errors(root=ROOT):
module = importlib.util.module_from_spec(spec) module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) spec.loader.exec_module(module)
except Exception as exc: except Exception as exc:
return [f'local/secrets.py could not be loaded: {exc}'] return [f'app/core/secrets.py could not be loaded: {exc}']
stack_names = set() stack_names = set()
stack = root / 'deploy' / 'stack.yaml' stack = root / 'deploy' / 'stack.yaml'
@@ -97,7 +97,7 @@ def secret_loading_errors(root=ROOT):
try: try:
module.load(environ) module.load(environ)
except Exception as exc: except Exception as exc:
failures.append(f'{name}_FILE is not resolved by local/secrets.py: {exc}') failures.append(f'{name}_FILE is not resolved by app/core/secrets.py: {exc}')
continue continue
if environ.get(name) != 'resolved-value': if environ.get(name) != 'resolved-value':
failures.append(f'{name}_FILE did not produce {name}') failures.append(f'{name}_FILE did not produce {name}')
@@ -107,7 +107,7 @@ def secret_loading_errors(root=ROOT):
except Exception: except Exception:
pass pass
else: else:
failures.append('local/secrets.py does not fail closed on an unreadable secret') failures.append('app/core/secrets.py does not fail closed on an unreadable secret')
return failures return failures

View File

@@ -100,8 +100,8 @@ class ProductionPreflightTests(unittest.TestCase):
def test_fake_checkout_is_explicitly_blocked(self): def test_fake_checkout_is_explicitly_blocked(self):
errors = source_errors() errors = source_errors()
self.assertTrue(any('local/runtime.py remains local-only' in error for error in errors)) self.assertTrue(any('app/runtime.py remains local-only' in error for error in errors))
self.assertTrue(any('local/worker.py remains local-only' in error for error in errors)) self.assertTrue(any('app/worker.py remains local-only' in error for error in errors))
def test_secret_reuse_and_incoherent_limits_are_rejected(self): def test_secret_reuse_and_incoherent_limits_are_rejected(self):
values = valid_config() values = valid_config()

View File

@@ -57,7 +57,7 @@ services:
db-init: db-init:
image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest} image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest}
command: python -m local.bootstrap command: python -m app.bootstrap
environment: environment:
DATABASE_ADMIN_HOST: db DATABASE_ADMIN_HOST: db
DATABASE_ADMIN_NAME: dtf DATABASE_ADMIN_NAME: dtf
@@ -114,7 +114,7 @@ services:
worker: worker:
image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest} image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest}
command: python -m local.worker command: python -m app.worker
environment: *app-environment environment: *app-environment
networks: [backend, egress] networks: [backend, egress]
read_only: true read_only: true
@@ -175,7 +175,7 @@ services:
configs: configs:
clamd_config: clamd_config:
file: ./local/clamd.conf file: ./infra/clamd.conf
volumes: volumes:
postgres-data: postgres-data:

View File

@@ -5,9 +5,12 @@ RUN apt-get update \
&& apt-get upgrade -y \ && apt-get upgrade -y \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
WORKDIR /app WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/ COPY infra/requirements.txt infra/requirements.lock /app/infra/
RUN pip install --no-cache-dir --require-hashes -r local/requirements.lock RUN pip install --no-cache-dir --require-hashes -r infra/requirements.lock
COPY local /app/local COPY app /app/app
# The local image carries the suites so they can run inside the stack network.
# deploy/Dockerfile.api deliberately does not: tests are not part of what ships.
COPY tests /app/tests
RUN useradd --uid 10001 --create-home dtf RUN useradd --uid 10001 --create-home dtf
USER dtf USER dtf
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1

View File

@@ -4,4 +4,4 @@
# makes an empty directory instead and the container fails to start. # makes an empty directory instead and the container fails to start.
ARG CLAMAV_IMAGE=clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4 ARG CLAMAV_IMAGE=clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4
FROM ${CLAMAV_IMAGE} FROM ${CLAMAV_IMAGE}
COPY local/clamd.conf /etc/clamav/clamd.conf COPY infra/clamd.conf /etc/clamav/clamd.conf

View File

@@ -1,6 +1,6 @@
# Provisioning script and policies baked in, for the same reason as the scanner. # Provisioning script and policies baked in, for the same reason as the scanner.
ARG MINIO_IMAGE=quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z ARG MINIO_IMAGE=quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z
FROM ${MINIO_IMAGE} FROM ${MINIO_IMAGE}
COPY local/storage-init.sh /init.sh COPY infra/storage-init.sh /init.sh
COPY local/storage-policy.json /policy.json COPY infra/storage-policy.json /policy.json
COPY local/storage-lifecycle.json /lifecycle.json COPY infra/storage-lifecycle.json /lifecycle.json

View File

@@ -1,8 +1,8 @@
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9 AS policy FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9 AS policy
WORKDIR /build WORKDIR /build
COPY web /build/web COPY web /build/web
COPY local /build/local COPY infra /build/infra
RUN python local/compile_web.py RUN python infra/compile_web.py
# Same pinned base as deploy/Dockerfile.web. # Same pinned base as deploy/Dockerfile.web.
FROM nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8 FROM nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8

View File

@@ -18,7 +18,7 @@ for html in (root / 'web').glob('*.html'):
if not re.search(r'\bsrc\s*=', attributes, re.I) and script.strip(): if not re.search(r'\bsrc\s*=', attributes, re.I) and script.strip():
hashes.append("'sha256-" + base64.b64encode(hashlib.sha256(script.encode()).digest()).decode() + "'") hashes.append("'sha256-" + base64.b64encode(hashlib.sha256(script.encode()).digest()).decode() + "'")
template = Path(os.environ.get('NGINX_TEMPLATE', root / 'local/nginx.conf.template')).read_text() template = Path(os.environ.get('NGINX_TEMPLATE', root / 'infra/nginx.conf.template')).read_text()
rendered = template.replace('@SCRIPT_HASHES@', ' '.join(hashes)) rendered = template.replace('@SCRIPT_HASHES@', ' '.join(hashes))
# Collapse the gap an empty hash list leaves behind, so the policy reads cleanly. # Collapse the gap an empty hash list leaves behind, so the policy reads cleanly.
rendered = re.sub(r"(script-src 'self')\s+;", r'\1;', rendered) rendered = re.sub(r"(script-src 'self')\s+;", r'\1;', rendered)

View File

@@ -10,4 +10,4 @@ docker run --rm \
--volume "$root:/src" \ --volume "$root:/src" \
--workdir /src \ --workdir /src \
python:3.12-slim \ python:3.12-slim \
sh -c 'python -m pip install --no-cache-dir --target /tmp/piptools pip==25.3 pip-tools==7.5.2 && PYTHONPATH=/tmp/piptools python -m piptools compile --generate-hashes --allow-unsafe --strip-extras --no-emit-index-url --output-file local/requirements.lock local/requirements.txt' sh -c 'python -m pip install --no-cache-dir --target /tmp/piptools pip==25.3 pip-tools==7.5.2 && PYTHONPATH=/tmp/piptools python -m piptools compile --generate-hashes --allow-unsafe --strip-extras --no-emit-index-url --output-file infra/requirements.lock infra/requirements.txt'

1
ops/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Operational commands: backup, readiness, audit, security summary."""

View File

@@ -1,8 +1,8 @@
"""Local alert triage: prints redacted counts; exits 1 when attention is required.""" """Local alert triage: prints redacted counts; exits 1 when attention is required."""
import json import json
from datetime import datetime, timezone from datetime import datetime, timezone
from .db import connect from app.core.db import connect
from .scanning import ClamAV from app.scanning import ClamAV
def scanner_status(): def scanner_status():
try: try:

View File

@@ -8,8 +8,8 @@ import tarfile
from datetime import datetime, timezone from datetime import datetime, timezone
from uuid import uuid4 from uuid import uuid4
from .adapters import LocalS3Storage, require_local from app.adapters import LocalS3Storage, require_local
from .db import connect from app.core.db import connect
CHUNK = 8 * 1024 * 1024 CHUNK = 8 * 1024 * 1024
MAX_OBJECT = min(128 * 1024 * 1024, int(os.environ.get('SCAN_MAX_BYTES', '134217728'))) MAX_OBJECT = min(128 * 1024 * 1024, int(os.environ.get('SCAN_MAX_BYTES', '134217728')))

1
tests/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Test suites. Not shipped in the production image."""

View File

@@ -59,7 +59,7 @@ try{
await site.click('[data-cam="tabela"]'); await site.click('[data-cam="tabela"]');
const root=await site.call('DOM.getDocument'); const root=await site.call('DOM.getDocument');
const input=await site.call('DOM.querySelector',{nodeId:root.root.nodeId,selector:'#inp'}); const input=await site.call('DOM.querySelector',{nodeId:root.root.nodeId,selector:'#inp'});
await site.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('local/fixtures/local-test.cdr')]}); await site.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('tests/fixtures/local-test.cdr')]});
await waitFor(()=>site.eval('!!document.querySelector("[data-comp]")'),'manual length field'); await waitFor(()=>site.eval('!!document.querySelector("[data-comp]")'),'manual length field');
await site.fill('[data-comp]','1.01','change'); await site.fill('[data-comp]','1.01','change');
await waitFor(()=>site.eval('!!itemAtual'),'cart calculation'); await waitFor(()=>site.eval('!!itemAtual'),'cart calculation');
@@ -99,7 +99,7 @@ try{
await waitFor(()=>kanban.eval(`!!document.querySelector('[data-order="${oid}"] [data-final-item]')`),'final upload controls'); await waitFor(()=>kanban.eval(`!!document.querySelector('[data-order="${oid}"] [data-final-item]')`),'final upload controls');
const doc=await kanban.call('DOM.getDocument'); const doc=await kanban.call('DOM.getDocument');
const input=await kanban.call('DOM.querySelector',{nodeId:doc.root.nodeId,selector:`[data-order="${oid}"] [data-final-item]`}); const input=await kanban.call('DOM.querySelector',{nodeId:doc.root.nodeId,selector:`[data-order="${oid}"] [data-final-item]`});
await kanban.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('local/fixtures/local-test.cdr')]}); await kanban.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('tests/fixtures/local-test.cdr')]});
await kanban.fill(`[data-order="${oid}"] input[placeholder="Nota da revisão"]`,'Browser test final file'); await kanban.fill(`[data-order="${oid}"] input[placeholder="Nota da revisão"]`,'Browser test final file');
await kanban.eval(`(()=>{const form=document.querySelector('[data-order="${oid}"] form');form.querySelector('[type=checkbox]').click();form.requestSubmit();})()`); await kanban.eval(`(()=>{const form=document.querySelector('[data-order="${oid}"] form');form.querySelector('[type=checkbox]').click();form.requestSubmit();})()`);
await waitFor(()=>kanban.eval(`board.orders.find(o=>o.id==='${oid}').version===2`),'final file approval'); await waitFor(()=>kanban.eval(`board.orders.find(o=>o.id==='${oid}').version===2`),'final file approval');

View File

@@ -2,9 +2,9 @@
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from uuid import uuid4 from uuid import uuid4
from botocore.exceptions import ClientError from botocore.exceptions import ClientError
from .adapters import LocalS3Storage from app.adapters import LocalS3Storage
from .db import connect from app.core.db import connect
from .worker import cleanup from app.worker import cleanup
def run(): def run():
storage=LocalS3Storage() storage=LocalS3Storage()

View File

@@ -2,10 +2,10 @@
import hashlib import hashlib
from unittest.mock import patch from unittest.mock import patch
from botocore.exceptions import ClientError from botocore.exceptions import ClientError
from .db import connect from app.core.db import connect
from .adapters import LocalS3Storage from app.adapters import LocalS3Storage
from .auth import client_ip, password_hash, password_matches from app.core.auth import client_ip, password_hash, password_matches
from .scanning import ClamAV, require_clean from app.scanning import ClamAV, require_clean
from fastapi import HTTPException from fastapi import HTTPException
class FakeRequest: class FakeRequest:
@@ -29,8 +29,8 @@ def check_client_ip():
def check_operator_accounts(): def check_operator_accounts():
"""Accounts are per person, and disabling one ends its access at once.""" """Accounts are per person, and disabling one ends its access at once."""
from uuid import uuid4 from uuid import uuid4
from .auth import password_hash, password_matches from app.core.auth import password_hash, password_matches
from .operators import seed_from_environment, set_active from app.operators import seed_from_environment, set_active
email='runtime-check-'+uuid4().hex[:8]+'@example.test' email='runtime-check-'+uuid4().hex[:8]+'@example.test'
with connect() as c: with connect() as c:
c.execute('INSERT INTO dtf_local.operators(id,email,name,password_hash) VALUES(%s,%s,%s,%s)', c.execute('INSERT INTO dtf_local.operators(id,email,name,password_hash) VALUES(%s,%s,%s,%s)',
@@ -79,7 +79,7 @@ def run():
require_clean({'complete':True,'scan_state':'clean'}) require_clean({'complete':True,'scan_state':'clean'})
assert ClamAV().ping() and ClamAV().version().startswith('ClamAV ') assert ClamAV().ping() and ClamAV().version().startswith('ClamAV ')
assert ClamAV().scan(None,134217729)[0]=='rejected' assert ClamAV().scan(None,134217729)[0]=='rejected'
with patch('local.scanning.socket.create_connection',side_effect=OSError('offline')): with patch('app.scanning.socket.create_connection',side_effect=OSError('offline')):
try:ClamAV().scan(None,1);raise AssertionError('Offline scanner returned success') try:ClamAV().scan(None,1);raise AssertionError('Offline scanner returned success')
except OSError:pass except OSError:pass
print('PASS: runtime DB/S3 least privilege, legacy/current password hashes, quarantine states and scanner size/offline behavior') print('PASS: runtime DB/S3 least privilege, legacy/current password hashes, quarantine states and scanner size/offline behavior')

View File

@@ -1,6 +1,6 @@
"""Harmless EICAR anti-malware test and blocked download/quote regressions.""" """Harmless EICAR anti-malware test and blocked download/quote regressions."""
from uuid import uuid4 from uuid import uuid4
from .smoke_test import Client, upload_bytes from tests.smoke_test import Client, upload_bytes
def run(): def run():
customer=Client();customer.call('/session') customer=Client();customer.call('/session')

View File

@@ -5,7 +5,7 @@ from urllib.error import HTTPError
from urllib.request import Request, urlopen from urllib.request import Request, urlopen
from urllib.parse import urlparse, parse_qs from urllib.parse import urlparse, parse_qs
from uuid import uuid4 from uuid import uuid4
from .smoke_test import Client, BASE, with_host from tests.smoke_test import Client, BASE, with_host
def raw(path, expected, headers=None, body=None): def raw(path, expected, headers=None, body=None):
request=Request(BASE+path, data=body, headers=with_host(headers)) request=Request(BASE+path, data=body, headers=with_host(headers))

View File

@@ -1,6 +1,6 @@
"""Local stack integration checks; creates and retains clearly named test orders. """Local stack integration checks; creates and retains clearly named test orders.
Run: python3 -m local.smoke_test. Standard library only. Honors .env/environment. Run: python3 -m tests.smoke_test. Standard library only. Honors .env/environment.
""" """
from concurrent.futures import ThreadPoolExecutor from concurrent.futures import ThreadPoolExecutor
import hashlib import hashlib

View File

@@ -13,8 +13,8 @@ def normalized(name):
class DependencyLockTests(unittest.TestCase): class DependencyLockTests(unittest.TestCase):
@classmethod @classmethod
def setUpClass(cls): def setUpClass(cls):
cls.direct_text = (ROOT / 'local/requirements.txt').read_text() cls.direct_text = (ROOT / 'infra/requirements.txt').read_text()
cls.lock_text = (ROOT / 'local/requirements.lock').read_text() cls.lock_text = (ROOT / 'infra/requirements.lock').read_text()
def test_every_direct_pin_matches_lock(self): def test_every_direct_pin_matches_lock(self):
direct = {normalized(name): version for name, version in direct = {normalized(name): version for name, version in
@@ -34,9 +34,9 @@ class DependencyLockTests(unittest.TestCase):
self.assertTrue(hashes, f'{match.group(1)} has no SHA-256 artifact hash') self.assertTrue(hashes, f'{match.group(1)} has no SHA-256 artifact hash')
def test_image_build_requires_the_lock_and_hashes(self): def test_image_build_requires_the_lock_and_hashes(self):
dockerfile = (ROOT / 'local/Dockerfile').read_text() dockerfile = (ROOT / 'infra/Dockerfile').read_text()
self.assertIn('requirements.lock', dockerfile) self.assertIn('requirements.lock', dockerfile)
self.assertIn('--require-hashes -r local/requirements.lock', dockerfile) self.assertIn('--require-hashes -r infra/requirements.lock', dockerfile)
def test_reproducible_generator_is_recorded(self): def test_reproducible_generator_is_recorded(self):
self.assertIn('./local/lock_dependencies.sh', self.lock_text[:300]) self.assertIn('./local/lock_dependencies.sh', self.lock_text[:300])

View File

@@ -1,9 +1,9 @@
"""Run from repository root: python3 -m unittest local.test_pricing -v.""" """Run from repository root: python3 -m unittest tests.test_pricing -v."""
import json import json
from pathlib import Path from pathlib import Path
import subprocess import subprocess
import unittest import unittest
from .pricing import price, TIERS from app.core.pricing import price, TIERS
class PricingTests(unittest.TestCase): class PricingTests(unittest.TestCase):
def test_every_grade_against_actual_site_javascript(self): def test_every_grade_against_actual_site_javascript(self):

View File

@@ -3,7 +3,7 @@ import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from local.secrets import SECRET_FILE_SETTINGS, load, read_secret from app.core.secrets import SECRET_FILE_SETTINGS, load, read_secret
class SecretFileTests(unittest.TestCase): class SecretFileTests(unittest.TestCase):

View File

@@ -2,7 +2,7 @@ import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from .staging_readiness import read_config, validate from ops.staging_readiness import read_config, validate
VALID = ''' VALID = '''

View File

@@ -1,7 +1,7 @@
"""Customer identity, correction and final-file trust boundaries against local stack.""" """Customer identity, correction and final-file trust boundaries against local stack."""
from uuid import uuid4 from uuid import uuid4
from urllib.request import urlopen from urllib.request import urlopen
from .smoke_test import Client, upload_bytes from tests.smoke_test import Client, upload_bytes
def run(): def run():
customer=Client();other=Client();customer.call('/session');other.call('/session') customer=Client();other=Client();customer.call('/session');other.call('/session')