refactor: lay the repository out by role
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Integration suite on a real stack (push) Successful in 1m25s
Build and deploy / Secret scan and release gate (push) Successful in 5s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m31s

local/ held six unrelated things under a name that stopped being true once it
became the production runtime: the service, the frontend, the tests, the ops
commands, the container definitions and the dependency lock, 65 files with
nothing to tell them apart.

  app/      the service: api/ routers, core/ for identity, database, models,
            prices and secret loading, and the worker, bootstrap and schema
  tests/    the twelve suites, no longer inside the shipped package
  ops/      backup, readiness, dependency audit, security summary
  infra/    Dockerfiles, gateway templates, ClamAV and storage configuration,
            the requirements and their hash lock
  web/      the Site, Kanban and portal pages with their scripts

deploy/Dockerfile.api now copies app/ alone, so the tests stop shipping to
production; the local image still carries them, because the suites run inside
the stack's network.

Five kinds of reference had to follow, and each was found by something different
rather than by reading. Imports of the form "from . import db" survived a rewrite
that only matched "from .db import". Tests kept relative imports of modules that
had left the package. A mock.patch target names its module in a string, where no
import rewriting can see it. The browser test resolves a fixture by path. And the
release gate's markers pointed at local/runtime.py and local/worker.py, which is
the decay its new marker test exists to catch — it caught it.

Verified from docker compose down -v: the stack starts, all six integration
suites, both browser suites and the twenty-nine unit tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Cauê Faleiros
2026-09-21 17:40:57 -03:00
parent c9f8122600
commit b329f76378
69 changed files with 146 additions and 139 deletions

View File

@@ -16,14 +16,14 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Run fast regression checks
run: |
python3 -m py_compile local/*.py deploy/*.py
python3 -m py_compile app/*.py app/**/*.py ops/*.py deploy/*.py
python3 -m unittest \
local.test_dependency_lock \
local.test_staging_readiness \
tests.test_dependency_lock \
tests.test_staging_readiness \
deploy.test_production_preflight \
local.test_pricing \
local.test_secrets -v
sh -n local/lock_dependencies.sh
tests.test_pricing \
tests.test_secrets -v
sh -n infra/lock_dependencies.sh
integration:
name: Integration suite on a real stack
@@ -75,13 +75,13 @@ jobs:
$COMPOSE exec -T \
-e SITE_BASE_URL=http://site \
-e SITE_HOST_HEADER=localhost \
api python -m "local.$suite"
api python -m "tests.$suite"
done
- name: Runtime and retention regressions
run: |
$COMPOSE exec -T api python -m local.retention_test
$COMPOSE exec -T api python -m local.runtime_security_test
$COMPOSE exec -T api python -m tests.retention_test
$COMPOSE exec -T api python -m tests.runtime_security_test
# These need a real Chrome. They are the only coverage for the artwork
# editor and the full customer journey, so install google-chrome-stable
@@ -110,8 +110,8 @@ jobs:
exit 0
fi
echo "Using $CHROME_BIN"
node local/artwork_browser_test.mjs
node local/browser_test.mjs
node tests/artwork_browser_test.mjs
node tests/browser_test.mjs
- name: Diagnostics on failure
if: failure()

View File

@@ -249,7 +249,7 @@ supports subsequent corrections or scoped enhancements.
The current implementation target is localhost before any production connection.
Use `docker-compose.yml`, `.env.example`, and `LOCAL_SETUP.md`. The runtime is in
`local/`; historical `portal/`, `kanban/`, `agente/`, and `schema.sql` are preserved
`app/`; historical `portal/`, `kanban/`, `agente/`, and `schema.sql` are preserved
as references and are not imported or started by Compose.
- One local `dtf-cloud` Compose project runs seven long-lived services: Site and
@@ -308,10 +308,10 @@ as references and are not imported or started by Compose.
`security_status` command summarizes authentication, rate-limit, scan, and
scanner/signature alerts without exposing secrets. Security regression tests,
exact-runtime Python dependency auditing, and Trivy image reports live under
`local/` and `output/security/`; open findings are documented in
`app/` and `output/security/`; open findings are documented in
`SECURITY_REPORT.md` and are not a production-readiness claim.
- All direct and transitive Python packages are pinned with artifact hashes in
`local/requirements.lock`; the API image build requires those hashes. The lock
`infra/requirements.lock`; the API image build requires those hashes. The lock
is regenerated in a disposable Python 3.12 container by
`local/lock_dependencies.sh`. A fresh exact-runtime audit found no known Python
advisories on 2026-09-15; this does not cover OS/container findings.
@@ -327,7 +327,7 @@ as references and are not imported or started by Compose.
not been deployed. Its fail-closed preflight intentionally rejects the current
source until production adapters, Docker-secret file loading, approved inputs,
restore rehearsal, image scans, and human security approval are complete.
- `python3 -m local.backup create-and-verify` creates a private, Git-ignored bundle
- `python3 -m app.backup create-and-verify` creates a private, Git-ignored bundle
containing a PostgreSQL dump plus every complete, unexpired object already marked
`clean`. SHA-256 manifests protect both parts. Verification restores the database
under a UUID name and the object bytes under a UUID MinIO prefix, hashes the

View File

@@ -1,6 +1,6 @@
# DTF local development
Read `CONTEXT.md` first. The current runtime lives in `local/`; older portal,
Read `CONTEXT.md` first. The service lives in `app/`; older portal,
Kanban, agent, requirements, and SQL files are historical prototypes.
## Start
@@ -51,7 +51,7 @@ operator interfaces.
## Browser test order
1. Open the Site. Choose **Arquivo por metro** and the manual/table-price path
(CDR/AI/PSD/TIFF). Select `local/fixtures/local-test.cdr`. This is deliberately
(CDR/AI/PSD/TIFF). Select `tests/fixtures/local-test.cdr`. This is deliberately
harmless text for upload testing, not a printable CDR file.
2. Enter **1.01 metres**. The original calculation bills **1.10 m × R$19.90 =
R$21.89**, with grade 0 and pickup. You can also use your own non-sensitive
@@ -142,8 +142,8 @@ pricing, plus assembly and invalid-input checks.
With the stack healthy, run the integration test (Python standard library only):
```bash
python3 -m local.smoke_test
python3 -m local.workflow_test
python3 -m tests.smoke_test
python3 -m tests.workflow_test
```
It checks direct two-part upload/resume, missing parts, session isolation,
@@ -159,10 +159,10 @@ Security and malware regressions are separate so an authorized harmless EICAR
test is unmistakable:
```bash
python3 -m local.security_test
python3 -m local.scanning_test
docker compose exec -T api python -m local.runtime_security_test
docker compose exec -T api python -m local.retention_test
python3 -m tests.security_test
python3 -m tests.scanning_test
docker compose exec -T api python3 -m tests.runtime_security_test
docker compose exec -T api python3 -m tests.retention_test
```
`local.scanning_test` stores an EICAR fixture as `SECURITY-EICAR.cdr`; ClamAV
@@ -172,7 +172,7 @@ is retained for at most three days, so it temporarily appears in alert summaries
For an automated real-browser walkthrough, install Chrome and use Node 22+:
```bash
node local/browser_test.mjs
node tests/browser_test.mjs
```
Set `CHROME_BIN` if Chrome is not at `/usr/bin/google-chrome-stable`. The test
@@ -217,7 +217,7 @@ again. The operator can still inspect order records.
## Local backup and restore check
```bash
python3 -m local.backup create-and-verify
python3 -m app.backup create-and-verify
```
This writes a private four-file bundle in `backups/` (ignored by Git and Docker
@@ -234,7 +234,7 @@ checks database counts, bundle hashes, every archived object's hash, and the byt
downloaded after restore, then removes only the temporary database and objects. It
never restores over active data. Keep every bundle file private: it contains
customer data, password hashes, and customer artwork. To verify it again, run
`python3 -m local.backup verify` followed by the printed
`python3 -m app.backup verify` followed by the printed
`backups/...manifest.json` path. Legacy database-only `.dump` backups remain
verifiable. Scheduling, offsite copies, and a production restore runbook remain
unfinished.
@@ -242,7 +242,7 @@ unfinished.
After building the current image, test retention with synthetic files:
```bash
docker compose exec -T api python -m local.retention_test
docker compose exec -T api python3 -m tests.retention_test
```
This checks that expired bytes are removed while unexpired files survive. It
@@ -270,7 +270,7 @@ exposes `/minio/health/ready`.
Run the redacted local alert summary inside the API network namespace:
```bash
docker compose exec -T api python -m local.security_status
docker compose exec -T api python3 -m app.security_status
```
Exit status 1 means attention is required. Review blocked artwork, rate limits,
@@ -306,12 +306,12 @@ Offline PDF previews may fall back to the prototype's manual/table-price path.
## Dependency lock
The API, worker, and database initializer install every Python dependency from
`local/requirements.lock` with `--require-hashes`. `local/requirements.txt`
`infra/requirements.lock` with `--require-hashes`. `infra/requirements.txt`
remains the human-maintained direct dependency list. After deliberately changing
a direct pin, regenerate the lock in the same Python 3.12 environment and rebuild:
```bash
./local/lock_dependencies.sh
./infra/lock_dependencies.sh
docker compose up --build -d --wait
```
@@ -339,7 +339,7 @@ contracts and owners. See `staging/README.md`.
Keep this stack local. Before connecting real services, confirm the production
checkout trust workflow, complete `PRODUCTION_INPUTS.md`, and pass the isolated
staging-readiness gate. Then implement the actual staging composition using the adapter contracts in
`local/adapters.py`: start with private S3 staging storage, CORS, signed multipart
`app/adapters.py`: start with private S3 staging storage, CORS, signed multipart
contract tests and scoped credentials injected outside Git. Add provider sandbox
adapters one at a time. Mercado Pago requires authenticated, signed, idempotent
webhook handling before real payment is allowed; Tiny/Olist and WhatsApp need
@@ -363,7 +363,7 @@ Run the source-only gate without credentials:
python3 deploy/production_preflight.py --source-only
```
It must remain blocked while `local/` supports only local fake adapters and does
It must remain blocked while `app/` supports only local fake adapters and does
not load Docker secret `*_FILE` settings. Do not bypass or delete this check.
After approved production implementations and inputs exist, follow
`PORTAINER.md` and `deploy/PRODUCTION_CHECKLIST.md`; release and deployment

View File

@@ -12,7 +12,7 @@
client inputs (1.1, 1.2, 2.10). Block 1 still waits on client inputs for
1.1/1.2.
**Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`,
**Last audit:** 2026-09-18, full read of `app/`, `dtf-site.html`, `deploy/`,
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
| Status | Meaning |
@@ -279,7 +279,7 @@ deliberately not bundled here.
Accounts now live in `dtf_local.operators`, one per person, with `movements.operator`
and `order_files.created_by` recording who actually acted. Administered from the API
container with `python -m local.operators` (list, add, password, disable, enable);
container with `python3 -m app.operators` (list, add, password, disable, enable);
passwords are read from the terminal so they never reach shell history or the
process list, and disabling revokes open sessions immediately rather than leaving
them valid for the rest of the eight-hour window.
@@ -417,9 +417,9 @@ charges. Fix as part of 1.1.
with a header saying they are background, not instructions.
- `[x]` 5.3 — Root `requirements.txt` deleted. It pinned by wildcard, listed
packages the system does not use, and sat next to the hash-locked
`local/requirements.lock` inviting the wrong one to be installed. Only historical
`infra/requirements.lock` inviting the wrong one to be installed. Only historical
documentation referred to it.
- `[x]` 5.4 — `pip` removed from `local/requirements.txt` and from the lock. Nothing
- `[x]` 5.4 — `pip` removed from `infra/requirements.txt` and from the lock. Nothing
depended on it; it was pinned only because it was listed directly, and installing
it put a package manager inside the read-only runtime image. The base image's own
pip performs the hash-enforced install. Verified: the image builds under

View File

@@ -84,7 +84,7 @@ not simply increasing the upload limit.
Run:
```bash
docker compose exec -T api python -m local.security_status
docker compose exec -T api python3 -m app.security_status
```
An exit status of 1 requires review. At closeout, attention was expected because
@@ -99,7 +99,7 @@ errors, unexplained authentication bursts, or stale signatures as incidents.
`pip-audit` inspected the exact packages installed in the hash-enforced rebuilt API
and found no known Python advisories on 2026-09-15. All 26 direct and transitive
runtime packages are pinned with artifact hashes in `local/requirements.lock`.
runtime packages are pinned with artifact hashes in `infra/requirements.lock`.
`local/lock_dependencies.sh` regenerates it in a disposable Python 3.12 container.
This is a point-in-time package-database result, not proof that the dependencies
or image are vulnerability-free. Scheduled lock refresh and audit automation remain

View File

@@ -7,10 +7,10 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException
from .. import db
from ..core import db
from ..artwork import submit_files
from ..auth import operator
from ..models import ArtworkSubmission, UploadStart
from ..core.auth import operator
from ..core.models import ArtworkSubmission, UploadStart
from ..runtime import file_rows, operator_identity
from .uploads import begin_upload, complete_upload, part_url, upload_status

View File

@@ -6,11 +6,11 @@ from fastapi import APIRouter, Depends, HTTPException, Request, Response
from psycopg.errors import UniqueViolation
from psycopg.types.json import Jsonb
from .. import db
from ..core import db
from ..artwork import submit_files
from ..auth import (DUMMY_PASSWORD_HASH, audit, client_ip, new_session, owner,
from ..core.auth import (DUMMY_PASSWORD_HASH, audit, client_ip, new_session, owner,
password_hash, password_matches, session_row, throttle, transfer_guest)
from ..models import ArtworkSubmission, Login, Register
from ..core.models import ArtworkSubmission, Login, Register
from ..runtime import STATES, file_rows, owned_order, storage
from ..scanning import require_clean

View File

@@ -3,9 +3,9 @@ import os
from fastapi import APIRouter, HTTPException, Request, Response
from .. import db
from ..auth import client_ip, owner, new_session, rate_limit
from ..models import Freight
from ..core import db
from ..core.auth import client_ip, owner, new_session, rate_limit
from ..core.models import Freight
from ..runtime import ENVIRONMENT, GUEST_SESSION_LIMIT, PART_BYTES, freight, storage
router = APIRouter()

View File

@@ -8,11 +8,11 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from psycopg.types.json import Jsonb
from .. import db
from ..auth import (COOKIE_SECURE, DUMMY_PASSWORD_HASH, audit, client_ip, operator,
from ..core import db
from ..core.auth import (COOKIE_SECURE, DUMMY_PASSWORD_HASH, audit, client_ip, operator,
password_matches, throttle)
from ..models import Move, OperatorLogin, Review
from ..pricing import price
from ..core.models import Move, OperatorLogin, Review
from ..core.pricing import price
from ..runtime import (BOARD_FINISHED_LIMIT, BOARD_QUOTE_LIMIT, STATES, TRANSITIONS,
enqueue, freight, quote_view, storage, upload_row)
from ..scanning import require_clean

View File

@@ -5,9 +5,9 @@ from uuid import UUID, uuid4
from fastapi import APIRouter, Depends, HTTPException
from psycopg.types.json import Jsonb
from .. import db
from ..auth import owner
from ..models import Pay
from ..core import db
from ..core.auth import owner
from ..core.models import Pay
from ..runtime import ENVIRONMENT, enqueue, payment, upload_row
from ..scanning import require_clean

View File

@@ -6,9 +6,9 @@ from uuid import UUID, uuid4
from fastapi import APIRouter, Depends, HTTPException
from psycopg.types.json import Jsonb
from .. import db
from ..auth import owner
from ..models import QuoteRequest
from ..core import db
from ..core.auth import owner
from ..core.models import QuoteRequest
from ..runtime import freight, quote_view, upload_row
from ..scanning import require_clean

View File

@@ -10,9 +10,9 @@ from uuid import UUID, uuid4
from botocore.exceptions import ClientError
from fastapi import APIRouter, Depends, HTTPException
from .. import db
from ..auth import audit, owner, rate_limit
from ..models import UploadStart
from ..core import db
from ..core.auth import audit, owner, rate_limit
from ..core.models import UploadStart
from ..runtime import PART_BYTES, storage, upload_row
router = APIRouter()

View File

@@ -10,8 +10,8 @@ from fastapi import FastAPI
from fastapi.responses import JSONResponse
from starlette.middleware.trustedhost import TrustedHostMiddleware
from . import db
from .auth import audit, client_ip
from .core import db
from .core.auth import audit, client_ip
from .runtime import ALLOWED_HOSTS, ALLOWED_ORIGINS, storage
from .api import artwork, customer, health, operator, orders, quotes, uploads

View File

@@ -4,7 +4,7 @@ from pathlib import Path
from urllib.parse import urlparse
import psycopg
from psycopg import sql
from .secrets import load as load_secret_files
from .core.secrets import load as load_secret_files
from .operators import seed_from_environment

1
app/core/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Shared foundations: identity, database, models, prices, secret loading."""

View File

@@ -6,7 +6,7 @@ import logging
import json
from uuid import UUID, uuid4
from fastapi import HTTPException, Request
from .db import connect
from ..core.db import connect
COOKIE_SECURE = os.environ.get('COOKIE_SECURE', 'false').lower() == 'true'

View File

@@ -19,4 +19,5 @@ def connect():
def initialize():
with connect() as c:
c.execute(Path(__file__).with_name('schema.sql').read_text())
# The schema lives at the package root, one level up from core/.
c.execute((Path(__file__).resolve().parent.parent / 'schema.sql').read_text())

View File

@@ -21,8 +21,8 @@ import os
import sys
from uuid import uuid4
from .auth import password_hash
from .db import connect
from .core.auth import password_hash
from .core.db import connect
MIN_PASSWORD = 12

View File

@@ -13,7 +13,7 @@ from fastapi import HTTPException
from psycopg.types.json import Jsonb
from .adapters import FakeFreight, FakePayment, LocalS3Storage, require_runtime
from .secrets import load as load_secret_files
from .core.secrets import load as load_secret_files
# Secret files must resolve before any configuration below is read.
load_secret_files()

View File

@@ -4,8 +4,8 @@ import socket
import struct
import time
from fastapi import HTTPException
from .auth import audit
from .db import connect
from .core.auth import audit
from .core.db import connect
def require_clean(row):
if not row['complete'] or row['scan_state'] != 'clean':

View File

@@ -6,8 +6,8 @@ import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from psycopg.types.json import Jsonb
from .adapters import FakeTiny, FakeWhatsApp, LocalS3Storage, require_runtime
from .secrets import load as load_secret_files
from .db import connect
from .core.secrets import load as load_secret_files
from .core.db import connect
from .scanning import ClamAV, scan_loop
load_secret_files()

View File

@@ -9,7 +9,7 @@
x-app: &app
build:
context: .
dockerfile: local/Dockerfile
dockerfile: infra/Dockerfile
environment: &environment
APP_ENV: local
DATABASE_URL: postgresql://${APP_DB_USER:-dtf_app}:${APP_DB_PASSWORD:-local-app-database-only}@db:5432/${POSTGRES_DB:-dtf_local}
@@ -88,8 +88,8 @@ services:
db-init:
build:
context: .
dockerfile: local/Dockerfile
command: python -m local.bootstrap
dockerfile: infra/Dockerfile
command: python -m app.bootstrap
environment:
# Local only: the app role keeps a password distinct from the administrator.
DATABASE_ADMIN_URL: postgresql://${POSTGRES_USER:-dtf_local}:${POSTGRES_PASSWORD:-local-database-only}@db:5432/${POSTGRES_DB:-dtf_local}
@@ -109,7 +109,7 @@ services:
storage-init:
build:
context: .
dockerfile: local/Dockerfile.storage-init
dockerfile: infra/Dockerfile.storage-init
args:
MINIO_IMAGE: ${MINIO_IMAGE:-quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z}
entrypoint: [/bin/sh, /init.sh]
@@ -128,7 +128,7 @@ services:
# Built, not bind-mounted: see local/Dockerfile.scanner.
build:
context: .
dockerfile: local/Dockerfile.scanner
dockerfile: infra/Dockerfile.scanner
args:
CLAMAV_IMAGE: ${CLAMAV_IMAGE:-clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4}
entrypoint: [clamd, --foreground=true, --config-file=/etc/clamav/clamd.conf]
@@ -146,7 +146,7 @@ services:
api:
<<: *app
command: uvicorn local.app:app --host 0.0.0.0 --port 8000 --no-access-log
command: uvicorn app.app:app --host 0.0.0.0 --port 8000 --no-access-log
depends_on:
db-init: {condition: service_completed_successfully}
storage-init: {condition: service_completed_successfully}
@@ -158,7 +158,7 @@ services:
worker:
<<: *app
command: python -m local.worker
command: python -m app.worker
depends_on:
api: {condition: service_healthy}
scanner: {condition: service_healthy}
@@ -171,7 +171,7 @@ services:
site:
build:
context: .
dockerfile: local/Dockerfile.web
dockerfile: infra/Dockerfile.web
environment:
S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}}
ports:
@@ -193,7 +193,7 @@ services:
kanban:
build:
context: .
dockerfile: local/Dockerfile.web
dockerfile: infra/Dockerfile.web
environment:
WEB_INDEX: kanban.html
S3_PUBLIC_ENDPOINT: ${S3_PUBLIC_ENDPOINT:-http://localhost:${STORAGE_PORT:-9000}}

View File

@@ -7,8 +7,8 @@ services:
readiness:
build:
context: .
dockerfile: local/Dockerfile
command: python -m local.staging_readiness /config/staging.env
dockerfile: infra/Dockerfile
command: python -m app.staging_readiness /config/staging.env
volumes:
- ./staging/staging.env:/config/staging.env:ro
network_mode: none

View File

@@ -18,11 +18,11 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/
RUN python -m pip install --no-cache-dir --require-hashes -r local/requirements.lock
COPY local /app/local
COPY infra/requirements.txt infra/requirements.lock /app/infra/
RUN python -m pip install --no-cache-dir --require-hashes -r infra/requirements.lock
COPY app /app/app
RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin dtf
USER 10001:10001
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PYTHONPATH=/app
EXPOSE 8000
CMD ["uvicorn", "local.app:app", "--host", "0.0.0.0", "--port", "8000", "--no-access-log"]
CMD ["uvicorn", "app.app:app", "--host", "0.0.0.0", "--port", "8000", "--no-access-log"]

View File

@@ -8,10 +8,10 @@ ARG NGINX_BASE_IMAGE=nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b
FROM ${PYTHON_BASE_IMAGE} AS policy
WORKDIR /build
COPY web /build/web
COPY local /build/local
COPY infra /build/infra
COPY deploy /build/deploy
ENV NGINX_TEMPLATE=/build/deploy/nginx.conf.template
RUN python local/compile_web.py
RUN python infra/compile_web.py
FROM ${NGINX_BASE_IMAGE}
# Same reason as the API image: a pinned base freezes its packages.

View File

@@ -38,10 +38,10 @@ SOURCE_BLOCKERS = {
# matching when R2 support landed; they were removed rather than left to rot.
# What remains is the real blocker: no production payment or messaging adapter
# exists, so these lines must change before a release can be meaningful.
'local/runtime.py': (
'app/runtime.py': (
'payment = FakePayment()',
),
'local/worker.py': (
'app/worker.py': (
"adapters = {'tiny': FakeTiny(), 'whatsapp': FakeWhatsApp()}",
),
}
@@ -68,7 +68,7 @@ def secret_loading_errors(root=ROOT):
import importlib.util
import tempfile
module_path = root / 'local' / 'secrets.py'
module_path = root / 'app' / 'core' / 'secrets.py'
if not module_path.exists():
return ['local runtime does not load the production Docker secret *_FILE settings']
try:
@@ -76,7 +76,7 @@ def secret_loading_errors(root=ROOT):
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
except Exception as exc:
return [f'local/secrets.py could not be loaded: {exc}']
return [f'app/core/secrets.py could not be loaded: {exc}']
stack_names = set()
stack = root / 'deploy' / 'stack.yaml'
@@ -97,7 +97,7 @@ def secret_loading_errors(root=ROOT):
try:
module.load(environ)
except Exception as exc:
failures.append(f'{name}_FILE is not resolved by local/secrets.py: {exc}')
failures.append(f'{name}_FILE is not resolved by app/core/secrets.py: {exc}')
continue
if environ.get(name) != 'resolved-value':
failures.append(f'{name}_FILE did not produce {name}')
@@ -107,7 +107,7 @@ def secret_loading_errors(root=ROOT):
except Exception:
pass
else:
failures.append('local/secrets.py does not fail closed on an unreadable secret')
failures.append('app/core/secrets.py does not fail closed on an unreadable secret')
return failures

View File

@@ -100,8 +100,8 @@ class ProductionPreflightTests(unittest.TestCase):
def test_fake_checkout_is_explicitly_blocked(self):
errors = source_errors()
self.assertTrue(any('local/runtime.py remains local-only' in error for error in errors))
self.assertTrue(any('local/worker.py remains local-only' in error for error in errors))
self.assertTrue(any('app/runtime.py remains local-only' in error for error in errors))
self.assertTrue(any('app/worker.py remains local-only' in error for error in errors))
def test_secret_reuse_and_incoherent_limits_are_rejected(self):
values = valid_config()

View File

@@ -57,7 +57,7 @@ services:
db-init:
image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest}
command: python -m local.bootstrap
command: python -m app.bootstrap
environment:
DATABASE_ADMIN_HOST: db
DATABASE_ADMIN_NAME: dtf
@@ -114,7 +114,7 @@ services:
worker:
image: ${API_IMAGE:-gitea.blyzer.com.br/blyzer/dtf-api}:${IMAGE_TAG:-latest}
command: python -m local.worker
command: python -m app.worker
environment: *app-environment
networks: [backend, egress]
read_only: true
@@ -175,7 +175,7 @@ services:
configs:
clamd_config:
file: ./local/clamd.conf
file: ./infra/clamd.conf
volumes:
postgres-data:

View File

@@ -5,9 +5,12 @@ RUN apt-get update \
&& apt-get upgrade -y \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/
RUN pip install --no-cache-dir --require-hashes -r local/requirements.lock
COPY local /app/local
COPY infra/requirements.txt infra/requirements.lock /app/infra/
RUN pip install --no-cache-dir --require-hashes -r infra/requirements.lock
COPY app /app/app
# The local image carries the suites so they can run inside the stack network.
# deploy/Dockerfile.api deliberately does not: tests are not part of what ships.
COPY tests /app/tests
RUN useradd --uid 10001 --create-home dtf
USER dtf
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1

View File

@@ -4,4 +4,4 @@
# makes an empty directory instead and the container fails to start.
ARG CLAMAV_IMAGE=clamav/clamav@sha256:9cb27d7660bdf66e9878c832cb433dd8aa152cfbe16f3c2c0084c80b04ae22b4
FROM ${CLAMAV_IMAGE}
COPY local/clamd.conf /etc/clamav/clamd.conf
COPY infra/clamd.conf /etc/clamav/clamd.conf

View File

@@ -1,6 +1,6 @@
# Provisioning script and policies baked in, for the same reason as the scanner.
ARG MINIO_IMAGE=quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z
FROM ${MINIO_IMAGE}
COPY local/storage-init.sh /init.sh
COPY local/storage-policy.json /policy.json
COPY local/storage-lifecycle.json /lifecycle.json
COPY infra/storage-init.sh /init.sh
COPY infra/storage-policy.json /policy.json
COPY infra/storage-lifecycle.json /lifecycle.json

View File

@@ -1,8 +1,8 @@
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9 AS policy
WORKDIR /build
COPY web /build/web
COPY local /build/local
RUN python local/compile_web.py
COPY infra /build/infra
RUN python infra/compile_web.py
# Same pinned base as deploy/Dockerfile.web.
FROM nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8

View File

@@ -18,7 +18,7 @@ for html in (root / 'web').glob('*.html'):
if not re.search(r'\bsrc\s*=', attributes, re.I) and script.strip():
hashes.append("'sha256-" + base64.b64encode(hashlib.sha256(script.encode()).digest()).decode() + "'")
template = Path(os.environ.get('NGINX_TEMPLATE', root / 'local/nginx.conf.template')).read_text()
template = Path(os.environ.get('NGINX_TEMPLATE', root / 'infra/nginx.conf.template')).read_text()
rendered = template.replace('@SCRIPT_HASHES@', ' '.join(hashes))
# Collapse the gap an empty hash list leaves behind, so the policy reads cleanly.
rendered = re.sub(r"(script-src 'self')\s+;", r'\1;', rendered)

View File

@@ -10,4 +10,4 @@ docker run --rm \
--volume "$root:/src" \
--workdir /src \
python:3.12-slim \
sh -c 'python -m pip install --no-cache-dir --target /tmp/piptools pip==25.3 pip-tools==7.5.2 && PYTHONPATH=/tmp/piptools python -m piptools compile --generate-hashes --allow-unsafe --strip-extras --no-emit-index-url --output-file local/requirements.lock local/requirements.txt'
sh -c 'python -m pip install --no-cache-dir --target /tmp/piptools pip==25.3 pip-tools==7.5.2 && PYTHONPATH=/tmp/piptools python -m piptools compile --generate-hashes --allow-unsafe --strip-extras --no-emit-index-url --output-file infra/requirements.lock infra/requirements.txt'

1
ops/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Operational commands: backup, readiness, audit, security summary."""

View File

@@ -1,8 +1,8 @@
"""Local alert triage: prints redacted counts; exits 1 when attention is required."""
import json
from datetime import datetime, timezone
from .db import connect
from .scanning import ClamAV
from app.core.db import connect
from app.scanning import ClamAV
def scanner_status():
try:

View File

@@ -8,8 +8,8 @@ import tarfile
from datetime import datetime, timezone
from uuid import uuid4
from .adapters import LocalS3Storage, require_local
from .db import connect
from app.adapters import LocalS3Storage, require_local
from app.core.db import connect
CHUNK = 8 * 1024 * 1024
MAX_OBJECT = min(128 * 1024 * 1024, int(os.environ.get('SCAN_MAX_BYTES', '134217728')))

1
tests/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Test suites. Not shipped in the production image."""

View File

@@ -59,7 +59,7 @@ try{
await site.click('[data-cam="tabela"]');
const root=await site.call('DOM.getDocument');
const input=await site.call('DOM.querySelector',{nodeId:root.root.nodeId,selector:'#inp'});
await site.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('local/fixtures/local-test.cdr')]});
await site.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('tests/fixtures/local-test.cdr')]});
await waitFor(()=>site.eval('!!document.querySelector("[data-comp]")'),'manual length field');
await site.fill('[data-comp]','1.01','change');
await waitFor(()=>site.eval('!!itemAtual'),'cart calculation');
@@ -99,7 +99,7 @@ try{
await waitFor(()=>kanban.eval(`!!document.querySelector('[data-order="${oid}"] [data-final-item]')`),'final upload controls');
const doc=await kanban.call('DOM.getDocument');
const input=await kanban.call('DOM.querySelector',{nodeId:doc.root.nodeId,selector:`[data-order="${oid}"] [data-final-item]`});
await kanban.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('local/fixtures/local-test.cdr')]});
await kanban.call('DOM.setFileInputFiles',{nodeId:input.nodeId,files:[resolve('tests/fixtures/local-test.cdr')]});
await kanban.fill(`[data-order="${oid}"] input[placeholder="Nota da revisão"]`,'Browser test final file');
await kanban.eval(`(()=>{const form=document.querySelector('[data-order="${oid}"] form');form.querySelector('[type=checkbox]').click();form.requestSubmit();})()`);
await waitFor(()=>kanban.eval(`board.orders.find(o=>o.id==='${oid}').version===2`),'final file approval');

View File

@@ -2,9 +2,9 @@
from datetime import datetime, timedelta, timezone
from uuid import uuid4
from botocore.exceptions import ClientError
from .adapters import LocalS3Storage
from .db import connect
from .worker import cleanup
from app.adapters import LocalS3Storage
from app.core.db import connect
from app.worker import cleanup
def run():
storage=LocalS3Storage()

View File

@@ -2,10 +2,10 @@
import hashlib
from unittest.mock import patch
from botocore.exceptions import ClientError
from .db import connect
from .adapters import LocalS3Storage
from .auth import client_ip, password_hash, password_matches
from .scanning import ClamAV, require_clean
from app.core.db import connect
from app.adapters import LocalS3Storage
from app.core.auth import client_ip, password_hash, password_matches
from app.scanning import ClamAV, require_clean
from fastapi import HTTPException
class FakeRequest:
@@ -29,8 +29,8 @@ def check_client_ip():
def check_operator_accounts():
"""Accounts are per person, and disabling one ends its access at once."""
from uuid import uuid4
from .auth import password_hash, password_matches
from .operators import seed_from_environment, set_active
from app.core.auth import password_hash, password_matches
from app.operators import seed_from_environment, set_active
email='runtime-check-'+uuid4().hex[:8]+'@example.test'
with connect() as c:
c.execute('INSERT INTO dtf_local.operators(id,email,name,password_hash) VALUES(%s,%s,%s,%s)',
@@ -79,7 +79,7 @@ def run():
require_clean({'complete':True,'scan_state':'clean'})
assert ClamAV().ping() and ClamAV().version().startswith('ClamAV ')
assert ClamAV().scan(None,134217729)[0]=='rejected'
with patch('local.scanning.socket.create_connection',side_effect=OSError('offline')):
with patch('app.scanning.socket.create_connection',side_effect=OSError('offline')):
try:ClamAV().scan(None,1);raise AssertionError('Offline scanner returned success')
except OSError:pass
print('PASS: runtime DB/S3 least privilege, legacy/current password hashes, quarantine states and scanner size/offline behavior')

View File

@@ -1,6 +1,6 @@
"""Harmless EICAR anti-malware test and blocked download/quote regressions."""
from uuid import uuid4
from .smoke_test import Client, upload_bytes
from tests.smoke_test import Client, upload_bytes
def run():
customer=Client();customer.call('/session')

View File

@@ -5,7 +5,7 @@ from urllib.error import HTTPError
from urllib.request import Request, urlopen
from urllib.parse import urlparse, parse_qs
from uuid import uuid4
from .smoke_test import Client, BASE, with_host
from tests.smoke_test import Client, BASE, with_host
def raw(path, expected, headers=None, body=None):
request=Request(BASE+path, data=body, headers=with_host(headers))

View File

@@ -1,6 +1,6 @@
"""Local stack integration checks; creates and retains clearly named test orders.
Run: python3 -m local.smoke_test. Standard library only. Honors .env/environment.
Run: python3 -m tests.smoke_test. Standard library only. Honors .env/environment.
"""
from concurrent.futures import ThreadPoolExecutor
import hashlib

View File

@@ -13,8 +13,8 @@ def normalized(name):
class DependencyLockTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.direct_text = (ROOT / 'local/requirements.txt').read_text()
cls.lock_text = (ROOT / 'local/requirements.lock').read_text()
cls.direct_text = (ROOT / 'infra/requirements.txt').read_text()
cls.lock_text = (ROOT / 'infra/requirements.lock').read_text()
def test_every_direct_pin_matches_lock(self):
direct = {normalized(name): version for name, version in
@@ -34,9 +34,9 @@ class DependencyLockTests(unittest.TestCase):
self.assertTrue(hashes, f'{match.group(1)} has no SHA-256 artifact hash')
def test_image_build_requires_the_lock_and_hashes(self):
dockerfile = (ROOT / 'local/Dockerfile').read_text()
dockerfile = (ROOT / 'infra/Dockerfile').read_text()
self.assertIn('requirements.lock', dockerfile)
self.assertIn('--require-hashes -r local/requirements.lock', dockerfile)
self.assertIn('--require-hashes -r infra/requirements.lock', dockerfile)
def test_reproducible_generator_is_recorded(self):
self.assertIn('./local/lock_dependencies.sh', self.lock_text[:300])

View File

@@ -1,9 +1,9 @@
"""Run from repository root: python3 -m unittest local.test_pricing -v."""
"""Run from repository root: python3 -m unittest tests.test_pricing -v."""
import json
from pathlib import Path
import subprocess
import unittest
from .pricing import price, TIERS
from app.core.pricing import price, TIERS
class PricingTests(unittest.TestCase):
def test_every_grade_against_actual_site_javascript(self):

View File

@@ -3,7 +3,7 @@ import tempfile
import unittest
from pathlib import Path
from local.secrets import SECRET_FILE_SETTINGS, load, read_secret
from app.core.secrets import SECRET_FILE_SETTINGS, load, read_secret
class SecretFileTests(unittest.TestCase):

View File

@@ -2,7 +2,7 @@ import tempfile
import unittest
from pathlib import Path
from .staging_readiness import read_config, validate
from ops.staging_readiness import read_config, validate
VALID = '''

View File

@@ -1,7 +1,7 @@
"""Customer identity, correction and final-file trust boundaries against local stack."""
from uuid import uuid4
from urllib.request import urlopen
from .smoke_test import Client, upload_bytes
from tests.smoke_test import Client, upload_bytes
def run():
customer=Client();other=Client();customer.call('/session');other.call('/session')