docs: record secret-file loading and the behavioural release gate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.gitea/workflows/deploy.yml

@@ -21,12 +21,81 @@ jobs:
             local.test_dependency_lock \
             local.test_staging_readiness \
             deploy.test_production_preflight \
-            local.test_pricing -v
+            local.test_pricing \
+            local.test_secrets -v
           sh -n local/lock_dependencies.sh
 
+  integration:
+    name: Integration suite on a real stack
+    needs: validate
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    env:
+      SITE_PORT: "8080"
+      KANBAN_PORT: "8081"
+      API_PORT: "8000"
+      COMPOSE: docker compose -f compose.local.yaml
+    steps:
+      - name: Checkout
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+
+      # py_compile cannot see an unresolved name, and the four unit tests above
+      # never start the application. A missing import in local/auth.py therefore
+      # reached production and returned 500 on every session, login and
+      # registration. These suites exercise the running stack and would have
+      # failed on it immediately.
+      - name: Start the stack
+        run: |
+          $COMPOSE up --build -d --wait --wait-timeout 600
+          $COMPOSE ps
+
+      - name: API and workflow regressions
+        run: |
+          python3 -m local.smoke_test
+          python3 -m local.workflow_test
+          python3 -m local.security_test
+          python3 -m local.scanning_test
+
+      - name: Runtime and retention regressions
+        run: |
+          $COMPOSE exec -T api python -m local.retention_test
+          $COMPOSE exec -T api python -m local.runtime_security_test
+
+      # These need a real Chrome. They are the only coverage for the artwork
+      # editor and the full customer journey, so install google-chrome-stable
+      # (or set CHROME_BIN) on the runner to make them gate deployments. The
+      # suites above stay hard gates either way.
+      - name: Browser regressions
+        run: |
+          for candidate in "$CHROME_BIN" /usr/bin/google-chrome-stable \
+              /usr/bin/google-chrome /usr/bin/chromium /usr/bin/chromium-browser; do
+            if [ -n "$candidate" ] && [ -x "$candidate" ]; then
+              export CHROME_BIN="$candidate"
+              break
+            fi
+          done
+          if [ ! -x "${CHROME_BIN:-}" ]; then
+            echo "::warning::No Chrome on this runner; browser regressions were NOT run."
+            echo "Install google-chrome-stable or set CHROME_BIN to gate on them."
+            exit 0
+          fi
+          echo "Using $CHROME_BIN"
+          node local/artwork_browser_test.mjs
+          node local/browser_test.mjs
+
+      - name: Diagnostics on failure
+        if: failure()
+        run: |
+          $COMPOSE ps || true
+          $COMPOSE logs --tail 200 api worker site kanban || true
+
+      - name: Tear down
+        if: always()
+        run: $COMPOSE down -v || true
+
   publish-and-deploy:
     name: Publish images and notify Portainer
-    needs: validate
+    needs: [validate, integration]
     if: gitea.event_name == 'push' && gitea.ref == 'refs/heads/main'
     runs-on: ubuntu-latest
     timeout-minutes: 45

ROADMAP.md

@@ -7,9 +7,9 @@
 > Update the **Current step** line and the item status every time something moves.
 > Add new findings at the bottom of the relevant block rather than rewriting history.
 
-**Current step:** Block 0 closed on 2026-09-18 (0.1–0.6 done and verified against a
+**Current step:** Block 0 closed, plus 2.1, 2.2, 2.3, 2.5 and 5.1. Next: 2.4 (the
-live stack). Next: Block 1 needs client inputs for 1.1/1.2, so Block 2 (2.1, 2.2,
+release gate the docs describe but the workflow never ran — partly addressed by
-2.3) and 5.1 are the ones that can start immediately.
+5.1), then 2.6/2.7. Block 1 still waits on client inputs for 1.1/1.2.
 
 **Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`,
 `.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
@@ -171,7 +171,7 @@ and ships only fake adapters, so the deployed system cannot take an order at all
 
 ## Block 2 · Security — before any public exposure
 
-### `[ ]` 2.1 — Rate limiting and audit logs are blind to the client `(F5)`
+### `[x]` 2.1 — Rate limiting and audit logs are blind to the client `(F5)`
 
 uvicorn runs without trusted proxy headers (`forwarded_allow_ips` defaults to
 `127.0.0.1`; nginx is a different container IP), so `request.client.host` is nginx
@@ -184,13 +184,13 @@ record has no attacker IP.
 - **Accept:** two clients on different IPs have independent buckets; audit rows
   carry the real IP.
 
-### `[ ]` 2.2 — The public site throttles itself `(F6)`
+### `[x]` 2.2 — The public site throttles itself `(F6)`
 
 `local/app.py:115` — `rate_limit('guest-sessions', ENVIRONMENT, 120, 900)` is keyed
 on the environment name: 120 new visitors per 15 minutes **site-wide** (~8/min).
 Normal traffic 429s. Key per source IP (after 2.1) and raise the ceiling.
 
-### `[ ]` 2.3 — `deploy/stack.yaml` cannot boot `(F7)`
+### `[x]` 2.3 — `deploy/stack.yaml` cannot boot `(F7)`
 
 It passes `DATABASE_URL_FILE`, `AWS_ACCESS_KEY_ID_FILE`, `OPERATOR_PASSWORD_FILE`,
 `OPERATOR_USER`. The code reads `DATABASE_URL`, `AWS_ACCESS_KEY_ID`,
@@ -212,7 +212,7 @@ unit tests, then builds, pushes `latest` and calls the webhook **unconditionally
 
 - Either implement the gate or correct both documents. Do not leave the gap.
 
-### `[ ]` 2.5 — The preflight has silently decayed `(F9)`
+### `[x]` 2.5 — The preflight has silently decayed `(F9)`
 
 It blocks by string-matching source. **4 of 6 markers are dead** after the R2
 refactor: `'This runtime only supports APP_ENV=local'`,
@@ -332,9 +332,13 @@ charges. Fix as part of 1.1.
 
 ## Block 5 · Hygiene and maintenance
 
-- `[ ]` 5.1 — CI coverage `(F33)`. The smoke, workflow, security, retention and
+- `[x]` 5.1 — CI coverage `(F33)`. An `integration` job now builds the localhost
-  browser suites exist under `local/` and would have caught 0.1 — none run in CI.
+  stack and runs smoke, workflow, security, scanning, retention, runtime security
-  Add a compose-backed job. **Highest leverage item in this block.**
+  and both browser suites; `publish-and-deploy` depends on it. Verified by
+  reintroducing the 0.1 defect: `py_compile` and the unit tests still passed while
+  `smoke_test` failed on `/session`, blocking the release. Browser tests skip with
+  a warning when the runner has no Chrome — **install `google-chrome-stable` on the
+  runner (or set `CHROME_BIN`) to make them gate as well.**
 - `[ ]` 5.2 — Remove or archive the dead prototypes `(F30)`: `portal/`, `kanban/`,
   `agente/`, root `schema.sql` (~1,500 lines describing an abandoned model). Several
   expose unauthenticated endpoints taking the acting user from the request body
@@ -395,6 +399,36 @@ charges. Fix as part of 1.1.
   guard that refuses identical credentials in both configuration forms.
 - `[x]` Checkout confirmation moved out of the panel the success path hides.
 
+### Block 2 and CI — 2026-09-18
+
+- `[x]` The web gateway overwrites `X-Forwarded-For` with the peer address instead
+  of appending to it, and `client_ip()` resolves the requester for rate-limit
+  buckets and security events. Verified: a forged `203.0.113.99` never reaches the
+  audit trail.
+- `[x]` Guest sessions are limited per source. The first attempt used 30/IP, which
+  the new regression caught as too tight for shared NAT — recreating the original
+  fault in a narrower form — so the ceiling is 240 per 15 minutes, overridable with
+  `GUEST_SESSION_LIMIT`.
+- `[x]` Security events now carry the source address (`operator_login_failed`,
+  `customer_login_failed`, `cross_origin_rejected`, `http_security_event`).
+- `[x]` CI runs the integration suites against a real stack before publishing.
+
+### 2026-09-21
+
+- `[x]` 2.3 — `local/secrets.py` resolves every `<NAME>_FILE` into `<NAME>` from the
+  API, worker and bootstrap entrypoints, failing closed on an unreadable or empty
+  secret and on a value supplied both ways. Verified by booting the API, the worker
+  and bootstrap with credentials supplied only as mounted files, including a
+  password containing `:/?#[]&=+$ ,%`. `OPERATOR_USER` in the stack became
+  `OPERATOR_EMAIL`, which is what the runtime reads.
+- `[x]` 2.5 — The gate now loads `local/secrets.py` and makes it resolve every
+  secret `deploy/stack.yaml` declares, plus asserts it fails closed. Verified
+  against a no-op loader (11 blockers) and one that swallows a missing file
+  (1 blocker); only the real implementation passes. The four marker strings that
+  stopped matching when R2 support landed were removed; the two describing real
+  blockers stay, so the gate still refuses a release while the payment and
+  messaging adapters are fake.
+
 ### Reporting
 
 - `[x]` Week-1 client report (`Relatorio-Semana-1-DTF.docx`), corrected 2026-09-18 to

deploy/nginx.conf.template

@@ -27,7 +27,9 @@ server {
         proxy_set_header Host $host;
         proxy_set_header X-Forwarded-Host $host;
         proxy_set_header X-Forwarded-Proto https;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        # Overwrite, never append: $proxy_add_x_forwarded_for keeps any header the
+        # client sent, and the leftmost value would then be attacker-controlled.
+        proxy_set_header X-Forwarded-For $remote_addr;
         proxy_connect_timeout 5s;
         proxy_read_timeout 30s;
         client_max_body_size 2m;

deploy/production_preflight.py

@@ -17,7 +17,7 @@ REQUIRED = (
     'IMAGE_TAG', 'PUBLIC_ORIGIN', 'PUBLIC_HOST', 'KANBAN_HOST',
     'SITE_PORT', 'KANBAN_PORT',
     'R2_ENDPOINT', 'R2_PUBLIC_ENDPOINT', 'R2_BUCKET',
-    'POSTGRES_DB', 'POSTGRES_USER', 'APP_DB_USER', 'POSTGRES_VOLUME', 'OPERATOR_USER',
+    'POSTGRES_DB', 'POSTGRES_USER', 'APP_DB_USER', 'POSTGRES_VOLUME', 'OPERATOR_EMAIL',
     'STORAGE_QUOTA_BYTES', 'OWNER_UPLOAD_QUOTA_BYTES', 'MAX_UPLOAD_BYTES',
     'UPLOAD_PART_BYTES', 'MAX_PENDING_UPLOADS', 'SCAN_MAX_BYTES',
     'PAYMENT_ADAPTER', 'FREIGHT_ADAPTER', 'TINY_ADAPTER', 'WHATSAPP_ADAPTER',
@@ -33,13 +33,12 @@ IMAGE_REPOSITORY = re.compile(
 DNS = re.compile(r'^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}$')
 NAME = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9_.-]{2,127}$')
 SOURCE_BLOCKERS = {
-    'local/adapters.py': (
+    # Markers must name something that is still true, or the gate weakens without
-        'This runtime only supports APP_ENV=local',
+    # failing. Four entries here described a local-only runtime and stopped
-        'Only local S3 storage is supported',
+    # matching when R2 support landed; they were removed rather than left to rot.
-    ),
+    # What remains is the real blocker: no production payment or messaging adapter
+    # exists, so these lines must change before a release can be meaningful.
     'local/app.py': (
-        "allowed_hosts=['localhost', '127.0.0.1']",
-        "'environment': 'local'",
         'payment = FakePayment()',
     ),
     'local/worker.py': (
@@ -55,12 +54,63 @@ def source_errors(root=ROOT):
         for marker in markers:
             if marker in text:
                 errors.append(f'{relative} remains local-only: {marker}')
-    secrets_module = root / 'local' / 'secrets.py'
+    errors.extend(secret_loading_errors(root))
-    if not secrets_module.exists() or 'DATABASE_URL_FILE' not in secrets_module.read_text():
-        errors.append('local runtime does not load the production Docker secret *_FILE settings')
     return errors
 
 
+def secret_loading_errors(root=ROOT):
+    """Exercise the secret loader instead of grepping it.
+
+    Searching for a string passes as soon as someone writes that string, and
+    fails when a working implementation happens to spell it differently. Load the
+    module and make it resolve a real file.
+    """
+    import importlib.util
+    import tempfile
+
+    module_path = root / 'local' / 'secrets.py'
+    if not module_path.exists():
+        return ['local runtime does not load the production Docker secret *_FILE settings']
+    try:
+        spec = importlib.util.spec_from_file_location('_preflight_secrets', module_path)
+        module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+    except Exception as exc:
+        return [f'local/secrets.py could not be loaded: {exc}']
+
+    stack_names = set()
+    stack = root / 'deploy' / 'stack.yaml'
+    if stack.exists():
+        for line in stack.read_text().splitlines():
+            if '_FILE:' in line:
+                key = line.split(':')[0].strip()
+                # The database image consumes this one; the application does not.
+                if key and key != 'POSTGRES_PASSWORD_FILE':
+                    stack_names.add(key[:-len('_FILE')])
+
+    failures = []
+    with tempfile.TemporaryDirectory() as directory:
+        for name in sorted(stack_names):
+            path = Path(directory) / name
+            path.write_text('resolved-value\n', encoding='utf-8')
+            environ = {f'{name}_FILE': str(path)}
+            try:
+                module.load(environ)
+            except Exception as exc:
+                failures.append(f'{name}_FILE is not resolved by local/secrets.py: {exc}')
+                continue
+            if environ.get(name) != 'resolved-value':
+                failures.append(f'{name}_FILE did not produce {name}')
+        # A missing secret must stop the service, never start it unconfigured.
+        try:
+            module.load({'DATABASE_URL_FILE': str(Path(directory) / 'absent')})
+        except Exception:
+            pass
+        else:
+            failures.append('local/secrets.py does not fail closed on an unreadable secret')
+    return failures
+
+
 def config_errors(values):
     errors = []
     for name in APPROVALS:

deploy/stack.yaml

@@ -9,7 +9,8 @@ x-app-environment: &app-environment
   AWS_ACCESS_KEY_ID_FILE: /run/secrets/r2_access_key_id
   AWS_SECRET_ACCESS_KEY_FILE: /run/secrets/r2_secret_access_key
   AWS_DEFAULT_REGION: auto
-  OPERATOR_USER: ${OPERATOR_USER:?set OPERATOR_USER}
+  # The Kanban authenticates by email; the runtime reads OPERATOR_EMAIL.
+  OPERATOR_EMAIL: ${OPERATOR_EMAIL:?set OPERATOR_EMAIL}
   OPERATOR_PASSWORD_FILE: /run/secrets/operator_password
   PAYMENT_ADAPTER: ${PAYMENT_ADAPTER:?set PAYMENT_ADAPTER}
   FREIGHT_ADAPTER: ${FREIGHT_ADAPTER:?set FREIGHT_ADAPTER}

deploy/test_production_preflight.py

@@ -27,7 +27,7 @@ def valid_config():
         'POSTGRES_USER': 'dtf_admin',
         'APP_DB_USER': 'dtf_app',
         'POSTGRES_VOLUME': 'dtf-postgres-data',
-        'OPERATOR_USER': 'dtf-operator',
+        'OPERATOR_EMAIL': 'operador@example.com',
         'STORAGE_QUOTA_BYTES': '53687091200',
         'OWNER_UPLOAD_QUOTA_BYTES': '10737418240',
         'MAX_UPLOAD_BYTES': '5368709120',

local/app.py

@@ -14,12 +14,14 @@ from starlette.middleware.trustedhost import TrustedHostMiddleware
 from psycopg.types.json import Jsonb
 
 from . import db
+from .secrets import load as load_secret_files
 from .adapters import FakeFreight, FakePayment, LocalS3Storage, require_runtime
 from .models import Freight, Move, Pay, QuoteRequest, Review, UploadStart, OperatorLogin
 from .pricing import price
-from .auth import COOKIE_SECURE, owner, session_row, new_session, operator, throttle, audit, rate_limit
+from .auth import COOKIE_SECURE, client_ip, owner, session_row, new_session, operator, throttle, audit, rate_limit
 from .scanning import require_clean
 
+load_secret_files()
 require_runtime()
 storage = LocalS3Storage()
 payment = FakePayment()
@@ -28,6 +30,10 @@ ENVIRONMENT = os.environ.get('APP_ENV', 'local')
 PUBLIC_ORIGIN = os.environ.get('PUBLIC_ORIGIN', 'http://localhost')
 ALLOWED_HOSTS = [host for host in os.environ.get('ALLOWED_HOSTS', 'localhost,127.0.0.1').split(',') if host]
 ALLOWED_ORIGINS = [origin for origin in os.environ.get('ALLOWED_ORIGINS', PUBLIC_ORIGIN).split(',') if origin]
+# Per source and generous: a browser needs one session and keeps the cookie, but
+# offices and mobile carriers put many real customers behind one address, so a
+# tight per-IP ceiling would lock out the same people the old global one did.
+GUEST_SESSION_LIMIT = int(os.environ.get('GUEST_SESSION_LIMIT', '240'))
 PART_BYTES = int(os.environ.get('UPLOAD_PART_BYTES', '8388608'))
 if not 5242880 <= PART_BYTES <= 67108864:
     raise RuntimeError('UPLOAD_PART_BYTES must be between 5 and 64 MiB')
@@ -56,7 +62,7 @@ def operator_login(body: OperatorLogin, request: Request, response: Response):
     valid_user = secrets.compare_digest(email.encode(), configured_email.encode())
     valid_password = secrets.compare_digest(body.password.encode(), os.environ['OPERATOR_PASSWORD'].encode())
     if not (valid_user and valid_password):
-        audit('operator_login_failed')
+        audit('operator_login_failed', ip=client_ip(request))
         raise HTTPException(401, 'Invalid operator login')
     token = secrets.token_urlsafe(32)
     with db.connect() as c:
@@ -84,11 +90,11 @@ async def safe_headers(request, call_next):
     if request.method not in ('GET','HEAD','OPTIONS'):
         origin = request.headers.get('origin')
         if request.headers.get('sec-fetch-site') == 'cross-site' or (origin and origin not in ALLOWED_ORIGINS):
-            audit('cross_origin_rejected')
+            audit('cross_origin_rejected', ip=client_ip(request))
             return JSONResponse({'detail':'Cross-origin request rejected'}, status_code=403)
     response = await call_next(request)
     if response.status_code in (401,403,429) or response.status_code>=500:
-        audit('http_security_event', method=request.method, status=response.status_code)
+        audit('http_security_event', method=request.method, status=response.status_code, ip=client_ip(request))
     response.headers['Cache-Control'] = 'no-store'
     response.headers['X-Content-Type-Options'] = 'nosniff'
     response.headers['Referrer-Policy'] = 'no-referrer'
@@ -111,7 +117,9 @@ def session(request: Request, response: Response):
     try:
         session_id = owner(request)
     except HTTPException:
-        rate_limit('guest-sessions', ENVIRONMENT, 120, 900)
+        # Per source, not per deployment: keyed on the environment name this was a
+        # single global bucket, so ~8 new visitors a minute exhausted it site-wide.
+        rate_limit('guest-sessions', client_ip(request), GUEST_SESSION_LIMIT, 900)
         with db.connect() as c:
             session_id = new_session(c, response)
     return {'environment': ENVIRONMENT, 'cart_scope': str(session_id), 'part_bytes': PART_BYTES,

local/auth.py

@@ -10,6 +10,20 @@ from .db import connect
 
 COOKIE_SECURE = os.environ.get('COOKIE_SECURE', 'false').lower() == 'true'
 
+def client_ip(request: Request):
+    """The requester's address, for rate-limit buckets and security events.
+
+    The API is only reachable through the web gateway, which replaces
+    X-Forwarded-For with the peer address it observed, so the header carries
+    exactly one value the client could not choose. Without this every request
+    looks like the gateway, which collapses per-source limits into one global
+    bucket and strips the address from the audit trail.
+    """
+    forwarded = request.headers.get('x-forwarded-for', '').split(',')[0].strip()
+    if forwarded:
+        return forwarded[:64]
+    return request.client.host if request.client else 'unknown'
+
 def password_hash(password, salt=None):
     salt = salt or secrets.token_hex(16)
     digest = hashlib.scrypt(password.encode(), salt=bytes.fromhex(salt), n=16384, r=8, p=5).hex()
@@ -82,7 +96,7 @@ def rate_limit(scope, identity, limit, seconds=900):
 
 def throttle(email, request):
     # Independent account and source buckets prevent bypass by rotating emails.
-    rate_limit('auth-source', request.client.host if request.client else 'local', 60)
+    rate_limit('auth-source', client_ip(request), 60)
     rate_limit('auth-account', email, 10)
 
 def operator(request: Request):

local/bootstrap.py

@@ -4,6 +4,7 @@ from pathlib import Path
 from urllib.parse import urlparse
 import psycopg
 from psycopg import sql
+from .secrets import load as load_secret_files
 
 
 def admin_connect():
@@ -25,6 +26,7 @@ def admin_password():
 
 
 def main():
+    load_secret_files()
     role = os.environ['APP_DB_USER']
     password = os.environ['APP_DB_PASSWORD']
     if password == admin_password():

local/customer.py

@@ -5,7 +5,7 @@ from fastapi import Depends, HTTPException, Request, Response
 from psycopg.errors import UniqueViolation
 from psycopg.types.json import Jsonb
 from . import db
-from .auth import owner, session_row, new_session, password_hash, password_matches, transfer_guest, throttle, DUMMY_PASSWORD_HASH, audit
+from .auth import owner, session_row, new_session, password_hash, password_matches, transfer_guest, throttle, DUMMY_PASSWORD_HASH, audit, client_ip
 from .models import Register, Login, UploadStart, ArtworkSubmission
 from .scanning import require_clean
 
@@ -45,7 +45,7 @@ def install_routes(app, operator, storage, begin, status, part, complete, upload
         stored = account['password_hash'] if account else DUMMY_PASSWORD_HASH
         matches = password_matches(body.password, stored)
         if not account or not matches:
-            audit('customer_login_failed')
+            audit('customer_login_failed', ip=client_ip(request))
             raise HTTPException(401, 'Invalid email or password')
         previous = current(request)
         with db.connect() as c:

local/nginx.conf.template

@@ -16,6 +16,7 @@ server {
         limit_req_status 429;
         proxy_pass http://api:8000;
         proxy_set_header Host $http_host;
+        proxy_set_header X-Forwarded-For $remote_addr;
         client_max_body_size 2m;
     }
     location / { try_files $uri $uri/ =404; }
@@ -30,5 +31,6 @@ server {
         limit_req_status 429;
         proxy_pass http://api:8000;
         proxy_set_header Host $http_host;
+        proxy_set_header X-Forwarded-For $remote_addr;
     }
 }

local/runtime_security_test.py

@@ -4,11 +4,30 @@ from unittest.mock import patch
 from botocore.exceptions import ClientError
 from .db import connect
 from .adapters import LocalS3Storage
-from .auth import password_hash, password_matches
+from .auth import client_ip, password_hash, password_matches
 from .scanning import ClamAV, require_clean
 from fastapi import HTTPException
 
+class FakeRequest:
+    def __init__(self, headers=None, peer='10.0.0.2'):
+        self.headers=headers or {}
+        self.client=type('C',(),{'host':peer})() if peer else None
+
+def check_client_ip():
+    """The gateway hands one address; a direct peer falls back to its own."""
+    # Gateway-set header wins over the connection peer, which is the gateway.
+    assert client_ip(FakeRequest({'x-forwarded-for':'198.51.100.9'}))=='198.51.100.9'
+    # Only the first entry is used, and it is length-capped.
+    assert client_ip(FakeRequest({'x-forwarded-for':'198.51.100.9, 10.0.0.2'}))=='198.51.100.9'
+    assert len(client_ip(FakeRequest({'x-forwarded-for':'a'*500})))<=64
+    # No header: the peer address, never a constant shared by every request.
+    assert client_ip(FakeRequest(peer='192.0.2.5'))=='192.0.2.5'
+    assert client_ip(FakeRequest({'x-forwarded-for':'   '},peer='192.0.2.5'))=='192.0.2.5'
+    assert client_ip(FakeRequest(peer=None))=='unknown'
+    print('PASS: client address resolution for rate-limit buckets and audit events')
+
 def run():
+    check_client_ip()
     with connect() as c:
         role=c.execute('SELECT rolsuper,rolcreatedb,rolcreaterole,rolbypassrls FROM pg_roles WHERE rolname=current_user').fetchone()
         assert not any(role.values()),role

local/secrets.py

@@ -0,0 +1,76 @@
+"""Resolve Docker secret files into the environment before configuration is read.
+
+Swarm mounts each secret as a file and the stack passes its path as `<NAME>_FILE`.
+Nothing read `_FILE` settings, so `deploy/stack.yaml` could not boot: the runtime
+looked for `DATABASE_URL`, `AWS_ACCESS_KEY_ID` and `OPERATOR_PASSWORD` while the
+stack supplied only the `_FILE` form.
+
+Call `load()` in every entrypoint before any configuration is read.
+
+Note for readers: this module is `local.secrets`. Python 3 resolves `import
+secrets` elsewhere in the package to the standard library, not to this file.
+"""
+import os
+
+# The settings production supplies as secret files. Any other `*_FILE` variable is
+# resolved the same way; this list documents the contract and is what the release
+# gate checks against, so keep it in step with `deploy/stack.yaml`.
+SECRET_FILE_SETTINGS = (
+    'DATABASE_URL',
+    'DATABASE_ADMIN_URL',
+    'DATABASE_PASSWORD',
+    'DATABASE_ADMIN_PASSWORD',
+    'APP_DB_PASSWORD',
+    'AWS_ACCESS_KEY_ID',
+    'AWS_SECRET_ACCESS_KEY',
+    'OPERATOR_PASSWORD',
+    'PAYMENT_TOKEN',
+    'PAYMENT_WEBHOOK_SECRET',
+    'TINY_TOKEN',
+    'WHATSAPP_TOKEN',
+)
+
+SUFFIX = '_FILE'
+
+
+def read_secret(path):
+    """One secret's value, without the newline an editor or `docker secret` adds.
+
+    Only a single trailing newline is removed: everything else is part of the
+    value, because a generated password may legitimately end in whitespace.
+    """
+    with open(path, 'r', encoding='utf-8') as handle:
+        value = handle.read()
+    if value.endswith('\r\n'):
+        return value[:-2]
+    if value.endswith('\n'):
+        return value[:-1]
+    return value
+
+
+def load(environ=None):
+    """Replace every `<NAME>_FILE` path with `<NAME>` holding the file's contents.
+
+    Fails closed. An unreadable secret, an empty one, or a name supplied both
+    directly and as a file is a configuration error, and starting anyway would
+    mean running with a credential nobody intended. Never logs a value.
+    """
+    environ = os.environ if environ is None else environ
+    resolved = []
+    for key in sorted(k for k in environ if k.endswith(SUFFIX) and len(k) > len(SUFFIX)):
+        name = key[:-len(SUFFIX)]
+        path = environ[key].strip()
+        if not path:
+            raise RuntimeError(f'{key} is set but empty; point it at a secret file')
+        if environ.get(name):
+            raise RuntimeError(
+                f'{name} and {key} are both set; supply the value or the file, not both')
+        try:
+            value = read_secret(path)
+        except OSError as exc:
+            raise RuntimeError(f'{key} could not be read: {exc.strerror}') from None
+        if not value:
+            raise RuntimeError(f'{key} points at an empty secret file')
+        environ[name] = value
+        resolved.append(name)
+    return resolved

local/security_test.py

@@ -63,4 +63,15 @@ def run():
     attacker.call('/operator/login',{'email':email,'password':'invalid'},expected=429)
     print('PASS: operator login throttling (only synthetic account bucket exhausted)')
 
+    # Guest sessions are limited per source, not once for the whole deployment.
+    # Keyed on the environment name this was a single global bucket of 120 per
+    # 15 minutes, which the suites above would already have eaten into.
+    for _ in range(25):
+        Client().call('/session')
+    # A forged forwarded address must not let a client pick another bucket: the
+    # gateway overwrites the header, so these count against the real source too.
+    for _ in range(5):
+        raw('/api/session',200,{'X-Forwarded-For':'203.0.113.7'})
+    print('PASS: guest sessions limited per source, forwarded address not client-controlled')
+
 if __name__=='__main__':run()

local/test_secrets.py

@@ -0,0 +1,93 @@
+"""Docker secret-file resolution. Standard library only; no stack required."""
+import tempfile
+import unittest
+from pathlib import Path
+
+from local.secrets import SECRET_FILE_SETTINGS, load, read_secret
+
+
+class SecretFileTests(unittest.TestCase):
+    def setUp(self):
+        self.dir = tempfile.TemporaryDirectory()
+        self.addCleanup(self.dir.cleanup)
+
+    def secret(self, content, name='secret'):
+        path = Path(self.dir.name) / name
+        path.write_text(content, encoding='utf-8')
+        return str(path)
+
+    def test_resolves_file_into_plain_setting(self):
+        env = {'DATABASE_URL_FILE': self.secret('postgresql://u:p@db:5432/dtf\n')}
+        self.assertEqual(load(env), ['DATABASE_URL'])
+        self.assertEqual(env['DATABASE_URL'], 'postgresql://u:p@db:5432/dtf')
+
+    def test_strips_only_one_trailing_newline(self):
+        # A generated password may legitimately end in whitespace, so only the
+        # newline `docker secret` or an editor appends may be removed.
+        self.assertEqual(read_secret(self.secret('p@ss \n')), 'p@ss ')
+        self.assertEqual(read_secret(self.secret('p@ss\n\n')), 'p@ss\n')
+        self.assertEqual(read_secret(self.secret('p@ss\r\n')), 'p@ss')
+        self.assertEqual(read_secret(self.secret('p@ss')), 'p@ss')
+
+    def test_preserves_characters_that_would_break_a_url(self):
+        value = 'p@ss:w/rd?#[]&=+$ ,%'
+        env = {'OPERATOR_PASSWORD_FILE': self.secret(value + '\n')}
+        load(env)
+        self.assertEqual(env['OPERATOR_PASSWORD'], value)
+
+    def test_resolves_every_documented_production_setting(self):
+        env = {f'{name}_FILE': self.secret(f'value-for-{name}', name)
+               for name in SECRET_FILE_SETTINGS}
+        load(env)
+        for name in SECRET_FILE_SETTINGS:
+            self.assertEqual(env[name], f'value-for-{name}')
+
+    def test_missing_file_fails_closed(self):
+        env = {'DATABASE_URL_FILE': str(Path(self.dir.name) / 'absent')}
+        with self.assertRaises(RuntimeError) as caught:
+            load(env)
+        self.assertIn('DATABASE_URL_FILE', str(caught.exception))
+        self.assertNotIn('DATABASE_URL', env)
+
+    def test_empty_secret_fails_closed(self):
+        with self.assertRaises(RuntimeError):
+            load({'OPERATOR_PASSWORD_FILE': self.secret('\n')})
+
+    def test_empty_path_fails_closed(self):
+        with self.assertRaises(RuntimeError):
+            load({'OPERATOR_PASSWORD_FILE': '   '})
+
+    def test_value_and_file_together_is_ambiguous(self):
+        env = {'OPERATOR_PASSWORD': 'inline',
+               'OPERATOR_PASSWORD_FILE': self.secret('from-file')}
+        with self.assertRaises(RuntimeError):
+            load(env)
+        self.assertEqual(env['OPERATOR_PASSWORD'], 'inline')
+
+    def test_never_puts_a_secret_in_the_error_text(self):
+        value = 'super-secret-value'
+        env = {'TINY_TOKEN': value, 'TINY_TOKEN_FILE': self.secret(value)}
+        with self.assertRaises(RuntimeError) as caught:
+            load(env)
+        self.assertNotIn(value, str(caught.exception))
+
+    def test_ignores_a_bare_suffix_and_leaves_other_settings_alone(self):
+        env = {'_FILE': '/nowhere', 'APP_ENV': 'production'}
+        self.assertEqual(load(env), [])
+        self.assertEqual(env['APP_ENV'], 'production')
+
+    def test_documented_list_matches_the_production_stack(self):
+        stack = Path(__file__).resolve().parent.parent / 'deploy' / 'stack.yaml'
+        if not stack.exists():
+            self.skipTest('production stack definition not present')
+        text = stack.read_text()
+        # POSTGRES_PASSWORD_FILE is consumed by the database image, not by us.
+        used = {line.split(':')[0].strip() for line in text.splitlines()
+                if '_FILE:' in line and 'POSTGRES_PASSWORD_FILE' not in line}
+        for key in used:
+            self.assertIn(key[:-len('_FILE')], SECRET_FILE_SETTINGS,
+                          f'{key} is passed by the stack but undocumented in secrets.py')
+
+
+if __name__ == '__main__':
+    unittest.main()

local/worker.py

@@ -6,9 +6,11 @@ import time
 from http.server import BaseHTTPRequestHandler, HTTPServer
 from psycopg.types.json import Jsonb
 from .adapters import FakeTiny, FakeWhatsApp, LocalS3Storage, require_runtime
+from .secrets import load as load_secret_files
 from .db import connect
 from .scanning import ClamAV, scan_loop
 
+load_secret_files()
 require_runtime()
 adapters = {'tiny': FakeTiny(), 'whatsapp': FakeWhatsApp()}
 last_tick = 0.0