blyzer/dtf-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit
Some checks failed
Validate, publish and deploy / validate (push) Successful in 2m2s
Details
Validate, publish and deploy / publish-and-deploy (push) Failing after 8s
Details

Browse Source
This commit is contained in:
Cauê Faleiros
2026-09-15 16:42:34 -03:00
commit 98c951d374
170 changed files with 95988 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
deploy/Dockerfile.api Normal file
View File

@@ -0,0 +1,18 @@
# syntax=docker/dockerfile:1
ARG PYTHON_BASE_IMAGE
FROM ${PYTHON_BASE_IMAGE}
ARG VCS_REF=unknown
LABEL org.opencontainers.image.title="DTF Portal/API" \
org.opencontainers.image.revision="$VCS_REF" \
org.opencontainers.image.source="DTF System repository"
WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/
RUN python -m pip install --no-cache-dir --require-hashes -r local/requirements.lock
COPY local /app/local
RUN useradd --uid 10001 --create-home --shell /usr/sbin/nologin dtf
USER 10001:10001
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PYTHONPATH=/app
EXPOSE 8000
CMD ["uvicorn", "local.app:app", "--host", "0.0.0.0", "--port", "8000", "--no-access-log"]

22
deploy/Dockerfile.web Normal file
View File

@@ -0,0 +1,22 @@
# syntax=docker/dockerfile:1
ARG PYTHON_BASE_IMAGE
ARG NGINX_BASE_IMAGE
FROM ${PYTHON_BASE_IMAGE} AS policy
WORKDIR /build
COPY dtf-site.html /build/dtf-site.html
COPY local /build/local
COPY deploy /build/deploy
ENV NGINX_TEMPLATE=/build/deploy/nginx.conf.template
RUN python local/compile_web.py
FROM ${NGINX_BASE_IMAGE}
ARG VCS_REF=unknown
LABEL org.opencontainers.image.title="DTF Site and Kanban" \
org.opencontainers.image.revision="$VCS_REF" \
org.opencontainers.image.source="DTF System repository"
ENV WEB_INDEX=index.html PUBLIC_HOST=invalid.example S3_PUBLIC_ENDPOINT=https://invalid.example
COPY --from=policy /build/default.conf.template /etc/nginx/templates/default.conf.template
COPY dtf-site.html /usr/share/nginx/html/index.html
COPY local/static/ /usr/share/nginx/html/
USER 101:101
EXPOSE 8080

46
deploy/PRODUCTION_CHECKLIST.md Normal file
View File

@@ -0,0 +1,46 @@
# Production release checklist
Every item is required. A checked box records reviewed evidence; it is not a
substitute for `production_preflight.py`, CI, staging acceptance, or change approval.
## Application and external contracts
- [ ] `PRODUCTION_INPUTS.md` has owners, decisions, and evidence for every item.
- [ ] Production R2, freight, Mercado Pago, Tiny/Olist, and WhatsApp adapters have
sandbox contract tests and least-privilege credentials.
- [ ] Payment webhook authenticity, replay handling, and idempotency are tested.
- [ ] Docker secret `*_FILE` loading is implemented and tested without logging values.
- [ ] Production account verification/recovery and the length/grade authority flow
are approved.
- [ ] No factory automation or automatic print pre-flight was added by inference.
## Platform and recovery
- [ ] DNS/TLS proxy routes and firewall rules are approved; only Site and Kanban
have published web ports.
- [ ] External, versioned Swarm secrets exist and match the configured names.
- [ ] The PostgreSQL volume is encrypted/backed up and pinned to the labeled node.
- [ ] Scheduled offsite database/object backups and an isolated restore rehearsal pass.
- [ ] ClamAV signatures are current and have a controlled update/rebuild process.
- [ ] Central alerts, logs, clocks, capacity, and on-call ownership are verified.
## Release and deploy
- [ ] Protected Gitea runner, `main` branch, registry permissions, and variables are reviewed.
- [ ] Base, database, and scanner images use approved immutable digests; application
SHA tags remain pullable and the digest resolved by each deployment is recorded.
- [ ] Full regressions and production preflight pass on the exact release commit.
- [ ] HIGH/CRITICAL Trivy findings are fixed or formally risk-accepted with evidence.
- [ ] Staging acceptance passes with provider sandboxes and production-like topology.
- [ ] Four production approval variables equal `approved` only after their reviews.
- [ ] The generated `docker stack config` contains no secret values or placeholders.
- [ ] Health probes, monitoring, rollback, and a backup restore are observed in staging.
- [ ] The Portainer webhook redeploys the one `dtf-cloud` stack and post-deploy
HTTPS health probes pass.
## Rollback
- [ ] Previous application image digests remain pullable.
- [ ] Portainer rollback by full commit `IMAGE_TAG` has been rehearsed; it does
not roll back the database.
- [ ] Database migration forward/restore ownership and maintenance procedure are approved.

16
deploy/README.md Normal file
View File

@@ -0,0 +1,16 @@
# Production deployment files
The DTF application is one Portainer-owned Docker Swarm stack. Gitea builds,
tests, scans, and publishes the two application images, then calls the stack's
Portainer webhook. Start with the short operator guide in `../PORTAINER.md`.
- `stack.yaml` — the single Portainer stack.
- `Dockerfile.api` and `Dockerfile.web` — prebuilt registry images.
- `portainer.env.example` — non-secret Portainer variables.
- `production_preflight.py` — fail-closed application/configuration validator.
- `PRODUCTION_CHECKLIST.md` — production evidence checklist.
The current application remains deliberately blocked from production because
real adapters and Docker-secret file loading are absent and current validation
base images have unresolved HIGH/CRITICAL findings. No real provider or
production service has been configured or contacted.

1
deploy/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Production deployment validation package."""

31
deploy/nginx.conf.template Normal file
View File

@@ -0,0 +1,31 @@
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=20r/s;
server {
listen 8080;
server_name ${PUBLIC_HOST};
if ($host != ${PUBLIC_HOST}) { return 400; }
root /usr/share/nginx/html;
index ${WEB_INDEX};
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Content-Type-Options nosniff always;
add_header Referrer-Policy no-referrer always;
add_header X-Frame-Options DENY always;
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
add_header Content-Security-Policy "default-src 'self'; script-src 'self' @SCRIPT_HASHES@ https://cdnjs.cloudflare.com; script-src-attr 'none'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self' data: blob: https://cdn.vnda.com.br; connect-src 'self' ${S3_PUBLIC_ENDPOINT} https://cdn.vnda.com.br https://cdnjs.cloudflare.com; worker-src 'self' blob: https://cdnjs.cloudflare.com; object-src 'none'; base-uri 'none'; frame-ancestors 'none'; form-action 'self'" always;
location = /health { access_log off; return 200 'ok'; }
location /api/ {
limit_req zone=api_limit burst=100 nodelay;
limit_req_status 429;
proxy_pass http://api:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
client_max_body_size 2m;
}
location / { try_files $uri $uri/ =404; }
}

51
deploy/portainer.env.example Normal file
View File

@@ -0,0 +1,51 @@
# Non-secret Portainer stack variables. Never put credential values in this file.
PRODUCTION_DEPLOY_ENABLED=TBD
PRODUCTION_INPUTS_APPROVED=TBD
PRODUCTION_SECURITY_REVIEW_APPROVED=TBD
PRODUCTION_RESTORE_REHEARSED=TBD
API_IMAGE=gitea.blyzer.com.br/blyzer/dtf-api
WEB_IMAGE=gitea.blyzer.com.br/blyzer/dtf-web
IMAGE_TAG=latest
POSTGRES_IMAGE=postgres@sha256:0000000000000000000000000000000000000000000000000000000000000000
CLAMAV_IMAGE=clamav/clamav@sha256:0000000000000000000000000000000000000000000000000000000000000000
PUBLIC_ORIGIN=https://dtf.example.invalid
PUBLIC_HOST=dtf.example.invalid
KANBAN_HOST=kanban-dtf.example.invalid
SITE_PORT=8080
KANBAN_PORT=8081
R2_ENDPOINT=TBD
R2_PUBLIC_ENDPOINT=TBD
R2_BUCKET=TBD
POSTGRES_DB=dtf
POSTGRES_USER=dtf_admin
APP_DB_USER=dtf_app
POSTGRES_VOLUME=TBD
OPERATOR_USER=TBD
PAYMENT_ADAPTER=TBD
FREIGHT_ADAPTER=TBD
TINY_ADAPTER=TBD
WHATSAPP_ADAPTER=TBD
STORAGE_QUOTA_BYTES=TBD
OWNER_UPLOAD_QUOTA_BYTES=TBD
MAX_PENDING_UPLOADS=10
MAX_UPLOAD_BYTES=5368709120
UPLOAD_PART_BYTES=8388608
SCAN_MAX_BYTES=134217728
# Names of external Portainer/Docker Swarm secrets, never their values.
DATABASE_URL_SECRET=TBD
DATABASE_ADMIN_URL_SECRET=TBD
DB_ADMIN_PASSWORD_SECRET=TBD
APP_DB_PASSWORD_SECRET=TBD
R2_ACCESS_KEY_ID_SECRET=TBD
R2_SECRET_ACCESS_KEY_SECRET=TBD
OPERATOR_PASSWORD_SECRET=TBD
PAYMENT_TOKEN_SECRET=TBD
PAYMENT_WEBHOOK_SECRET=TBD
TINY_TOKEN_SECRET=TBD
WHATSAPP_TOKEN_SECRET=TBD

159
deploy/production_preflight.py Normal file
View File

@@ -0,0 +1,159 @@
"""Fail closed until the application and non-secret production inputs are ready."""
import os
from pathlib import Path
import re
import sys
from urllib.parse import urlparse
ROOT = Path(__file__).resolve().parent.parent
APPROVALS = (
'PRODUCTION_DEPLOY_ENABLED',
'PRODUCTION_INPUTS_APPROVED',
'PRODUCTION_SECURITY_REVIEW_APPROVED',
'PRODUCTION_RESTORE_REHEARSED',
)
REQUIRED = (
'API_IMAGE', 'WEB_IMAGE', 'POSTGRES_IMAGE', 'CLAMAV_IMAGE',
'IMAGE_TAG', 'PUBLIC_ORIGIN', 'PUBLIC_HOST', 'KANBAN_HOST',
'SITE_PORT', 'KANBAN_PORT',
'R2_ENDPOINT', 'R2_PUBLIC_ENDPOINT', 'R2_BUCKET',
'POSTGRES_DB', 'POSTGRES_USER', 'APP_DB_USER', 'POSTGRES_VOLUME', 'OPERATOR_USER',
'STORAGE_QUOTA_BYTES', 'OWNER_UPLOAD_QUOTA_BYTES', 'MAX_UPLOAD_BYTES',
'UPLOAD_PART_BYTES', 'MAX_PENDING_UPLOADS', 'SCAN_MAX_BYTES',
'PAYMENT_ADAPTER', 'FREIGHT_ADAPTER', 'TINY_ADAPTER', 'WHATSAPP_ADAPTER',
'DATABASE_URL_SECRET', 'DATABASE_ADMIN_URL_SECRET',
'DB_ADMIN_PASSWORD_SECRET', 'APP_DB_PASSWORD_SECRET',
'R2_ACCESS_KEY_ID_SECRET', 'R2_SECRET_ACCESS_KEY_SECRET',
'OPERATOR_PASSWORD_SECRET', 'PAYMENT_TOKEN_SECRET',
'PAYMENT_WEBHOOK_SECRET', 'TINY_TOKEN_SECRET', 'WHATSAPP_TOKEN_SECRET',
)
IMMUTABLE_IMAGE = re.compile(r'^[a-z0-9][a-z0-9._:/-]*@sha256:([0-9a-f]{64})$')
IMAGE_REPOSITORY = re.compile(
r'^[a-z0-9][a-z0-9.-]*(?::[0-9]{1,5})?(?:/[a-z0-9][a-z0-9._-]*)+$')
DNS = re.compile(r'^(?=.{1,253}$)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}$')
NAME = re.compile(r'^[a-zA-Z0-9][a-zA-Z0-9_.-]{2,127}$')
SOURCE_BLOCKERS = {
'local/adapters.py': (
'This runtime only supports APP_ENV=local',
'Only local S3 storage is supported',
),
'local/app.py': (
"allowed_hosts=['localhost', '127.0.0.1']",
"'environment': 'local'",
'payment = FakePayment()',
),
'local/worker.py': (
"adapters = {'tiny': FakeTiny(), 'whatsapp': FakeWhatsApp()}",
),
}
def source_errors(root=ROOT):
errors = []
for relative, markers in SOURCE_BLOCKERS.items():
text = (root / relative).read_text()
for marker in markers:
if marker in text:
errors.append(f'{relative} remains local-only: {marker}')
secrets_module = root / 'local' / 'secrets.py'
if not secrets_module.exists() or 'DATABASE_URL_FILE' not in secrets_module.read_text():
errors.append('local runtime does not load the production Docker secret *_FILE settings')
return errors
def config_errors(values):
errors = []
for name in APPROVALS:
if values.get(name) != 'approved':
errors.append(f'{name} must equal approved')
for name in REQUIRED:
value = values.get(name, '')
if not value or value.lower() in {'tbd', 'todo', 'replace-me', 'unconfirmed'}:
errors.append(f'{name} is missing or unresolved')
for name in ('API_IMAGE', 'WEB_IMAGE'):
if not IMAGE_REPOSITORY.fullmatch(values.get(name, '')):
errors.append(f'{name} must be a lowercase registry repository without a tag')
for name in ('POSTGRES_IMAGE', 'CLAMAV_IMAGE'):
match = IMMUTABLE_IMAGE.fullmatch(values.get(name, ''))
if not match or match.group(1) == '0' * 64:
errors.append(f'{name} must be an immutable non-placeholder digest reference')
image_tag = values.get('IMAGE_TAG', '')
if image_tag != 'latest' and not re.fullmatch(r'(?:[0-9a-f]{40}|[0-9a-f]{64})', image_tag):
errors.append('IMAGE_TAG must be latest or a full commit SHA published by Gitea')
origin = urlparse(values.get('PUBLIC_ORIGIN', ''))
if (origin.scheme != 'https' or not origin.hostname or origin.path not in ('', '/')
or origin.params or origin.query or origin.fragment):
errors.append('PUBLIC_ORIGIN must be an HTTPS origin without a path')
for name in ('PUBLIC_HOST', 'KANBAN_HOST'):
if not DNS.fullmatch(values.get(name, '')):
errors.append(f'{name} must be a valid lowercase DNS hostname')
if origin.hostname and values.get('PUBLIC_HOST') != origin.hostname:
errors.append('PUBLIC_ORIGIN hostname must equal PUBLIC_HOST')
for name in ('R2_ENDPOINT', 'R2_PUBLIC_ENDPOINT'):
endpoint = urlparse(values.get(name, ''))
host = endpoint.hostname or ''
if (endpoint.scheme != 'https' or not host.endswith('.r2.cloudflarestorage.com')
or endpoint.path not in ('', '/') or endpoint.query or endpoint.fragment):
errors.append(f'{name} must be an HTTPS Cloudflare R2 S3 API endpoint')
bucket = values.get('R2_BUCKET', '')
if bucket and not re.fullmatch(r'[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]', bucket):
errors.append('R2_BUCKET must be a valid S3 bucket name')
for name in ('PAYMENT_ADAPTER', 'FREIGHT_ADAPTER', 'TINY_ADAPTER', 'WHATSAPP_ADAPTER'):
if values.get(name, '').lower() in {'', 'fake', 'mock', 'local'}:
errors.append(f'{name} must select an approved non-fake implementation')
for name in REQUIRED:
if name.endswith('_SECRET') and values.get(name) and not NAME.fullmatch(values[name]):
errors.append(f'{name} must name a pre-provisioned external Swarm secret')
secret_names = [values.get(name, '') for name in REQUIRED if name.endswith('_SECRET')]
populated_secret_names = [name for name in secret_names if name]
if len(populated_secret_names) != len(set(populated_secret_names)):
errors.append('Every production secret setting must use a distinct external Swarm secret')
if values.get('POSTGRES_USER') == values.get('APP_DB_USER'):
errors.append('Database administrator and runtime user must differ')
if values.get('PUBLIC_HOST') == values.get('KANBAN_HOST'):
errors.append('Site and Kanban hosts must differ')
numeric = {}
for name in (
'STORAGE_QUOTA_BYTES', 'OWNER_UPLOAD_QUOTA_BYTES', 'MAX_UPLOAD_BYTES',
'UPLOAD_PART_BYTES', 'MAX_PENDING_UPLOADS', 'SCAN_MAX_BYTES'):
try:
numeric[name] = int(values.get(name, '0'))
if numeric[name] <= 0:
raise ValueError
except ValueError:
errors.append(f'{name} must be a positive integer')
if numeric.get('OWNER_UPLOAD_QUOTA_BYTES', 0) > numeric.get('STORAGE_QUOTA_BYTES', 0):
errors.append('OWNER_UPLOAD_QUOTA_BYTES cannot exceed STORAGE_QUOTA_BYTES')
if numeric.get('SCAN_MAX_BYTES', 0) > numeric.get('MAX_UPLOAD_BYTES', 0):
errors.append('SCAN_MAX_BYTES cannot exceed MAX_UPLOAD_BYTES')
ports = {}
for name in ('SITE_PORT', 'KANBAN_PORT'):
try:
ports[name] = int(values.get(name, '0'))
if not 1024 <= ports[name] <= 65535:
raise ValueError
except ValueError:
errors.append(f'{name} must be an unprivileged TCP port from 1024 to 65535')
if ports.get('SITE_PORT') == ports.get('KANBAN_PORT'):
errors.append('SITE_PORT and KANBAN_PORT must differ')
return errors
def main(source_only=False):
errors = source_errors()
if not source_only:
errors.extend(config_errors(os.environ))
if errors:
print('BLOCKED: production preflight failed:')
for error in errors:
print(f'- {error}')
return 2
print('PASS: production source and non-secret deployment metadata passed preflight.')
print('This does not replace staging acceptance, image scanning, or human approval.')
return 0
if __name__ == '__main__':
if len(sys.argv) > 2 or (len(sys.argv) == 2 and sys.argv[1] != '--source-only'):
raise SystemExit('usage: python deploy/production_preflight.py [--source-only]')
raise SystemExit(main(len(sys.argv) == 2))

309
deploy/stack.yaml Normal file
View File

@@ -0,0 +1,309 @@
version: "3.8"
x-app-environment: &app-environment
APP_ENV: production
DATABASE_URL_FILE: /run/secrets/database_url
S3_ENDPOINT: ${R2_ENDPOINT:?set R2_ENDPOINT}
S3_PUBLIC_ENDPOINT: ${R2_PUBLIC_ENDPOINT:?set R2_PUBLIC_ENDPOINT}
S3_BUCKET: ${R2_BUCKET:?set R2_BUCKET}
AWS_ACCESS_KEY_ID_FILE: /run/secrets/r2_access_key_id
AWS_SECRET_ACCESS_KEY_FILE: /run/secrets/r2_secret_access_key
AWS_DEFAULT_REGION: auto
OPERATOR_USER: ${OPERATOR_USER:?set OPERATOR_USER}
OPERATOR_PASSWORD_FILE: /run/secrets/operator_password
PAYMENT_ADAPTER: ${PAYMENT_ADAPTER:?set PAYMENT_ADAPTER}
FREIGHT_ADAPTER: ${FREIGHT_ADAPTER:?set FREIGHT_ADAPTER}
TINY_ADAPTER: ${TINY_ADAPTER:?set TINY_ADAPTER}
WHATSAPP_ADAPTER: ${WHATSAPP_ADAPTER:?set WHATSAPP_ADAPTER}
STORAGE_ADAPTER: s3-r2
PAYMENT_TOKEN_FILE: /run/secrets/payment_token
PAYMENT_WEBHOOK_SECRET_FILE: /run/secrets/payment_webhook_secret
TINY_TOKEN_FILE: /run/secrets/tiny_token
WHATSAPP_TOKEN_FILE: /run/secrets/whatsapp_token
PUBLIC_ORIGIN: ${PUBLIC_ORIGIN:?set PUBLIC_ORIGIN}
PUBLIC_HOST: ${PUBLIC_HOST:?set PUBLIC_HOST}
ALLOWED_HOSTS: ${PUBLIC_HOST:?set PUBLIC_HOST},${KANBAN_HOST:?set KANBAN_HOST}
ALLOWED_ORIGINS: ${PUBLIC_ORIGIN:?set PUBLIC_ORIGIN},https://${KANBAN_HOST:?set KANBAN_HOST}
COOKIE_SECURE: "true"
MAX_UPLOAD_BYTES: ${MAX_UPLOAD_BYTES:-5368709120}
UPLOAD_PART_BYTES: ${UPLOAD_PART_BYTES:-8388608}
STORAGE_QUOTA_BYTES: ${STORAGE_QUOTA_BYTES:?set STORAGE_QUOTA_BYTES}
OWNER_UPLOAD_QUOTA_BYTES: ${OWNER_UPLOAD_QUOTA_BYTES:?set OWNER_UPLOAD_QUOTA_BYTES}
MAX_PENDING_UPLOADS: ${MAX_PENDING_UPLOADS:-10}
SCAN_MAX_BYTES: ${SCAN_MAX_BYTES:-134217728}
x-app-secrets: &app-secrets
- database_url
- r2_access_key_id
- r2_secret_access_key
- operator_password
- payment_token
- payment_webhook_secret
- tiny_token
- whatsapp_token
x-rolling: &rolling
update_config:
parallelism: 1
delay: 10s
order: start-first
failure_action: rollback
monitor: 45s
rollback_config:
parallelism: 1
delay: 5s
order: start-first
failure_action: pause
monitor: 45s
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 5
window: 60s
services:
db:
image: ${POSTGRES_IMAGE:?set POSTGRES_IMAGE}
environment:
POSTGRES_DB: ${POSTGRES_DB:?set POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER:?set POSTGRES_USER}
POSTGRES_PASSWORD_FILE: /run/secrets/db_admin_password
secrets: [db_admin_password]
volumes:
- postgres-data:/var/lib/postgresql/data
networks: [backend]
healthcheck:
test: [CMD-SHELL, 'pg_isready -U "$$POSTGRES_USER" -d "$$POSTGRES_DB"']
interval: 10s
timeout: 5s
retries: 12
start_period: 20s
stop_grace_period: 60s
deploy:
replicas: 1
placement:
constraints: [node.labels.dtf_database == true]
update_config:
parallelism: 1
order: stop-first
failure_action: rollback
monitor: 60s
rollback_config:
parallelism: 1
order: stop-first
failure_action: pause
monitor: 60s
restart_policy:
condition: on-failure
delay: 10s
max_attempts: 5
window: 120s
resources:
limits: {cpus: "2.0", memory: 4G}
reservations: {cpus: "0.5", memory: 1G}
db-init:
image: ${API_IMAGE:?set API_IMAGE}:${IMAGE_TAG:-latest}
command: python -m local.bootstrap
environment:
APP_ENV: production
DATABASE_ADMIN_URL_FILE: /run/secrets/database_admin_url
APP_DB_USER: ${APP_DB_USER:?set APP_DB_USER}
APP_DB_PASSWORD_FILE: /run/secrets/app_db_password
secrets: [database_admin_url, app_db_password]
networks: [backend]
deploy:
replicas: 1
restart_policy: {condition: none}
placement:
constraints: [node.platform.os == linux]
resources:
limits: {cpus: "0.5", memory: 512M}
scanner:
image: ${CLAMAV_IMAGE:?set CLAMAV_IMAGE}
user: "100:101"
entrypoint: [clamd, --foreground=true, --config-file=/etc/clamav/clamd.conf]
configs:
- source: clamd_config
target: /etc/clamav/clamd.conf
mode: 0444
networks: [backend]
read_only: true
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
tmpfs:
- /tmp:uid=100,gid=101,mode=0750
- /run/clamav:uid=100,gid=101,mode=0750
- /var/log/clamav:uid=100,gid=101,mode=0750
healthcheck:
test: [CMD, clamdscan, --config-file=/etc/clamav/clamd.conf, --ping, "3"]
interval: 15s
timeout: 5s
retries: 20
start_period: 90s
deploy:
replicas: 1
restart_policy: {condition: on-failure, delay: 10s}
resources:
limits: {cpus: "2.0", memory: 3G}
reservations: {cpus: "0.5", memory: 1G}
api:
image: ${API_IMAGE:?set API_IMAGE}:${IMAGE_TAG:-latest}
environment: *app-environment
secrets: *app-secrets
networks: [backend, egress]
read_only: true
tmpfs: [/tmp]
init: true
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
healthcheck:
test:
- CMD-SHELL
- >-
python -c "import os,urllib.request; r=urllib.request.Request('http://localhost:8000/health',headers={'Host':os.environ['PUBLIC_HOST']}); urllib.request.urlopen(r,timeout=3)"
interval: 10s
timeout: 5s
retries: 12
start_period: 30s
stop_grace_period: 30s
deploy:
<<: *rolling
replicas: 2
resources:
limits: {cpus: "1.0", memory: 1G}
reservations: {cpus: "0.25", memory: 256M}
worker:
image: ${API_IMAGE:?set API_IMAGE}:${IMAGE_TAG:-latest}
command: python -m local.worker
environment:
<<: *app-environment
CLAMD_HOST: scanner
secrets: *app-secrets
networks: [backend, egress]
read_only: true
tmpfs: [/tmp]
init: true
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
healthcheck:
test: [CMD, python, -c, "import urllib.request; urllib.request.urlopen('http://localhost:8002/health',timeout=3)"]
interval: 15s
timeout: 5s
retries: 12
start_period: 90s
stop_grace_period: 60s
deploy:
<<: *rolling
replicas: 1
update_config:
parallelism: 1
order: stop-first
failure_action: rollback
monitor: 60s
rollback_config:
parallelism: 1
order: stop-first
failure_action: pause
monitor: 60s
resources:
limits: {cpus: "1.5", memory: 2G}
reservations: {cpus: "0.25", memory: 512M}
site:
image: ${WEB_IMAGE:?set WEB_IMAGE}:${IMAGE_TAG:-latest}
environment:
WEB_INDEX: index.html
PUBLIC_HOST: ${PUBLIC_HOST:?set PUBLIC_HOST}
S3_PUBLIC_ENDPOINT: ${R2_PUBLIC_ENDPOINT:?set R2_PUBLIC_ENDPOINT}
networks: [backend]
ports:
- target: 8080
published: ${SITE_PORT:-8080}
protocol: tcp
mode: ingress
read_only: true
tmpfs:
- /tmp:uid=101,gid=101,mode=0750
- /var/cache/nginx:uid=101,gid=101,mode=0750
- /var/run:uid=101,gid=101,mode=0750
- /etc/nginx/conf.d:uid=101,gid=101,mode=0750
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
healthcheck:
test: [CMD-SHELL, 'wget -q --header="Host: $$PUBLIC_HOST" -O /dev/null http://127.0.0.1:8080/health']
interval: 10s
timeout: 5s
retries: 12
start_period: 15s
deploy:
<<: *rolling
replicas: 2
resources:
limits: {cpus: "0.5", memory: 256M}
reservations: {cpus: "0.1", memory: 64M}
kanban:
image: ${WEB_IMAGE:?set WEB_IMAGE}:${IMAGE_TAG:-latest}
environment:
WEB_INDEX: kanban.html
PUBLIC_HOST: ${KANBAN_HOST:?set KANBAN_HOST}
S3_PUBLIC_ENDPOINT: ${R2_PUBLIC_ENDPOINT:?set R2_PUBLIC_ENDPOINT}
networks: [backend]
ports:
- target: 8080
published: ${KANBAN_PORT:-8081}
protocol: tcp
mode: ingress
read_only: true
tmpfs:
- /tmp:uid=101,gid=101,mode=0750
- /var/cache/nginx:uid=101,gid=101,mode=0750
- /var/run:uid=101,gid=101,mode=0750
- /etc/nginx/conf.d:uid=101,gid=101,mode=0750
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
healthcheck:
test: [CMD-SHELL, 'wget -q --header="Host: $$PUBLIC_HOST" -O /dev/null http://127.0.0.1:8080/health']
interval: 10s
timeout: 5s
retries: 12
start_period: 15s
deploy:
<<: *rolling
replicas: 1
resources:
limits: {cpus: "0.5", memory: 256M}
reservations: {cpus: "0.1", memory: 64M}
configs:
clamd_config:
file: ../local/clamd.conf
secrets:
database_url: {external: true, name: "${DATABASE_URL_SECRET:?set DATABASE_URL_SECRET}"}
database_admin_url: {external: true, name: "${DATABASE_ADMIN_URL_SECRET:?set DATABASE_ADMIN_URL_SECRET}"}
db_admin_password: {external: true, name: "${DB_ADMIN_PASSWORD_SECRET:?set DB_ADMIN_PASSWORD_SECRET}"}
app_db_password: {external: true, name: "${APP_DB_PASSWORD_SECRET:?set APP_DB_PASSWORD_SECRET}"}
r2_access_key_id: {external: true, name: "${R2_ACCESS_KEY_ID_SECRET:?set R2_ACCESS_KEY_ID_SECRET}"}
r2_secret_access_key: {external: true, name: "${R2_SECRET_ACCESS_KEY_SECRET:?set R2_SECRET_ACCESS_KEY_SECRET}"}
operator_password: {external: true, name: "${OPERATOR_PASSWORD_SECRET:?set OPERATOR_PASSWORD_SECRET}"}
payment_token: {external: true, name: "${PAYMENT_TOKEN_SECRET:?set PAYMENT_TOKEN_SECRET}"}
payment_webhook_secret: {external: true, name: "${PAYMENT_WEBHOOK_SECRET:?set PAYMENT_WEBHOOK_SECRET}"}
tiny_token: {external: true, name: "${TINY_TOKEN_SECRET:?set TINY_TOKEN_SECRET}"}
whatsapp_token: {external: true, name: "${WHATSAPP_TOKEN_SECRET:?set WHATSAPP_TOKEN_SECRET}"}
volumes:
postgres-data:
external: true
name: ${POSTGRES_VOLUME:?set POSTGRES_VOLUME}
networks:
backend:
driver: overlay
internal: true
egress:
driver: overlay

98
deploy/test_production_preflight.py Normal file
View File

@@ -0,0 +1,98 @@
import unittest
from .production_preflight import config_errors, source_errors
def valid_config():
digest = '1' * 64
values = {
'PRODUCTION_DEPLOY_ENABLED': 'approved',
'PRODUCTION_INPUTS_APPROVED': 'approved',
'PRODUCTION_SECURITY_REVIEW_APPROVED': 'approved',
'PRODUCTION_RESTORE_REHEARSED': 'approved',
'API_IMAGE': 'registry.example.com/dropstar/dtf-api',
'WEB_IMAGE': 'registry.example.com/dropstar/dtf-web',
'IMAGE_TAG': 'latest',
'POSTGRES_IMAGE': f'postgres@sha256:{digest}',
'CLAMAV_IMAGE': f'clamav/clamav@sha256:{digest}',
'PUBLIC_ORIGIN': 'https://dtf.example.com',
'PUBLIC_HOST': 'dtf.example.com',
'KANBAN_HOST': 'kanban-dtf.example.com',
'SITE_PORT': '8080',
'KANBAN_PORT': '8081',
'R2_ENDPOINT': 'https://account.r2.cloudflarestorage.com',
'R2_PUBLIC_ENDPOINT': 'https://account.r2.cloudflarestorage.com',
'R2_BUCKET': 'dtf-production-artwork',
'POSTGRES_DB': 'dtf',
'POSTGRES_USER': 'dtf_admin',
'APP_DB_USER': 'dtf_app',
'POSTGRES_VOLUME': 'dtf-postgres-data',
'OPERATOR_USER': 'dtf-operator',
'STORAGE_QUOTA_BYTES': '53687091200',
'OWNER_UPLOAD_QUOTA_BYTES': '10737418240',
'MAX_UPLOAD_BYTES': '5368709120',
'UPLOAD_PART_BYTES': '8388608',
'MAX_PENDING_UPLOADS': '10',
'SCAN_MAX_BYTES': '134217728',
'PAYMENT_ADAPTER': 'mercado-pago',
'FREIGHT_ADAPTER': 'approved-freight',
'TINY_ADAPTER': 'tiny-olist',
'WHATSAPP_ADAPTER': 'approved-whatsapp',
}
for name in (
'DATABASE_URL_SECRET', 'DATABASE_ADMIN_URL_SECRET',
'DB_ADMIN_PASSWORD_SECRET', 'APP_DB_PASSWORD_SECRET',
'R2_ACCESS_KEY_ID_SECRET', 'R2_SECRET_ACCESS_KEY_SECRET',
'OPERATOR_PASSWORD_SECRET', 'PAYMENT_TOKEN_SECRET',
'PAYMENT_WEBHOOK_SECRET', 'TINY_TOKEN_SECRET', 'WHATSAPP_TOKEN_SECRET',
):
values[name] = 'dtf_prod_' + name.lower()
return values
class ProductionPreflightTests(unittest.TestCase):
def test_structurally_complete_metadata_passes(self):
self.assertEqual(config_errors(valid_config()), [])
def test_mutable_images_and_fake_adapters_fail(self):
values = valid_config()
values['API_IMAGE'] = 'registry.example.com/dropstar/dtf-api:latest'
values['PAYMENT_ADAPTER'] = 'fake'
errors = config_errors(values)
self.assertTrue(any('API_IMAGE' in error for error in errors))
self.assertTrue(any('PAYMENT_ADAPTER' in error for error in errors))
def test_image_tag_and_public_ports_are_validated(self):
values = valid_config()
values['IMAGE_TAG'] = 'main'
values['KANBAN_PORT'] = values['SITE_PORT']
errors = config_errors(values)
self.assertTrue(any('IMAGE_TAG' in error for error in errors))
self.assertTrue(any('must differ' in error for error in errors))
def test_approval_and_secret_name_are_enforced(self):
values = valid_config()
values['PRODUCTION_DEPLOY_ENABLED'] = 'yes'
values['DATABASE_URL_SECRET'] = '../unsafe'
errors = config_errors(values)
self.assertTrue(any('PRODUCTION_DEPLOY_ENABLED' in error for error in errors))
self.assertTrue(any('DATABASE_URL_SECRET' in error for error in errors))
def test_current_application_is_explicitly_blocked(self):
errors = source_errors()
self.assertTrue(any('local/adapters.py remains local-only' in error for error in errors))
self.assertTrue(any('local/app.py remains local-only' in error for error in errors))
def test_secret_reuse_and_incoherent_limits_are_rejected(self):
values = valid_config()
values['PAYMENT_TOKEN_SECRET'] = values['TINY_TOKEN_SECRET']
values['OWNER_UPLOAD_QUOTA_BYTES'] = str(int(values['STORAGE_QUOTA_BYTES']) + 1)
values['SCAN_MAX_BYTES'] = str(int(values['MAX_UPLOAD_BYTES']) + 1)
errors = config_errors(values)
self.assertTrue(any('distinct external Swarm secret' in error for error in errors))
self.assertIn('OWNER_UPLOAD_QUOTA_BYTES cannot exceed STORAGE_QUOTA_BYTES', errors)
self.assertIn('SCAN_MAX_BYTES cannot exceed MAX_UPLOAD_BYTES', errors)
if __name__ == '__main__':
unittest.main()