Commit Graph

20 Commits

Author SHA1 Message Date
Cauê Faleiros
cfcbe545f1 ci: require manual gated releases from main
All checks were successful
Build and deploy / Validate source (push) Successful in 1m28s
Build and deploy / Integration suite on a real stack (push) Successful in 4m3s
Build and deploy / Secret scan and release gate (push) Successful in 11s
Build and deploy / Publish images and notify Portainer (push) Has been skipped
2026-09-23 11:27:18 -03:00
Cauê Faleiros
24013458c9 fix: harden week-two ordering, artwork and operations 2026-09-23 10:40:18 -03:00
Cauê Faleiros
7386469404 docs: move the engineering documents into docs/
All checks were successful
Build and deploy / Validate source (push) Successful in 5s
Build and deploy / Integration suite on a real stack (push) Successful in 1m18s
Build and deploy / Secret scan and release gate (push) Successful in 6s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m26s
Thirteen files at the repository root, seven of them documents. Only README.md
earns a place there; the rest are now in docs/ beside the meeting notes, the
client roadmap and the historical material.

The compose files stay. docker-compose.yml is the path the dtf-cloud Portainer
stack reads, so moving it would break deployment, and Docker resolves a compose
file's relative build contexts against its own directory, so moving the other
two would silently break every build. Both reasons are now written down where
someone would otherwise try it.

Correcting references turned up a live fault: the Portainer stack creation
instructions still named deploy/stack.yaml as the compose path. That file was
removed, so anyone recreating the stack from these instructions would have
failed. It names docker-compose.yml now, with the reason it stays at the root.

ROADMAP.md keeps the paths its closed findings were written with, and says so at
the top. Those entries record where a fault was when it was found; rewriting
them to match a later layout would make the record less true, not more.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 17:52:20 -03:00
Cauê Faleiros
b329f76378 refactor: lay the repository out by role
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Integration suite on a real stack (push) Successful in 1m25s
Build and deploy / Secret scan and release gate (push) Successful in 5s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m31s
local/ held six unrelated things under a name that stopped being true once it
became the production runtime: the service, the frontend, the tests, the ops
commands, the container definitions and the dependency lock, 65 files with
nothing to tell them apart.

  app/      the service: api/ routers, core/ for identity, database, models,
            prices and secret loading, and the worker, bootstrap and schema
  tests/    the twelve suites, no longer inside the shipped package
  ops/      backup, readiness, dependency audit, security summary
  infra/    Dockerfiles, gateway templates, ClamAV and storage configuration,
            the requirements and their hash lock
  web/      the Site, Kanban and portal pages with their scripts

deploy/Dockerfile.api now copies app/ alone, so the tests stop shipping to
production; the local image still carries them, because the suites run inside
the stack's network.

Five kinds of reference had to follow, and each was found by something different
rather than by reading. Imports of the form "from . import db" survived a rewrite
that only matched "from .db import". Tests kept relative imports of modules that
had left the package. A mock.patch target names its module in a string, where no
import rewriting can see it. The browser test resolves a fixture by path. And the
release gate's markers pointed at local/runtime.py and local/worker.py, which is
the decay its new marker test exists to catch — it caught it.

Verified from docker compose down -v: the stack starts, all six integration
suites, both browser suites and the twenty-nine unit tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 17:40:57 -03:00
Cauê Faleiros
c9f8122600 refactor: give the frontend its own directory and split the API into routers
The Site's page sat at the repository root while its scripts lived in
local/static, a split with no reason behind it. They are together in web/ now,
with the page as index.html, which is also what the image serves.

app.py held the adapters, the configuration, the shared query helpers and
nineteen routes; customer.py held fourteen more but could not import from it
without a cycle, so it was wired by passing nine callables into install_routes.
Configuration and shared helpers move to local/runtime.py, the rules for
attaching artwork to an order move to local/artwork.py where a customer
correction and an operator final-file set can share them, and the routes become
seven routers under local/api. app.py is 48 lines that create the application,
apply the middleware and include them. Routers import downwards only.

Three faults came out of the extraction and are worth recording, because each
passed a check that looked sufficient. ast reports a function's line at the def,
so every decorator on the line above fell outside the extracted range: twelve
routes and the security middleware were defined but never registered, and the
files still imported and parsed cleanly. Names the old closure renamed on the
way in, and a Jsonb import, were missing in three modules. A name-resolution
pass over every new module found those; the route count matching the original
exactly, 32, is what confirmed the first.

The release gate's marker for the fake payment adapter pointed at app.py and the
adapter moved to runtime.py, so the gate passed while the condition it guards was
unchanged. That is the same silent decay 2.5 set out to fix. A test now asserts
every marker still matches something in its file, so the next move fails loudly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 17:22:00 -03:00
Cauê Faleiros
da903db32a feat: serve pdf.js from this origin instead of a CDN
All checks were successful
Build and deploy / Validate source (push) Successful in 5s
Build and deploy / Integration suite on a real stack (push) Successful in 1m10s
Build and deploy / Secret scan and release gate (push) Successful in 5s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m43s
The Site pulled pdf.js 3.11.174 from cdnjs with no integrity attribute, and the
policy trusted the whole of cdnjs.cloudflare.com for both script-src and
worker-src. Anything that host served would have executed, and a customer
measuring a PDF sheet depended on it being reachable.

Vendor both files instead of pinning a hash: it removes the dependency rather
than constraining it, and lets the policy name only 'self'. Provenance and
SHA-256 digests are recorded in local/static/vendor/README.md, verified on
download against the SRI digests cdnjs publishes for that release.

cdnjs is now absent from script-src, worker-src and connect-src in both gateway
templates. Workers are 'self' plus blob:, which the Site needs for the worker it
constructs itself.

Verified in a browser against the running stack: pdf.js loads from /vendor/, the
blob worker starts, and a real seven-page PDF parses with no CSP violation. Both
browser suites and the full integration suite pass.

The version is deliberately unchanged. 3.11.174 is old, but its known eval path
is already closed by isEvalSupported:false, and upgrading is an API change that
needs its own testing rather than riding along with this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 13:56:02 -03:00
Cauê Faleiros
9926d3ab55 chore: remove the unused second stack definition
deploy/stack.yaml arrived in the first commit and was never deployed. Portainer
runs the repository's docker-compose.yml. Keeping both meant two definitions
drifting apart, with the documentation naming the one nobody used, which is how
the credential question came up at all.

The hardening it offered is narrower than it looks: Docker secrets keep values
out of docker inspect and the Portainer console, but local/secrets.py loads them
into the process environment regardless, and anyone able to read docker inspect
can already read the secret files. With a single Portainer user, the benefit that
remains does not outweigh maintaining a divergent copy.

local/secrets.py stays: inert against the deployed file, and it lets a stack
switch to Docker secrets later without touching code. The preflight and its tests
degrade cleanly when no such stack is present.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 13:48:22 -03:00
Cauê Faleiros
3b92813491 fix: recover the customer address behind the host reverse proxy
Some checks failed
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Integration suite on a real stack (push) Failing after 51s
Build and deploy / Secret scan and release gate (push) Successful in 9s
Build and deploy / Publish images and notify Portainer (push) Has been skipped
The production gateway does not face the internet: nginx-proxy-manager owns
80/443 on the host and proxies to it. So $remote_addr inside the gateway is that
proxy, and overwriting X-Forwarded-For with it discarded the customer address
the proxy had already recorded. Every request would have been attributed to one
internal address, which is exactly the fault 2.1 set out to fix, reintroduced in
production only.

Use real_ip to take the customer address from the proxy's header, trusting only
private networks. A request that reaches the published port directly from the
internet is not trusted, so its header is ignored and $remote_addr stays the
real peer: the anti-spoofing property is kept.

Also downgrade 2.9. TLS is not missing, it is terminated by that proxy. The gap
is that the repository never says so, which would break every session cookie if
the stack moved to a host without one.

Validated with nginx -t against the rendered production configuration.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 13:07:56 -03:00
Cauê Faleiros
4c9fa2436e build: pin base image digests and clear every fixable image finding
The production images built on mutable tags with --pull, so the same commit
could produce different bases, and neither Dockerfile upgraded its OS packages
even though the local ones did. The published API image carried 56 HIGH and 3
CRITICAL findings, 15 of them with an upstream fix available.

Pin both bases by digest and upgrade OS packages in the production images. That
removes all 3 CRITICAL and 13 of the 15 fixable findings. The remaining two,
msgpack and setuptools, come from a third-party SBOM; neither package is
importable or listed by pip in the built image, which I confirmed rather than
taking the previous report's word for it.

The web image could not be fixed this way: the official 1.28 line pins
nginx=1.28.3-r1 in /etc/apk/world, so apk upgrade leaves five HIGH findings in
place even though Alpine ships 1.28.3-r7. Moving to nginx:alpine (1.31.6)
clears them completely; 1.29-alpine scans worse, at 37 HIGH. Same uid 101 and
the same template entrypoint, and the local images now use the same pinned
bases so the integration suite exercises what ships. Full suite passes on
nginx 1.31.6, including the browser end-to-end.

With both images at zero CRITICAL, the image scan now blocks on CRITICAL and
reports HIGH, instead of reporting everything. PYTHON_BASE_IMAGE and
NGINX_BASE_IMAGE are wired through to the builds so a base can move forward
without editing the repository, which is what PORTAINER.md already promised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:58:32 -03:00
Cauê Faleiros
341f154c36 feat: load Docker secret files so the production stack can boot
deploy/stack.yaml passes DATABASE_URL_FILE, AWS_ACCESS_KEY_ID_FILE,
OPERATOR_PASSWORD_FILE and the provider tokens as Swarm secret paths, but the
runtime only ever read the plain names. That stack could not start: the database
URL and R2 credentials were absent, and operator login raised KeyError, so it
returned 500 instead of the intended 503.

local/secrets.py resolves every <NAME>_FILE into <NAME> before configuration is
read, from the API, worker and bootstrap entrypoints. It fails closed on an
unreadable or empty secret and on a name supplied both directly and as a file,
because starting with a credential nobody intended is worse than not starting.
Only one trailing newline is stripped, so a generated password keeps any
whitespace that belongs to it, and no value reaches an error message.

The stack also passed OPERATOR_USER while the Kanban authenticates by email;
it now passes OPERATOR_EMAIL, matching the runtime.

The release gate checked this by searching local/secrets.py for the literal
"DATABASE_URL_FILE", which would pass for any file containing that string. It
now loads the module and makes it resolve every secret the stack declares, and
asserts it fails closed on a missing one. Four marker strings that stopped
matching when R2 support landed are removed rather than left to rot; the two
that still describe real blockers stay, so the gate continues to refuse a
release while payment and messaging adapters are fake.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:36:59 -03:00
Cauê Faleiros
52ae13c9a1 fix: rate-limit and audit by real client address
uvicorn does not trust forwarded headers from a peer outside
forwarded_allow_ips, so request.client.host was the web gateway for every
request. The auth-source bucket therefore counted all customers together:
60 failed logins from one attacker locked out everyone. Security events
recorded the gateway address, which made the audit trail useless for
attribution.

The gateway now overwrites X-Forwarded-For with the peer address it observed
instead of appending to whatever the client sent, so the header carries one
value the client cannot choose, and client_ip() resolves it with a fallback to
the connection peer.

The guest-session limiter was keyed on the environment name, making it one
global bucket of 120 per 15 minutes: roughly eight new visitors a minute for
the whole site before legitimate traffic started receiving 429. It is now per
source, and the ceiling is deliberately generous because offices and mobile
carriers put many real customers behind a single address.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 18:48:31 -03:00
Cauê Faleiros
88a2a6f060 fix: prevent stale Kanban script after deployment
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Publish images and notify Portainer (push) Successful in 53s
2026-09-18 14:27:33 -03:00
Cauê Faleiros
4d707009ba feat: configure Kanban login with optional email
All checks were successful
Build and deploy / Validate source (push) Successful in 10s
Build and deploy / Publish images and notify Portainer (push) Successful in 56s
2026-09-18 14:11:55 -03:00
Cauê Faleiros
d4190ebfeb Revert "feat: configure Kanban login by operator email"
All checks were successful
Build and deploy / Validate source (push) Successful in 12s
Build and deploy / Publish images and notify Portainer (push) Successful in 54s
This reverts commit 508fa03664.
2026-09-18 13:45:08 -03:00
Cauê Faleiros
508fa03664 feat: configure Kanban login by operator email
All checks were successful
Build and deploy / Validate source (push) Successful in 13s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m2s
2026-09-18 12:54:04 -03:00
Cauê Faleiros
fbb620bd85 fix: keep web services up during API rollout 2026-09-18 12:17:34 -03:00
Cauê Faleiros
7605ca9918 fix: start web services in Portainer Swarm
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Publish images and notify Portainer (push) Successful in 43s
2026-09-18 12:02:31 -03:00
Cauê Faleiros
e3e37f674d feat: run DTF stack with Cloudflare R2
All checks were successful
Build and deploy / Validate source (push) Successful in 7s
Build and deploy / Publish images and notify Portainer (push) Successful in 44s
2026-09-18 11:51:54 -03:00
Cauê Faleiros
6c0c94373a ci: publish DTF images and notify Portainer
All checks were successful
Build and deploy / Validate source (push) Successful in 1m29s
Build and deploy / Publish images and notify Portainer (push) Successful in 1m17s
2026-09-17 17:21:21 -03:00
Cauê Faleiros
98c951d374 first commit
Some checks failed
Validate, publish and deploy / validate (push) Successful in 2m2s
Validate, publish and deploy / publish-and-deploy (push) Failing after 8s
2026-09-15 16:42:34 -03:00