Compare commits

...

4 Commits

Author SHA1 Message Date
Cauê Faleiros
010c2a162f docs: record base image pinning and the CRITICAL image gate
Some checks failed
Build and deploy / Validate source (push) Successful in 6s
Build and deploy / Integration suite on a real stack (push) Failing after 6s
Build and deploy / Secret scan and release gate (push) Successful in 11s
Build and deploy / Publish images and notify Portainer (push) Has been skipped
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:58:46 -03:00
Cauê Faleiros
4c9fa2436e build: pin base image digests and clear every fixable image finding
The production images built on mutable tags with --pull, so the same commit
could produce different bases, and neither Dockerfile upgraded its OS packages
even though the local ones did. The published API image carried 56 HIGH and 3
CRITICAL findings, 15 of them with an upstream fix available.

Pin both bases by digest and upgrade OS packages in the production images. That
removes all 3 CRITICAL and 13 of the 15 fixable findings. The remaining two,
msgpack and setuptools, come from a third-party SBOM; neither package is
importable or listed by pip in the built image, which I confirmed rather than
taking the previous report's word for it.

The web image could not be fixed this way: the official 1.28 line pins
nginx=1.28.3-r1 in /etc/apk/world, so apk upgrade leaves five HIGH findings in
place even though Alpine ships 1.28.3-r7. Moving to nginx:alpine (1.31.6)
clears them completely; 1.29-alpine scans worse, at 37 HIGH. Same uid 101 and
the same template entrypoint, and the local images now use the same pinned
bases so the integration suite exercises what ships. Full suite passes on
nginx 1.31.6, including the browser end-to-end.

With both images at zero CRITICAL, the image scan now blocks on CRITICAL and
reports HIGH, instead of reporting everything. PYTHON_BASE_IMAGE and
NGINX_BASE_IMAGE are wired through to the builds so a base can move forward
without editing the repository, which is what PORTAINER.md already promised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:58:32 -03:00
Cauê Faleiros
bbcc8ab100 docs: record the release gates and what they do not cover
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:45:23 -03:00
Cauê Faleiros
9da2a7dcad ci: add the release gates, and document the ones that do not gate
PORTAINER.md and SECURITY_REPORT.md described a pipeline that required
regressions, HIGH/CRITICAL secret, misconfiguration and image gates, and stated
that the source preflight stopped this application from publishing. None of it
ran: the workflow built and called the webhook unconditionally.

Add a blocking Trivy secret scan. Verified both ways: a planted AWS key pair,
GitHub token and private key block the job, and the repository passes clean.
Note that Trivy allowlists documented example credentials, so this gate is a
backstop, not permission to commit secrets.

The source preflight now runs on every push and always prints its verdict, but
enforces only when ENFORCE_PRODUCTION_PREFLIGHT is true. Enforcing it today
would block every deployment, because it refuses a release while the payment
and messaging adapters are fake, which is the deliberate state the stack runs
in. Set the variable when real adapters land.

Image vulnerabilities are reported after each build rather than enforced. The
current bases carry 56 HIGH and 3 CRITICAL findings, only 15 of them with an
upstream fix, so failing on them would stop releases without making anything
safer. Pinning digests and triaging the fixable ones is ROADMAP 2.6.

Both documents now carry a table of what gates and what does not, instead of
describing checks that did not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 11:45:11 -03:00
8 changed files with 191 additions and 24 deletions

View File

@@ -93,12 +93,61 @@ jobs:
if: always()
run: $COMPOSE down -v || true
scan:
name: Secret scan and release gate
needs: validate
runs-on: ubuntu-latest
timeout-minutes: 30
env:
TRIVY_IMAGE: ${{ vars.TRIVY_IMAGE }}
ENFORCE_PRODUCTION_PREFLIGHT: ${{ vars.ENFORCE_PRODUCTION_PREFLIGHT }}
steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
# Blocking. A credential committed by accident must never reach the
# registry or the deployed stack, and the repository is clean today, so
# this gate costs nothing until it is actually needed.
- name: Secret scan
run: |
image="${TRIVY_IMAGE:-aquasec/trivy:0.58.1}"
docker run --rm -v "$PWD:/src:ro" "$image" \
fs --scanners secret --exit-code 1 --severity HIGH,CRITICAL \
--no-progress /src
# PORTAINER.md described this as blocking publication. It never ran at
# all, and turning it on unconditionally would block every deploy: the
# source preflight refuses a release while the payment and messaging
# adapters are fake, which is the deliberate state the stack runs in
# today. So its verdict is always printed, and enforcement is opt-in.
# Set the repository variable ENFORCE_PRODUCTION_PREFLIGHT to "true" once
# real adapters land, and this becomes the gate the documentation claims.
- name: Production source preflight
run: |
set +e
python3 deploy/production_preflight.py --source-only
verdict=$?
set -e
if [ "$verdict" -eq 0 ]; then
echo "Source preflight passes."
exit 0
fi
if [ "${ENFORCE_PRODUCTION_PREFLIGHT:-false}" = "true" ]; then
echo "::error::Source preflight blocked the release."
exit "$verdict"
fi
echo "::warning::Source preflight reports blockers (advisory; set ENFORCE_PRODUCTION_PREFLIGHT=true to gate)."
publish-and-deploy:
name: Publish images and notify Portainer
needs: [validate, integration]
needs: [validate, integration, scan]
if: gitea.event_name == 'push' && gitea.ref == 'refs/heads/main'
runs-on: ubuntu-latest
timeout-minutes: 45
env:
TRIVY_IMAGE: ${{ vars.TRIVY_IMAGE }}
PYTHON_BASE_IMAGE: ${{ vars.PYTHON_BASE_IMAGE }}
NGINX_BASE_IMAGE: ${{ vars.NGINX_BASE_IMAGE }}
steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
@@ -114,7 +163,12 @@ jobs:
- name: Build and publish API
run: |
image="gitea.blyzer.com.br/blyzer/dtf-api"
docker build --pull --file deploy/Dockerfile.api \
# The Dockerfiles pin digests themselves; these variables let a base be
# moved forward without editing the repository. --pull is intentionally
# absent: a digest already names one immutable image.
set --
[ -n "$PYTHON_BASE_IMAGE" ] && set -- --build-arg PYTHON_BASE_IMAGE="$PYTHON_BASE_IMAGE"
docker build --file deploy/Dockerfile.api "$@" \
--build-arg VCS_REF="${{ gitea.sha }}" \
--tag "$image:latest" --tag "$image:${{ gitea.sha }}" .
docker push "$image:latest"
@@ -122,11 +176,40 @@ jobs:
- name: Build and publish web
run: |
image="gitea.blyzer.com.br/blyzer/dtf-web"
docker build --pull --file deploy/Dockerfile.web \
set --
[ -n "$PYTHON_BASE_IMAGE" ] && set -- --build-arg PYTHON_BASE_IMAGE="$PYTHON_BASE_IMAGE"
[ -n "$NGINX_BASE_IMAGE" ] && set -- "$@" --build-arg NGINX_BASE_IMAGE="$NGINX_BASE_IMAGE"
docker build --file deploy/Dockerfile.web "$@" \
--build-arg VCS_REF="${{ gitea.sha }}" \
--tag "$image:latest" --tag "$image:${{ gitea.sha }}" .
docker push "$image:latest"
docker push "$image:${{ gitea.sha }}"
# CRITICAL blocks, HIGH is reported. Both images carry zero CRITICAL after
# the base pinning and OS upgrades, so this gate holds the line already
# reached. The remaining HIGH findings have no upstream fix, so failing on
# them would stop releases without making anything safer.
- name: Image vulnerabilities
run: |
image="${TRIVY_IMAGE:-aquasec/trivy:0.58.1}"
failed=0
for target in \
"gitea.blyzer.com.br/blyzer/dtf-api:${{ gitea.sha }}" \
"gitea.blyzer.com.br/blyzer/dtf-web:${{ gitea.sha }}"; do
echo "--- $target (HIGH, reported)"
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock "$image" \
image --scanners vuln --severity HIGH --no-progress \
--format table --exit-code 0 "$target" ||
echo "::warning::Could not scan $target for HIGH findings"
echo "--- $target (CRITICAL, blocking)"
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock "$image" \
image --scanners vuln --severity CRITICAL --no-progress \
--format table --exit-code 1 "$target" || failed=1
done
if [ "$failed" -ne 0 ]; then
echo "::error::A CRITICAL vulnerability was found in a published image."
exit 1
fi
- name: Trigger Portainer redeployment
env:
PORTAINER_WEBHOOK: ${{ secrets.PORTAINER_WEBHOOK }}

View File

@@ -10,10 +10,38 @@ Cloudflare R2, so MinIO is not part of this stack.
## 1. Gitea configuration
The single workflow is `.gitea/workflows/deploy.yml`. Pull requests run static
validation. A push to `main` runs the full isolated test suite, builds and scans
the production images, publishes both `latest` and the full commit SHA, then
calls Portainer.
The single workflow is `.gitea/workflows/deploy.yml`. Every push and pull request
runs static validation, the integration suite against a real stack, and a secret
scan. A push to `main` then builds the production images, publishes both `latest`
and the full commit SHA, reports their vulnerabilities, and calls Portainer.
What actually gates a deployment:
| Check | Gates? |
|---|---|
| `py_compile` and the unit tests | yes |
| Integration suite on a live stack (smoke, workflow, security, scanning, retention, runtime) | yes |
| Browser suites | only when the runner has Chrome; otherwise warns and continues |
| Trivy secret scan (HIGH/CRITICAL) | yes |
| Source preflight (`deploy/production_preflight.py --source-only`) | only when `ENFORCE_PRODUCTION_PREFLIGHT` is `true` |
| Trivy image vulnerabilities, CRITICAL | yes |
| Trivy image vulnerabilities, HIGH | no — reported after the build |
The source preflight is advisory by default because it refuses a release while
the payment and messaging adapters are fake, which is the deliberate state the
stack runs in today. Enforcing it now would block every deployment. Set the
repository variable `ENFORCE_PRODUCTION_PREFLIGHT` to `true` once real adapters
land, and it becomes a hard gate.
CRITICAL image findings block. Both images carry none: the bases are pinned by
digest, both Dockerfiles upgrade their OS packages, and the web image moved off
the 1.28 nginx line, which pins `nginx=1.28.3-r1` in `/etc/apk/world` and so
cannot be patched in place. HIGH findings are reported rather than enforced
because the remainder have no upstream fix.
`PYTHON_BASE_IMAGE` and `NGINX_BASE_IMAGE` override the digests pinned in the
Dockerfiles. Update the Dockerfile default in the same change, so the repository
still records what a build used.
Repository variables:
@@ -27,9 +55,10 @@ Repository secrets:
- `REGISTRY_USERNAME` and `REGISTRY_TOKEN` — package write credentials.
- `PORTAINER_WEBHOOK` — webhook generated by the `dtf-cloud` Portainer stack.
The webhook is called only after tests and HIGH/CRITICAL secret,
misconfiguration, and image gates pass. The current local-only application
fails the source preflight intentionally, so it cannot publish yet.
The webhook is called only after the gating checks in the table above pass.
`ENFORCE_PRODUCTION_PREFLIGHT` and `TRIVY_IMAGE` are optional repository
variables; without them the preflight is advisory and a pinned default scanner
image is used.
## 2. One-time Portainer resources

View File

@@ -7,9 +7,9 @@
> Update the **Current step** line and the item status every time something moves.
> Add new findings at the bottom of the relevant block rather than rewriting history.
**Current step:** Block 0 closed, plus 2.1, 2.2, 2.3, 2.5 and 5.1. Next: 2.4 (the
release gate the docs describe but the workflow never ran — partly addressed by
5.1), then 2.6/2.7. Block 1 still waits on client inputs for 1.1/1.2.
**Current step:** Block 0 closed, plus 2.1–2.6 and 5.1. Next: 2.7 (vendor or
integrity-pin pdf.js), then 2.8–2.11. Block 1 still waits on client inputs for
1.1/1.2.
**Last audit:** 2026-09-18, full read of `local/`, `dtf-site.html`, `deploy/`,
`.gitea/`, docs and legacy prototypes. Findings below carry their audit IDs.
@@ -202,7 +202,7 @@ It passes `DATABASE_URL_FILE`, `AWS_ACCESS_KEY_ID_FILE`, `OPERATOR_PASSWORD_FILE
- **Accept:** the stack renders and boots against Swarm secrets; missing operator
config yields 503, not 500.
### `[ ]` 2.4 — The documented release gate does not exist `(F8)`
### `[x]` 2.4 — The documented release gate does not exist `(F8)`
`PORTAINER.md` and `SECURITY_REPORT.md` claim the workflow runs the full isolated
suite, Trivy HIGH/CRITICAL image gates, secret scanning and the source preflight
@@ -222,7 +222,7 @@ refactor: `'This runtime only supports APP_ENV=local'`,
- Replace marker matching with behavioural assertions (import the module, assert
the adapter classes in use).
### `[ ]` 2.6 — Base images are not pinned `(F10)`
### `[x]` 2.6 — Base images are not pinned `(F10)`
Dockerfiles default to mutable `python:3.12-slim` / `nginx:1.28-alpine`, the
workflow passes no digest build-args, and `--pull` makes builds non-reproducible —
@@ -429,6 +429,35 @@ charges. Fix as part of 1.1.
blockers stay, so the gate still refuses a release while the payment and
messaging adapters are fake.
- `[x]` 2.4 — A blocking Trivy secret scan was added and verified both ways: a
planted AWS key pair, GitHub token and private key block the job; the repository
passes clean. Worth knowing: Trivy allowlists documented example credentials, so
my first probe passed with AWS's own sample keys — the gate is a backstop, not
permission to commit secrets. The source preflight now runs and always prints its
verdict, enforcing only when `ENFORCE_PRODUCTION_PREFLIGHT` is `true`; enforcing
it today would block every deploy, since it refuses a release while the adapters
are fake. Image vulnerabilities are reported after each build, not enforced —
56 HIGH and 3 CRITICAL, only 15 with an upstream fix. `PORTAINER.md` and
`SECURITY_REPORT.md` now carry a table of what gates and what does not, replacing
descriptions of checks that never ran.
- `[x]` 2.6 — Both bases pinned by digest, OS packages upgraded in the production
images, and the web image moved off the nginx 1.28 line.
| Image | Before | After |
|---|---|---|
| API | 56 HIGH, 3 CRITICAL (15 fixable) | 46 HIGH, 0 CRITICAL |
| Web | 5 HIGH, all unfixable in place | 0 HIGH, 0 CRITICAL |
The 1.28 nginx pins `nginx=1.28.3-r1` in `/etc/apk/world`, so `apk upgrade`
cannot patch it even though Alpine ships `-r7`; `nginx:alpine` (1.31.6) is clean
while `1.29-alpine` scans worse at 37 HIGH. The two remaining "fixable" API
findings are `msgpack` and `setuptools`, which I confirmed are absent from the
built image rather than trusting the earlier report. Local images now share the
pinned bases, so the integration suite exercises what ships; full suite passes on
nginx 1.31.6. With both images at zero CRITICAL, the image scan now **gates on
CRITICAL** and reports HIGH.
### Reporting
- `[x]` Week-1 client report (`Relatorio-Semana-1-DTF.docx`), corrected 2026-09-18 to

View File

@@ -129,9 +129,16 @@ change when the advisory database or selected base digest changes.
## Production delivery security boundary
The files in `deploy/` and `.gitea/workflows/` are a guarded delivery mechanism,
not an approval to operate the current application on the public internet. The
single Gitea workflow requires a protected Docker runner, exact commit checkout,
digest-pinned base/scanner images, regressions, and HIGH/CRITICAL Trivy gates.
not an approval to operate the current application on the public internet.
Corrected 2026-09-21: an earlier version of this section described gates the
workflow did not contain. The workflow now runs static validation, the
integration suite against a live stack, and a blocking Trivy secret scan before
publishing. The source preflight is advisory unless
`ENFORCE_PRODUCTION_PREFLIGHT` is set, and image vulnerabilities are reported
rather than enforced, because the current bases carry HIGH/CRITICAL findings
with no upstream fix. Base images are still mutable tags, not digests.
`PORTAINER.md` holds the authoritative table of what gates and what does not.
It publishes both `latest` and the full commit SHA, then calls the Portainer
webhook. Application/provider secrets are created directly as versioned external
Swarm secrets and never cross the workflow. Rollback selects the prior commit SHA

View File

@@ -1,5 +1,8 @@
# syntax=docker/dockerfile:1
ARG PYTHON_BASE_IMAGE=python:3.12-slim
# Pinned by digest so a rebuild of the same commit produces the same base.
# Override with the PYTHON_BASE_IMAGE repository variable to move it forward
# deliberately, and update this default in the same change.
ARG PYTHON_BASE_IMAGE=python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9
FROM ${PYTHON_BASE_IMAGE}
ARG VCS_REF=unknown
@@ -7,6 +10,13 @@ LABEL org.opencontainers.image.title="DTF Portal/API" \
org.opencontainers.image.revision="$VCS_REF" \
org.opencontainers.image.source="DTF System repository"
# The base is pinned, so its OS packages are frozen at the digest's build date.
# Upgrade them here or the image ships known-fixed Debian vulnerabilities, which
# is what the production image was doing while the local one already did this.
RUN apt-get update \
&& apt-get upgrade -y \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY local/requirements.txt local/requirements.lock /app/local/
RUN python -m pip install --no-cache-dir --require-hashes -r local/requirements.lock

View File

@@ -1,6 +1,10 @@
# syntax=docker/dockerfile:1
ARG PYTHON_BASE_IMAGE=python:3.12-slim
ARG NGINX_BASE_IMAGE=nginx:1.28-alpine
ARG PYTHON_BASE_IMAGE=python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9
# Both bases are pinned by digest; override with the repository variables to
# move them forward deliberately.
# nginx 1.31.6. The 1.28 line pins nginx=1.28.3-r1 in /etc/apk/world, so its five
# HIGH findings cannot be upgraded in place; 1.29 scans worse. This one is clean.
ARG NGINX_BASE_IMAGE=nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8
FROM ${PYTHON_BASE_IMAGE} AS policy
WORKDIR /build
COPY dtf-site.html /build/dtf-site.html
@@ -10,6 +14,8 @@ ENV NGINX_TEMPLATE=/build/deploy/nginx.conf.template
RUN python local/compile_web.py
FROM ${NGINX_BASE_IMAGE}
# Same reason as the API image: a pinned base freezes its packages.
RUN apk upgrade --no-cache
ARG VCS_REF=unknown
LABEL org.opencontainers.image.title="DTF Site and Kanban" \
org.opencontainers.image.revision="$VCS_REF" \

View File

@@ -1,4 +1,6 @@
FROM python:3.12-slim
# Same pinned base as deploy/Dockerfile.api, so the integration suite exercises
# the image that ships rather than a different one.
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9
RUN apt-get update \
&& apt-get upgrade -y \
&& rm -rf /var/lib/apt/lists/*

View File

@@ -1,10 +1,11 @@
FROM python:3.12-slim AS policy
FROM python:3.12-slim@sha256:2f17fc044b579bab302c2e8054d3a686e2cb9a83de48e70534b94cd8ebbe06a9 AS policy
WORKDIR /build
COPY dtf-site.html /build/dtf-site.html
COPY local /build/local
RUN python local/compile_web.py
FROM nginx:1.28-alpine
# Same pinned base as deploy/Dockerfile.web.
FROM nginx:alpine@sha256:62ff2089abf5a9ed33bd232895bef5e22f7bb4b200675cec49a5ebc48e3d4ac8
RUN apk upgrade --no-cache
ENV WEB_INDEX=index.html
COPY --from=policy /build/default.conf.template /etc/nginx/templates/default.conf.template