From 7ff93627e423c8b68c26ea85398a582b0a08bf6d Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Thu, 20 Aug 2026 10:00:35 +0200 Subject: [PATCH 1/2] ci(2212): the fixtures drift check must fail when it cannot run `Backend fixtures drift check` is being armed as a required context (backend#2212). Its activation-phase fail-open has to go first: when BACKEND_CONTRACTS_TOKEN was unreadable the step printed a warning and exited 0, so a check that never executed reported as a passing one. Inert-not-red was the right call while the secret did not exist; the secret has existed since 2026-08-05, and once the context is required an exit-0-when-unable is strictly worse than an advisory guard, because it also looks solved (backend#2183). `cli` is PUBLIC, so the two reasons the token can be missing are different things and the step now splits three ways: token present -> run the check absent, fork PR -> FAIL. GitHub withholds repo secrets from forks by design, so the check genuinely cannot run. A maintainer verifies internal/api/testdata/*.json by hand and applies `skip-fixtures-drift` -- a permanent artifact on the PR, the same model as skip-fr-gate. Silently passing forks would fail open on exactly the contributions that deserve the most scrutiny. absent, same-repo -> FAIL. Rotated, removed or expired: a misconfiguration that used to read as a clean run. `types: [.., labeled, unlabeled]` added to the pull_request trigger, because without them the default opened/synchronize/reopened means applying the override label changes nothing until the next push -- the same defect Bugbot caught on version-bump-gate-caller.yml's skip-version-gate. Every ${{ }} goes through env:, none into the run: body. Mutation-proved, all five paths, by running the step body against a stubbed sync script: override label present exit 0 (OVERRIDDEN warning) token present exit 0 (real check ran) token absent, fork PR exit 1 (could not run) token absent, same-repo exit 1 (secret missing) token present, script reports drift exit 3 (exec propagates the status) The last one matters: `exec` replaces the shell, so a real drift failure still fails the step rather than being swallowed. Label `skip-fixtures-drift` created on this repo. Refs tracebloc/backend#2212 Co-Authored-By: Claude Opus 5 --- .github/workflows/backend-fixtures-drift.yml | 51 ++++++++++++++++++-- 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/.github/workflows/backend-fixtures-drift.yml b/.github/workflows/backend-fixtures-drift.yml index 0ab93a36..cf6af3ee 100644 --- a/.github/workflows/backend-fixtures-drift.yml +++ b/.github/workflows/backend-fixtures-drift.yml @@ -19,12 +19,24 @@ name: Backend fixtures drift # org Actions secret. Until that secret exists the job SKIPS with a warning # instead of failing: the gate is inert, not red (same activation model as # the public PII gate's denylist secret). +# +# THAT ACTIVATION PHASE IS OVER. The secret has existed since 2026-08-05 and +# this check is being armed as a required context (backend#2212), so the +# skip-with-a-warning path is now a fail-open: a *required* check that exits 0 +# when it cannot run is worse than an advisory one, because it also looks +# solved. It now FAILS instead -- see the step below for the three-way split +# and why a public repo needs an override label rather than a silent pass. on: push: branches: [develop, main] pull_request: branches: [develop, main] + # labeled/unlabeled so the skip-fixtures-drift override actually re-runs the + # gate. The default types are opened/synchronize/reopened, so without these + # applying the label would change nothing until the next push -- exactly the + # bug Bugbot caught on version-bump-gate-caller.yml's skip-version-gate. + types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled] workflow_dispatch: permissions: @@ -42,13 +54,44 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # Three-way, because this repo is PUBLIC and the two reasons the token can + # be missing are not the same thing: + # + # token present -> run the check (the normal path) + # absent, fork PR -> FAIL. GitHub does not expose repo secrets to + # forks by design, so the check genuinely + # cannot run. A maintainer verifies the + # fixtures by hand and applies + # `skip-fixtures-drift`, which is a permanent + # artifact on the PR -- the same model as + # skip-fr-gate. Passing forks silently would + # fail open on exactly the contributions that + # warrant the most scrutiny. + # absent, not a fork -> FAIL. The secret was removed, rotated or + # expired. That is a misconfiguration, and it + # used to read as a clean run. + # + # Every ${{ }} goes through env:, never into the run: body. - name: scripts/sync-backend-fixtures.sh --check env: BACKEND_CONTRACTS_TOKEN: ${{ secrets.BACKEND_CONTRACTS_TOKEN }} + IS_FORK: ${{ github.event.pull_request.head.repo.fork || false }} + OVERRIDE: ${{ contains(github.event.pull_request.labels.*.name, 'skip-fixtures-drift') }} run: | - if [ -z "$BACKEND_CONTRACTS_TOKEN" ]; then - echo "::warning::BACKEND_CONTRACTS_TOKEN is not set — skipping the backend fixtures drift check." \ - "Add a read-only (Contents: read) token for tracebloc/backend as a repo/org Actions secret to activate this gate." + set -uo pipefail + + if [ "$OVERRIDE" = "true" ]; then + echo "::warning title=Backend fixtures drift OVERRIDDEN::The skip-fixtures-drift label is applied, so this gate did NOT verify internal/api/testdata/*.json against tracebloc/backend. Whoever applied the label is asserting they checked the fixtures by hand. The label stays on the PR as the record." exit 0 fi - ./scripts/sync-backend-fixtures.sh --check + + if [ -n "$BACKEND_CONTRACTS_TOKEN" ]; then + exec ./scripts/sync-backend-fixtures.sh --check + fi + + if [ "$IS_FORK" = "true" ]; then + echo "::error title=Backend fixtures drift could not run::This PR is from a fork, and GitHub does not expose BACKEND_CONTRACTS_TOKEN to forks, so the vendored fixtures could not be checked against tracebloc/backend. A maintainer must verify internal/api/testdata/*.json by hand and apply the 'skip-fixtures-drift' label. Refusing to report green on a check that did not run (backend#2212)." + else + echo "::error title=BACKEND_CONTRACTS_TOKEN is missing::The secret is not readable on this run, so the backend fixtures drift check did not execute. It is expected on develop/main PRs from this repo -- if it was rotated or removed, restore a read-only (Contents: read) token for tracebloc/backend as a repo/org Actions secret. This step used to exit 0 here, which reported a check that never ran as a passing one (backend#2212)." + fi + exit 1 From 3e3429ca065eb8907cad6e41ee9ed7385bc2bbde Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Thu, 20 Aug 2026 10:09:22 +0200 Subject: [PATCH 2/2] fix(2212): Dependabot PRs are not forks, and would have been blocked I claimed in chat that this PR was unaffected by the Dependabot finding on averaging-service#367. Wrong, and this repo is the worse case of the two. Dependabot branches live in THIS repo, not a fork, so `github.event.pull_request.head.repo.fork` is FALSE on them -- verified on the real #530: head.repo.fork=false, head.repo.full_name=tracebloc/cli. Their runs still receive only Dependabot-scoped secrets, so BACKEND_CONTRACTS_TOKEN is empty. Under the previous commit that combination landed in the "absent, same-repo -> misconfiguration -> FAIL" branch, which would have blocked every Dependabot PR once the context is armed. Not theoretical: this repo has 4 Dependabot PRs, #530 is OPEN right now, and it currently reports `Backend fixtures drift check: success` -- the fail-open passing vacuously on a live PR today. So Dependabot gets a fourth branch, passing with a ::notice::. Safe for the same structural reason as averaging-service#367, via a different always-running guard: a dependency bump cannot alter internal/api/testdata/*.json, and if it did, internal/api/contracts_test.go replays every fixture through the real decode paths under the REQUIRED `Test` check with no token. Drift against the pinned backend ref is re-checked by the push run on develop/main, where Actions secrets are available. Mutation-proved, all five: Dependabot PR (fork=false, no token) exit 0 (notice: deferred) fork PR, no token exit 1 (could not run) human same-repo, no token exit 1 (secret missing) token present exit 0 (real check ran) override label exit 0 (OVERRIDDEN warning) PR_AUTHOR uses github.event.pull_request.user.login, not github.actor, so it stays correct across re-runs. Refs tracebloc/backend#2212 Co-Authored-By: Claude Opus 5 --- .github/workflows/backend-fixtures-drift.yml | 25 ++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/.github/workflows/backend-fixtures-drift.yml b/.github/workflows/backend-fixtures-drift.yml index cf6af3ee..4c5fc8eb 100644 --- a/.github/workflows/backend-fixtures-drift.yml +++ b/.github/workflows/backend-fixtures-drift.yml @@ -58,6 +58,24 @@ jobs: # be missing are not the same thing: # # token present -> run the check (the normal path) + # absent, Dependabot PR -> PASS with a ::notice::. Dependabot runs get + # DEPENDABOT-scoped secrets, never Actions + # secrets, so the token is empty on them + # however correctly it is set -- and unlike a + # fork, head.repo.fork is FALSE, so they would + # otherwise land in the misconfiguration branch + # below and block every security bump (Bugbot, + # averaging-service#367). Safe to defer here: + # a dependency bump cannot alter + # internal/api/testdata/*.json, and if it did, + # internal/api/contracts_test.go replays every + # fixture through the real decode paths under + # the REQUIRED `Test` check, with no token. + # Divergence from the pinned backend ref is + # re-checked by the push run on develop/main, + # where Actions secrets are available. Set the + # token as a Dependabot secret too and this + # branch stops being reached. # absent, fork PR -> FAIL. GitHub does not expose repo secrets to # forks by design, so the check genuinely # cannot run. A maintainer verifies the @@ -76,6 +94,8 @@ jobs: env: BACKEND_CONTRACTS_TOKEN: ${{ secrets.BACKEND_CONTRACTS_TOKEN }} IS_FORK: ${{ github.event.pull_request.head.repo.fork || false }} + # The PR author, not github.actor: stable across re-runs of the same PR. + PR_AUTHOR: ${{ github.event.pull_request.user.login }} OVERRIDE: ${{ contains(github.event.pull_request.labels.*.name, 'skip-fixtures-drift') }} run: | set -uo pipefail @@ -89,6 +109,11 @@ jobs: exec ./scripts/sync-backend-fixtures.sh --check fi + if [ "$PR_AUTHOR" = "dependabot[bot]" ]; then + echo "::notice title=Backend fixtures drift deferred::Dependabot runs receive Dependabot-scoped secrets, not Actions secrets, so BACKEND_CONTRACTS_TOKEN is empty here by design. A dependency bump cannot alter internal/api/testdata/*.json, and internal/api/contracts_test.go replays every fixture through the real decode paths under the required Test check without needing a token. Drift against the pinned backend ref is re-checked by the push run on develop/main. Set BACKEND_CONTRACTS_TOKEN as a Dependabot secret too for full coverage here." + exit 0 + fi + if [ "$IS_FORK" = "true" ]; then echo "::error title=Backend fixtures drift could not run::This PR is from a fork, and GitHub does not expose BACKEND_CONTRACTS_TOKEN to forks, so the vendored fixtures could not be checked against tracebloc/backend. A maintainer must verify internal/api/testdata/*.json by hand and apply the 'skip-fixtures-drift' label. Refusing to report green on a check that did not run (backend#2212)." else