Skip to content

release-train: develop -> staging - #889

Merged
tracebloc-release-train[bot] merged 8 commits into
stagingfrom
release-train/to-staging
Aug 27, 2026
Merged

release-train: develop -> staging#889
tracebloc-release-train[bot] merged 8 commits into
stagingfrom
release-train/to-staging

Conversation

@LukasWodka

@LukasWodkaLukasWodka commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Automated promotion by the release train (RFC-0008 D14). Head is the train-managed release-train/to-staging branch (a mirror of develop), so it never collides with a human PR. Merged only when the fr-gate is green.


Note

Medium Risk
Touches security-adjacent ingestor pin policy and operator mirroring paths, plus required CI trigger semantics; changes are mostly fail-closed hardening with broad test coverage, but a misconfigured ackDrift could mask real digest drift.

Overview
Automated develop → staging promotion bumps the client chart to 1.9.78 and bundles operator, CI, and watcher fixes that were on develop.

Blocked-registry mirroring (backend#2633) replaces the broken helm template | grep image guidance with scripts/list-images.sh, which derives chart images plus jobs-manager–spawned ingestor and training images, fails closed on partial lists, and wires TRACEBLOC_CA_BUNDLE into curl_secure. mirror-enumeration-complete.sh is added to make drift, and docs/INSTALL.md points operators at the new script.

Ingestor digest watch (backend#2673) adds chart images.ingestor.ackDrift so check-digest-drift.sh treats the intentional prod float-vs-prodDigest gap as ACKNOWLEDGED (still reds other pins, lapsed ack lines, or an unhealthy pin). Docs in values.yaml, SECURITY.md, and MIGRATION.md / rotate-mysql-root.md spell out 1.9.71+ gates and silent-fail paths on older charts.

CI retarget gap (backend#2701) adds edited to required workflows (drift-checks, helm-unit, standard-checks, version-bump gate) and extends helm-unittest-gated.sh so retargeted PRs do not sit forever on “waiting for status.” Workflow staleness alerter (backend#2702) now fails closed when dedup gh search / gh issue view cannot complete.

Test hygiene (backend#2686) adds pyyaml-preflight.bats and PyYAML import guards across yaml-parsing test scripts; BUGBOT.md documents the class.

Reviewed by Cursor Bugbot for commit a30324c. Bugbot is set up for automated code reviews on this repo. Configure here.

aptraceblocand others added 8 commits August 27, 2026 16:41
…loc CLI (backend#2700) (#882)
The `_assess_classify: all signals true -> healthy` test stubbed every leaf
probe except `_assess_cli_outdated`, so the classify path shelled out to the
real `tracebloc version` on the machine running bats. On Linux CI (no CLI on
PATH) that fails open to "not outdated" and the test reaches healthy; on a
machine with a real below-floor CLI (e.g. 0.7.0 in ~/.local/bin) the floor
check correctly fires cli-outdated, so the box classifies degraded and the
healthy assertion fails.
Not an assess.sh bug — a below-floor CLI IS cli-outdated. Stub the leaf to
"above the floor" (return 1), matching the sibling classify tests
(assess.bats:309/323/341/361/380), so the test no longer depends on whatever
tracebloc happens to be installed on the host. Test-only; no scripts/lib
change (no manifest regen / CODEOWNERS review).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…6) (#880)
* fix(tests): preflight PyYAML in every yaml-parsing guard (backend#2686)
The chart guards check `command -v python3` and then `import yaml` inside a
python heredoc. `command -v python3` proves the interpreter exists, not the
PyYAML module: on a runner with python3 but no PyYAML the import dies as a bare
ModuleNotFoundError traceback (in a .sh guard) or degrades to a silent skip (in
a .bats one — indistinguishable from a pass in a required gate), where the
sibling helm-unittest-error-assertions.sh already gives `[ERROR] PyYAML
required`.
Bugbot flagged one instance (the CronJob guard, backend#2686 / client#869).
Fixing the class, not the instance: every scripts/tests guard that imports the
yaml module now preflights it — the .sh guards with the interpreter-then-module
try/except the siblings carry, chart-pull-secret.bats with a require_pymodule
helper mirroring its require_tool (local skip / CI hard-fail), and the two
image-refresh .bats with the same named refusal in their inline python.
Adds scripts/tests/pyyaml-preflight.bats, which derives the guard list from the
tree (no hardcoded names), fails closed on zero guards, and behaviourally
proves a real guard refuses cleanly when PyYAML is absent. Records the recurring
finding in .cursor/BUGBOT.md per CLAUDE.md.
Closestracebloc/backend#2686
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tests): harden pyyaml-preflight assertions with || return 1 (backend#2686)
bats-hygiene.bats flags standalone [ ]/[[ ]] assertions that lack a
`|| return 1` closer: bats only propagates the LAST command's status (and
[[ ]] escapes errexit on bash 3.2), so a non-final failing assertion is
silently advisory. Append the closer to every assertion in the new suite.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tests): make the PyYAML-preflight enforcer structural + complete (backend#2686)
Addresses review on client#880 (Saqlain):
- Enumerate the `from yaml import ...` idiom too, not just `import ... yaml` —
a guard written that way was silently skipped, contradicting the fails-closed
claim.
- Detect the preflight STRUCTURALLY (the import must sit in a `try:` whose
`except` names ImportError/ModuleNotFoundError, or a shell-level
require_pymodule/require_yaml_tooling gate), not by a whole-file substring —
a stray "PyYAML required" in a comment no longer satisfies the rule.
- Exclude the enforcer file itself from the denominator (it is not a
guard-under-test); the count is now the real guards only.
- The "PyYAML present" case now asserts only that the preflight does not fire,
not the sibling guard's full repo-wide verdict, so an unrelated helm-unittest
defect cannot misattribute a failure to this suite.
Verified: the hardened detector flags an unguarded `from yaml import` and an
unguarded `import yaml` whose file merely mentions "PyYAML required" in a
comment, while passing every real guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tests): make the PyYAML-preflight enforcer AST-structural, not heuristic (backend#2686)
Bugbot on client#880: the previous enforcer could still fail open. Two gaps,
both now closed:
- The shell-gate acceptance and the try/except detection were whole-file/
windowed substring checks, so a `require_pymodule yaml` in a COMMENT, or a
`try:` that closes before the import with an `except ImportError` within five
lines, satisfied the rule without actually guarding the import.
Now the check parses the embedded python with `ast` and confirms every yaml
import (import/from, module `yaml`) sits lexically inside a `try` whose handler
names ImportError/ModuleNotFoundError; the shell-level gate
(require_pymodule/require_yaml_tooling/one-line probe) is matched on
comment-stripped lines only; heredoc/`-c` openers quoted inside a comment no
longer open a bogus block (the `<<TAG`-in-a-comment trap bats-hygiene itself
handles); and a single-line inline `python3 -c 'import yaml...'` is enumerated
via the extracted snippet. Unparseable yaml-bearing source fails closed.
Verified against the tree (all 17 real guards pass) and crafted negatives: an
unguarded import whose file only mentions "PyYAML required"/"require_pymodule
yaml" in a comment, a `try:` closed before the import, a `from yaml import`,
and an `except` that catches the wrong error are each flagged; a properly
guarded file and a real shell-gated file pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tests): close the last enumerator-completeness holes in the PyYAML enforcer (backend#2686)
Two more Bugbot mediums on client#880, both fixed:
- Double-quoted `python3 -c "import yaml; …"` (single line) was neither
extracted nor gate-matched, so such a guard dropped out of the denominator
and the class check stayed green. Extraction now matches the SAME quote that
opened the -c string (either ' or "), and the one-line gate probe accepts
both quotes.
- `has_shell_gate` accepted the `require_yaml_tooling() {` DEFINITION as an
exemption, so copying the helper without calling it marked a file
preflighted. The gate now requires a CALL (`require_yaml_tooling` not
followed by `(`); `require_pymodule` was already safe via its `yaml` arg.
Also broadened heredoc-opener matching to quoted/double-quoted/bare spellings
so a future guard cannot slip through on the heredoc spelling.
Verified: the 17 real guards still pass; a double-quoted single-line import and
a define-but-never-call helper are each flagged; a file that actually CALLS the
helper passes; all earlier negatives still caught.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…t divergence (backend#2673) (#877)
* fix(ci): teach digest-drift to acknowledge the ingestor prod-pin float divergence (backend#2673)
client/values.yaml pins images.ingestor.prodDigest (v0.8.2) deliberately BEHIND
the channelTags.prod "0.8" float: data-ingestors#468 (first in v0.8.8) drops the
edgeuser DB_USER fallback while prod's serviceDbAccountsByEnv.prod is still false,
so the float roams the unsafe set and the pin must stay in {v0.8.0..v0.8.4}
(docs/SECURITY.md §4.1.1). The pin and the check are both correct — so the fix is
NOT to advance the pin.
The problem was signaling: check-digest-drift.sh reds if ANY pin drifts, so the
intentional, permanent ingestor divergence made the nightly red every night
(0 passes since 2026-08-14). That trained everyone to ignore it (backend#2386)
AND masked any NEW, actionable drift on squid/mysql-client behind the standing
red — the exact failure backend#1853 exists to prevent, re-emerging one level up.
Option A (acknowledged drift). A pin may now declare an `ackDrift: {line, reason}`
block next to it; check-digest-drift.sh then treats that pin's float-vs-pin
divergence as EXPECTED — job GREEN, printed ACKNOWLEDGED with the reason and the
SECURITY.md §4.1.1 pointer — while STILL re-verifying, every run, that the pin
itself resolves to a healthy multi-arch index. It is a CLASS not a digest (the
float may roam any 0.8.x patch without re-alarming), it LAPSES if the float
changes line, and ANY other pin drifting — or the acknowledged pin ceasing to
resolve or going single-arch — still reds. The pin does not move.
Declared in the chart (behind CODEOWNERS review), not in the script, so lifting
the acknowledgement is a reviewed chart change. When the boundary is resolved
(prod flag flips / a prod-safe 0.8.x is cut), delete the block in the same change
that advances the pin.
- scripts/check-digest-drift.sh: ackDrift discovery + pin_platforms re-verify +
ACKNOWLEDGED / lapsed / unhealthy classification and summary.
- scripts/tests/check-digest-drift.bats: 8 cases (green, conditional-red x2,
lapsed, per-pin non-masking, inert-on-agreement, reason verbatim, summary).
- client/values.yaml + values.schema.json: the ackDrift block and its schema.
- client/Chart.yaml: version/appVersion 1.9.75 -> 1.9.76 (chart content changed).
- Makefile: the digest-drift target comment no longer claims a permanent red.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore(chart): bump to 1.9.77 after merging develop (backend#2673)
develop advanced to 1.9.76 (client#875) after this branch bumped to the same
version, so the chart-version-guard saw no bump. Re-bump one patch above the
new develop tip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(ci): bind ackDrift to the prodDigest leaf, not the whole image block (backend#2673)
Addresses @saqlainsyed007 review on client#877 (two latent false-green vectors
on a drift guard whose whole job is not to false-green):
1. The ack was BLOCK-scoped: ack_line/ack_reason attached to every pin the
image block emitted, so a populated sibling `digest:` in images.ingestor
would inherit the ack and its genuine drift would print ACKNOWLEDGED (exit 0).
Dormant only because that digest is \"\" today.
2. It was EMIT-ORDER-dependent: ackDrift: had to textually precede prodDigest:
or the ack pair was still empty at emit -> plain DRIFT -> spurious permanent
red (the exact failure this PR removes). A values.yaml reorder would break it.
Both die by deferring prodDigest emission to the block boundary (flush_prod):
the pin is emitted only once the whole block is read, so (a) the ack binds to
the prodDigest leaf alone and a sibling digest: never carries it, and (b) key
order no longer matters. Block repo/float are also resolved at flush time, so
those are order-independent too.
Also (review #5): pin_platforms now runs the stub path through the SAME
unknown/-drop + dedup as the registry path, so the bats seam exercises the
attestation filter and the reported platform set never shows unknown/unknown.
Tests: +3 regression cases (sibling-digest still reds; ackDrift-after-prodDigest
still acknowledges; attestation filtered) -> 31 total, all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…2702) (#884)
alert-workflow-staleness.sh hid `gh search issues` / `gh issue view`
failures with `2>/dev/null` and read the resulting empty output as
"not yet tracked", so a transient search error filed a fresh duplicate
`work-type:bug` per stale workflow on every daily cron run until search
recovered. Same class as backend#2631's filer-dedup gap (a separate repo,
no shared helper).
Capture gh's exit status instead of swallowing it: a failed search — or a
failed candidate read — now aborts loud (exit 2, temp body cleaned up)
rather than proceeding to create. A genuine zero-result (gh exits 0, empty
stdout) still green-lights the create, so healthy dedup is unchanged.
Adds an offline `gh` stub to the bats suite covering both fail-closed paths
(search error, candidate-read error), the dedup-hit skip, and the
genuine-empty files-anyway case. Updates the script header and
docs/WORKFLOW-STALENESS.md exit-code contract to match.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…88) (#885)
* fix(docs): correct the PR template's closing-keyword note (backend#2188)
* fix(docs): drop the false cross-repo-never-fires clause (backend#2188 review)
#887)
* docs(migration): state the chart floor the root-rotation runbook needs (backend#2591)
The rotate-mysql-root runbook's precondition 1 verifies the gate by reading
MYSQL_ROOT_PASSWORD out of the release Secret, and tells the operator to
"enable the gate first" when that read comes back zero. On a chart below
1.9.71 that instruction cannot succeed and does not say so.
`rotateMysqlRoot` landed in client-1.9.71 (client#822). values.schema.json
does not close additionalProperties, so on an older chart
`--set rotateMysqlRoot=true` is accepted and exits 0 while rendering no
MYSQL_ROOT_PASSWORD. Measured on client-1.9.63 via `helm template
--set rotateMysqlRoot=true`: exit 0, zero occurrences of the key. The
operator then re-reads zero and is pointed back at the step they just ran,
with nothing in the loop naming the chart version as the cause.
Add the version check as the first thing in that precondition, and a
MIGRATION.md section for 1.9.71 covering what crossing it actually touches:
the gate itself (false for dev/stg/prod, so a no-op on upgrade) and the
Collector token Role/RoleBinding, which lost their `enabled` gate and now
render unconditionally in the node-agents namespace — a namespace that is
not the release namespace by default, so the upgrading identity needs reach
there. Measured by rendering 1.9.63 and 1.9.71 offline with CLIENT_ENV=prod:
the object set gains exactly those two objects and loses nothing.
Also records what the upgrade does NOT change: serviceDbAccountsByEnv still
resolves false for prod, and requests-proxy renders byte-identically across
the two versions apart from chart labels.
Docs only — no chart content, so the version-bump guard is N/A.
Refs backend#2591
* docs(migration): say which version each new values key actually arrived at (backend#2591)
@saqlainsyed007, verified independently against the chart rather than taken on
trust: `bootstrapDbPassword`, `bootstrapDbReparent` and
`bootstrapDbReparentByEnv` first appear in e03edbb (#785) at Chart 1.9.67,
while `mysqlRootPassword`, `rotateMysqlRoot` and `rotateMysqlRootByEnv` first
appear in 09bb86c (#822) at 1.9.71.
All six are genuinely new to the 1.9.63 fleet this section targets, so the list
was correct for that jump and wrong for anyone crossing 1.9.67 to 1.9.70 -- who
would find the bootstrapDb* keys already present and reasonably wonder what
else the note had got wrong.
Split by introducing version, with the method written down so the next person
derives it instead of recalling it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
… re-evaluate (backend#2701) (#886)
* ci(gates): fire required checks on pull_request `edited` so retargets re-evaluate (backend#2701)
A required status check filtered by `branches:` with no `types:` uses GitHub's
defaults [opened, synchronize, reopened]. A base-branch change (a RETARGET)
fires only `edited`, which the defaults omit -- so a PR opened against a base a
workflow never ran on and then retargeted onto a gated base never produces the
check and sits at "Expected - waiting for status to be reported" forever, the
same permanent-pending state the paths trap causes, reached via the trigger.
Add `edited` (and spell out the three defaults it replaces) to the three
documented-required, non-path-filtered workflows that share this trap:
- helm-unit.yaml (`Helm unit tests`) -- the Bugbot finding
- drift-checks.yaml (`Source-of-truth drift`)
- standard-checks.yml (`Lint`, `Unit tests`)
Encode the rule in helm-unit's own guard (helm-unittest-gated.sh, run by the
required `Source-of-truth drift` job): it now fails closed when helm-unit's
`pull_request` types omit `edited` or drop a default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci(gates): add `edited` to version-bump-gate so a retarget re-runs it (backend#2701)
version-bump-gate-caller.yml is a hard-fail (soft-fail: false) develop gate with
`branches: [develop]` and no `paths:`, but its types omitted `edited` -- the same
base-change trap this PR fixes on the other required, non-path-filtered gates. A
PR opened against a non-develop base and retargeted onto develop would never
re-run the gate (a retarget fires only `edited`, and `synchronize` needs a push).
Its sibling fr-gate-caller.yml already carries `edited` for the identical reason
(backend#1945/.github#237).
Raised in review by @saqlainsyed007 as a missed class member. A repo-wide sweep
now confirms every `branches:`-filtered, non-path-filtered gate carries `edited`
except e2e-auth-proxy.yaml, which stays deferred to its required-flip (documented
in the PR body).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…backend#2633) (#881)
* docs(install): enumerate the COMPLETE mirror pull set, and guard it (backend#2633)
A blocked-registry operator followed INSTALL.md, mirrored what it listed,
installed cleanly -- and ImagePullBackOffed on the first experiment. The
enumeration command was wrong three ways, all measured on develop:
1. It ERRORS. `helm template ./client` exits 1 on storageClass.provisioner
and clientId. Redirect stderr and you get silence, which reads as
"nothing to mirror".
2. It cannot see run-time-spawned images. The training images and the
ingestor are named by jobs-manager at spawn time; `grep -c client-` over
the rendered output is 0.
3. It misses conditional templates (the GPU device plugins render only with
gpu.devicePlugin.enabled=true, not the default).
Adds scripts/list-images.sh, which prints the complete set and DERIVES every
part of it (rule 1): chart images from a render of the operator own values,
the mirror prefix from the rendered JOB_IMAGE_HOST, the ingestor from
INGESTOR_IMAGE_REPOSITORY/TAG/DIGEST, and the task set from the registry own
repository list. It FAILS CLOSED (rule 3): a failed render, an unreadable
registry, or zero tasks is an error, never an empty section.
NOT changed, because the ticket headline claim is false: global.imageRegistry
DOES re-home the training images -- the chart derives JOB_IMAGE_HOST from it
and client/tests/global_image_registry_test.yaml asserts exactly that. Telling
operators to set JOB_IMAGE_HOST by hand would add a way to get it wrong. The
defect was knowing WHAT to copy, not how to re-home it.
scripts/tests/mirror-enumeration-complete.sh joins the required
Source-of-truth drift job. It reads the enumeration command OUT of INSTALL.md
rather than restating it, derives the asserted section headers from the doc own
example block, and checks the refusals exit non-zero without printing a partial
list. Mutation-proved 7/7 with the anchor asserted applied each time: doc
reverting to `helm template | grep`, doc renaming a section, doc no longer
invoking the script, the ingestor extractor taking the next line, the training
path dropping the registry namespace, and each of the two refusals turned into
an exit 0.
Two defects found in this script by its own output, both instances of classes
this repo has been bitten by before: `awk | head -1 | sed` under pipefail
exited 141 (SIGPIPE, backend#2264), and the ingestor extractor matched a
rendered TEMPLATE COMMENT mentioning INGESTOR_IMAGE_REPOSITORY and returned
prose -- the comment-satisfies-the-pattern trap from backend#2632.
Evidence: make drift -> all 25 guards green; helm unittest -> 586 passed;
shellcheck -S warning -x + bash -n clean on both new scripts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(list-images): derive the training tag from the render; fail closed on a bad registry page (backend#2633)
Both findings were Bugbot HIGH, both correct, and both were the fail-open shape
this script's own header claims to avoid.
1. THE TRAINING TAG IGNORED THE RENDER. --env defaulted to prod and stamped that
onto the task images while every other line in the output came from the helm
render, which already emits CLIENT_ENV. A values file selecting `stg` produced
`:prod` task images beside `:stg` control-plane images, so the operator mirrors
the wrong training set and the first non-prod experiment ImagePullBackOffs --
the exact failure this tool exists to prevent, reintroduced by the one value
that was not derived (rule 1).
The tag now comes from the rendered CLIENT_ENV. --env stays as an explicit
override for enumerating a different environment than the values describe, and
a disagreement is REFUSED (exit 2) rather than silently resolved: a mismatch
is far more likely a mistake than an intention. An absent CLIENT_ENV is a
refusal too, not a guess.
2. REGISTRY JSON PARSE FAILURES FAILED OPEN. The interpreter calls carried
`2>/dev/null || echo ""` and `|| true`, and python3 was never preflighted. A
missing interpreter, a truncated body, or a later pagination page that failed
to parse all read as "no more names" -- so after ONE successful page the script
printed a PARTIAL training list and exited 0. A partial list is worse than no
list: it looks complete.
One interpreter call now emits the next-page URL and the names together, so a
parse failure cannot half-succeed, and its exit status is checked. python3 is
preflighted alongside curl. Pagination is bounded at 50 pages rather than
looping or printing what it had. Suppressing the interpreter's stderr is safe
here precisely because the status is checked -- and the first 200 bytes of the
response are printed instead, which is the useful diagnostic.
Also in this commit:
* Six early-closing pipes replaced with here-strings in the new guard, caught by
this repo's own `quality / pipefail early-close` check: under errexit+pipefail
a reader that closes early SIGPIPEs upstream and the pipeline exits 141, so a
guard that dies mid-run reports nothing rather than a finding. It had already
caught four in list-images.sh. The seventh hit was my own comment quoting the
pattern -- the comment-vs-code trap for the fourth time this session -- reworded
rather than left to confuse the checker.
* TRACEBLOC_REGISTRY_URL, so the fail-closed path above is TESTABLE. It could not
be exercised before: common.sh prepends the system PATH, so stubbing `curl` does
not work, and an unexercised guard is indistinguishable from one that does not
work (rule 5).
* The guard grew both refusals as permanent coverage (17 assertions, min raised to
12), and it caught its own setup error doing so: the file exports
TRACEBLOC_TASK_REPOS globally to stay offline, and with it set the registry
branch is never entered, so the refusal under test was unreachable and the
script exited 0. Fixed with `env -u`.
* Doc header comparison now matches the STABLE PREFIX of each section, because
the training header is parameterised by CLIENT_ENV by design and the doc cannot
name a fixed tag without being wrong for every non-prod install. Renaming a
section is still caught; a differing tag is no longer a false failure. The guard
found this drift itself when the doc was updated.
Evidence: make drift -> all 25 guards green; helm unittest -> 586 passed;
mirror-enumeration-complete -> OK (17 assertions); 7/7 mutations still caught with
the anchor asserted applied; shellcheck -S warning -x + bash -n clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(list-images): the partial-list regression had no fixture (Bugbot #881)
Bugbot is right, and the proof is clean. The suite had a case named "unparseable
registry response", but its fixture is a SINGLE bad page. With one unparseable
page the script collects zero names, so the LATER zero-repos check produces
exactly the observables `refuses` asserts -- non-zero exit, empty stdout -- and
the parser exit status is never what the test measures.
So the regression list-images.sh own comment describes was uncovered: "after ONE
successful page the script printed a PARTIAL training list and exited 0".
Added the fixture that produces that state: a good page whose `next` points at
the bad one. `task_repos` is then non-empty when page 2 fails, so the zero-repos
check cannot mask the difference.
MUTATION-PROVED, and it demonstrates both halves of the finding. Replacing the
parser `exit 1` with `:` -- the original defect -- reddens ONLY the new
assertion:
FAIL: a good page followed by an unparseable one ...: exited 0.
1 failure(s) across 18 assertion(s)
The pre-existing case stayed green under that mutation, which is precisely what
Bugbot said would happen.
The single-page case is kept rather than replaced: it is still a valid
end-to-end refusal, and its comment now says plainly that it does not pin the
parser so nobody mistakes it for coverage again.
bash -n and shellcheck -S warning clean; 18 assertions pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(list-images): pin the fetch failure that happens AFTER a good page (backend#2633)
Bugbot, medium, and a sharp one: the guard pinned the PARSE failure after one
good page but never the FETCH failure after one good page. The behaviour was
already correct -- the proof was not.
Why the distinction is the whole finding: a first-page fetch miss is caught
downstream by the zero-repositories check, so replacing the fetch's `exit 1`
with a loop `break` left BOTH existing refusals green while the script printed a
partial training list and exited 0. The guard proved one path and merely looked
like it proved the other.
The case is constructed rather than reasoned about: page 1 resolves and yields
one repo, its `next` points at a file that does not exist, so page 2's fetch
fails with a non-empty accumulator. Nothing may reach stdout and the exit must be
non-zero. 19 assertions now, minimum raised to 13.
Mutation-proved with the anchor asserted applied -- swapping that `exit 1` for
`break`, exactly as the finding describes, now fails with
"registry fetch failing AFTER a good page: exited 0".
Evidence: make drift -> all 25 guards green; helm unittest -> 586 passed; the
earlier 7 mutations still all caught; shellcheck -S warning -x + bash -n clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(list-images): show curl's error and name the remedy that fits it (backend#2633)
Bugbot, medium, and it lands squarely on this PR's own subject. The fetch branch
discarded curl's stderr and then told the operator to set TRACEBLOC_TASK_REPOS --
unconditionally. Behind a TLS-inspecting proxy the real failure is the CA trust
path, so the one remedy shown was the one that cannot help, and the operator was
never pointed at TRACEBLOC_CA_BUNDLE, which docs/INSTALL.md documents for exactly
that case. Clean symptom, wrong cause, wrong fix -- the shape this whole ticket
is about. The parse branch beside it already printed the response body; this one
printed nothing.
curl's own stderr is now kept and shown, and the remedy is chosen from it:
certificate/x509/SSL/TLS -> "this is a TLS trust failure", CURL_CA_BUNDLE /
TRACEBLOC_CA_BUNDLE, and the INSTALL.md section
cannot resolve host -> DNS, then TRACEBLOC_REGISTRY_URL for a mirror
anything else -> TRACEBLOC_REGISTRY_URL or TRACEBLOC_TASK_REPOS
All three verified by construction rather than by reading the code:
expired.badssl.com routes to the TLS arm, an .invalid host to the DNS arm, and a
missing file:// path to the generic arm. Two of them are in the guard now (21
assertions, minimum raised to 15) using the offline file:// case; the TLS and DNS
arms match on curl's message text and need a real endpoint, so they are stated as
not covered rather than left to look covered.
Mutation-proved 3/3 with the anchor asserted applied: stderr redirected to
/dev/null again (the reported bug), the diagnostic suppressed, and the remedy no
longer named.
Also three `# style-guard: allow` markers. check-style.sh rule 3 greps `\bcurl\b`
and exempts comment lines but not string literals, so a diagnostic that NAMES the
tool trips the rule that exists to stop calling it directly. Used the documented
opt-out with the reason recorded at the site; rewording the messages to dodge the
grep would make them worse for the operator, which is the wrong trade.
Evidence: make drift -> all 25 guards green; helm unittest -> 586 passed; the
earlier 7 mutations still caught; shellcheck -S warning -x + bash -n clean.
Chart at 1.9.79.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(list-images): a one-line list container is still a container (backend#2633)
- image: (dash and key on one line) never matched the extractor anchor, so
tracebloc/mysql-client -- rendered exactly that way by mysql-deployment.yaml --
was dropped from every pull set this tool produced. On a blocked-registry
install the mysql pod ImagePullBackOffs: the backend#2633 failure, produced by
the tool that exists to prevent it.
The guard now judges completeness STRUCTURALLY -- PyYAML walks the render and
every image key it finds must appear in the output -- so a third YAML form
cannot slip past the way this one did. A vacuity guard fails loudly if the
chart stops emitting a list item, and both are mutation-proved.
Also: TRACEBLOC_CA_BUNDLE is now mapped onto CURL_CA_BUNDLE (only if unset --
these replace the trust store rather than add to it), so the TLS remedy names a
knob that is actually wired. A new derived check refuses any refusal message
that names a TRACEBLOC_* variable the script never reads.
And the guard now RUNS: it is a .sh, and no workflow invoked it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* revert(ci): the enumeration guard already gates via make drift (backend#2633)
I added it to the required Unit tests job on the claim that no workflow ran
it. That claim was wrong, and it was wrong because I grepped .github/workflows
and stopped there. scripts/tests/mirror-enumeration-complete.sh is in the
Makefile's DRIFT_GUARDS list, which `make drift` runs, which IS the required
`Source-of-truth drift` context on develop -- and drift-checks.yaml deliberately
names no guard so that adding one to the Makefile gates automatically.
So the step was a duplicate run of an already-required guard, and it also
contradicted the design the drift job states in its own comments.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
ContributorAuthor

bugbot run

@saqlainsyed007
saqlainsyed007 removed the request for review from saadqbalAugust 27, 2026 18:37
@tracebloc-release-traintracebloc-release-trainBot added the gate-nudge Toggled by the release train to (re-)fire the fr-gate label Aug 27, 2026

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a30324c. Configure here.

Comment threadscripts/list-images.sh
@tracebloc-release-traintracebloc-release-trainBot removed the gate-nudge Toggled by the release train to (re-)fire the fr-gate label Aug 27, 2026
@LukasWodka

Copy link
Copy Markdown
ContributorAuthor

Promoted with 1 open Medium/Low Bugbot finding(s), per the severity policy in release-train's README (High stops the line; Medium/Low are recorded and ship, at both hops):

grep under set -euo pipefail kills script before empty-check

This is a second look at once-reviewed code -- it passed per-feature review on the source branch, and has NOT had functional review yet (that happens on staging). Fix forward on develop if any is real.

What the train did with each:

@tracebloc-release-train
tracebloc-release-trainBot merged commit 8c05a18 into stagingAug 27, 2026
59 of 60 checks passed
@tracebloc-release-train
tracebloc-release-trainBot deleted the release-train/to-staging branch August 27, 2026 19:38
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@LukasWodka@aptracebloc@saqlainsyed007