Uh oh!
There was an error while loading. Please reload this page.
fix(ci): scope the shard-attestation verdict to the current run attempt - #12066
Merged
Conversation
Artifacts live on the RUN, not the attempt, so every attempt after the first downloads whatever the earlier ones published. `judge()` compared only the run id -- one level coarser than the artifact namespace it judges -- so after a pre-judgement `filter` death (fail-open per THE FILTER CONTRACT #4928, which is untouched here) `rerun_failed_jobs` could never converge: the legs correctly stayed skipped on attempt 2, the roster adjusted to expected-0, and attempt 1's own credentials were then refused one by one as "unexpected attestation ... that no declared leg accounts for". Measured on run 32806633489. The credential already carried `run_attempt` and the verdict already PRINTED it -- it just never COMPARED it. So this is a verdict change only: no payload change, and nothing in ci.yml moves. The rule is "latest per shard WITHIN ONE RUN", not "only this attempt's": - a credential from an earlier attempt of this run that no declared leg accounts for is discarded with a log line, granting no pass (the green comes from #4928's expected-0, which it never enters); - a credential from an earlier attempt that a declared leg DOES account for still counts. `overwrite: true` on every upload means a re-running leg replaces its own artifact, so what survives is the credential of a leg that did NOT re-run -- and that leg keeps its earlier conclusion in `needs.<job>.result`. Refusing it would turn the ordinary rerun_failed_jobs case (one flaky shard) permanently red: the same defect, one button over; - everything else keeps its old verdict. The tolerance demands a positive "this run, an earlier attempt": a foreign credential from the CURRENT attempt is still refused in the same words, so is one from another run, and an absent or unreadable attempt buys no exemption. --self-test grows the measured sequence (both gates), the current-attempt counter-limb, the partial-re-run pin that blocks the "current attempt only" mis-fix, and the fail-closed cases on both sides of the comparison. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
yinlianghui
marked this pull request as ready for review
August 25, 2026 07:39
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#11998
judge()compared onlyrecord.run_id— one level coarser than the artifact namespace it judges. Artifacts live on the RUN, not on the attempt, so every attempt after the first downloads whatever the earlier ones published. After a pre-judgementfilterdeath the sequence closes on itself: the fail-open filter contract (#4928) lets all 9 shards run and attest on attempt 1;rerun_failed_jobsre-runs the filter, which now judges the diff honestly and correctly skips the legs; the roster adjusts to expected-0; and attempt 1's own credentials are then refused one at a time. No attempt of that run can converge, and a fullrerun_workflow_runhits the identical wall.Measured on run 32806633489 (attempt 2), quoted verbatim from that run's job logs:
The premise, corrected
The card locates the defect in the credential: that it "lacks the one field (
run_attempt)". It does not lack it.--emithas stampedrun_attemptfromGITHUB_RUN_ATTEMPTall along, andjudge()already printed it in the roster listing — it simply never compared it. So the repair is a verdict change alone: no payload change, and nothing in.github/workflows/moves. The static drift guard is green unchanged, andlint.ymlis not touched.The mechanism: latest per shard WITHIN ONE RUN — not "only the current attempt's"
Triage named two mechanisms and did not choose between them. The choice is forced by two facts on the tree:
overwrite: trueis set on every attestation upload. A leg that re-runs therefore REPLACES its own artifact. The store already holds exactly the latest credential per shard; there is no per-attempt accumulation for a namespace to disambiguate. Namespacing artifacts by attempt buys nothing here and costs thename:/pattern:contracts the static guard enforces.needs.JOB.resultthe gate reads.So "accept only the current attempt's" — whether spelled as an attempt-namespaced artifact or as a verify-side filter — reads the ordinary
rerun_failed_jobscase (one flaky shard re-runs alone, the other five carried over) as five missing credentials. That is this same never-converges defect moved one button over, onto the commoner case. It is refused here and pinned against.The
...but its inputs may have changedworry that makes latest-per-shard look risky in general cannot arise within one run: a run is pinned to one commit and one workflow file, and a re-run replays the same event payload. The only way the inputs differ is a different RUN — which the pre-existingrun_idveto already refuses, and which is exactly what a base merge mints (that is how PR #11987 escaped, leaving the class behind for this card).What the verdict now does
carried overin the roster listingrun_idveto, unchanged)run_attempt, on either side of the comparisonThe strictness is not weakened: the tolerance demands a positive "this run, an earlier attempt", and an unreadable attempt buys no exemption — so the worst a garbled field can do is restore the strict verdict, never a silent green. Discarding also grants no pass on its own: that branch only ever ADDED an error, and the green in the measured sequence comes from #4928's expected-0, which the discarded credential never enters.
Reverse verification (3 ablations, each restored by an
EXIT INT TERMtrap)Mutations were applied with an exact-literal mutator that exits non-zero unless the anchor matches exactly once, and each was confirmed on disk by grepping for the injected marker AND for the deleted text before anything was read. This gate runs from source (
node scripts/…), so no build/diststep stands between the edit and the measurement.carriedOveralways false)run_idhalf of the tolerance droppedAblation A leaves the partial-re-run assertion green, which is the correct asymmetry: without the discard path those credentials are still on the roster and still counted. Ablation B is the one that matters for the strictness ruling — a blanket tolerance takes the pre-existing contradiction pins down with it.
Gates
Families derived at the final commit with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack(no hand-written path list) — 9 matched, all run, all green atd288d27c6:check:agent-test-spelling·check:cross-package-test-inputs·check:entry-guard·check:parse-guard·check:pnpm-filter-targets·check:shard-attestation·check-ci-filter-parity.mjs·check-cross-package-test-inputs.mjs·check-shard-attestation.mjscheck:shard-attestationverdict at that commit:ESLint was run narrowed to the changed file, declared as a narrowing with its three measurements: the file is in ESLint's own linted population (its
--format jsonresult carries 0 warnings, so it was linted rather than ignored); the file count is 1, read from that same JSON; and this repo's singleeslint.config.mjsenables type-aware linting for no file at all (noparserOptions.project, no typed rules — the config says so at its line 325 with a measured positive control), so a one-file diff cannot move the verdict on any untouched file. The repo-wide sweep is CI's run.No changeset: CI plumbing under
scripts/, nothing published —skip-changeset.Generated by Claude Code