Skip to content

fix(ci): scope the shard-attestation verdict to the current run attempt - #12066

Merged
yinlianghui merged 1 commit into
mainfrom
claude/issue-11998-attestation-run-attempt
Aug 25, 2026
Merged

fix(ci): scope the shard-attestation verdict to the current run attempt#12066
yinlianghui merged 1 commit into
mainfrom
claude/issue-11998-attestation-run-attempt

Conversation

@yinlianghui

@yinlianghuiyinlianghui commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Fixes#11998

judge() compared only record.run_id — one level coarser than the artifact namespace it judges. Artifacts live on the RUN, not on the attempt, so every attempt after the first downloads whatever the earlier ones published. After a pre-judgement filter death the sequence closes on itself: the fail-open filter contract (#4928) lets all 9 shards run and attest on attempt 1; rerun_failed_jobs re-runs the filter, which now judges the diff honestly and correctly skips the legs; the roster adjusts to expected-0; and attempt 1's own credentials are then refused one at a time. No attempt of that run can converge, and a full rerun_workflow_run hits the identical wall.

Measured on run 32806633489 (attempt 2), quoted verbatim from that run's job logs:

Test Core: unexpected attestation 'test-1-of-6' that no declared leg accounts for. (x6)
Dogfood Regression Gate: unexpected attestation 'dogfood-1-of-3' ... (x4)

The premise, corrected

The card locates the defect in the credential: that it "lacks the one field (run_attempt)". It does not lack it. --emit has stamped run_attempt from GITHUB_RUN_ATTEMPT all along, and judge() already printed it in the roster listing — it simply never compared it. So the repair is a verdict change alone: no payload change, and nothing in .github/workflows/ moves. The static drift guard is green unchanged, and lint.yml is not touched.

The mechanism: latest per shard WITHIN ONE RUN — not "only the current attempt's"

Triage named two mechanisms and did not choose between them. The choice is forced by two facts on the tree:

  1. overwrite: true is set on every attestation upload. A leg that re-runs therefore REPLACES its own artifact. The store already holds exactly the latest credential per shard; there is no per-attempt accumulation for a namespace to disambiguate. Namespacing artifacts by attempt buys nothing here and costs the name:/pattern: contracts the static guard enforces.
  2. What survives into a later attempt is the credential of a leg that did NOT re-run — and such a leg also keeps its earlier conclusion in the needs.JOB.result the gate reads.

So "accept only the current attempt's" — whether spelled as an attempt-namespaced artifact or as a verify-side filter — reads the ordinaryrerun_failed_jobs case (one flaky shard re-runs alone, the other five carried over) as five missing credentials. That is this same never-converges defect moved one button over, onto the commoner case. It is refused here and pinned against.

The ...but its inputs may have changed worry that makes latest-per-shard look risky in general cannot arise within one run: a run is pinned to one commit and one workflow file, and a re-run replays the same event payload. The only way the inputs differ is a different RUN — which the pre-existing run_id veto already refuses, and which is exactly what a base merge mints (that is how PR #11987 escaped, leaving the class behind for this card).

What the verdict now does

credentialverdict
earlier attempt of THIS run, no declared leg accounts for itdiscarded, with a log line naming it and both attempts
earlier attempt of THIS run, a declared leg DOES account for itcounted, and marked carried over in the roster listing
CURRENT attempt, no declared leg accounts for itrefused, in the same words as before
another run, any attempt it claimsrefused (run_id veto, unchanged)
absent / unreadable run_attempt, on either side of the comparisonjudged exactly as before

The strictness is not weakened: the tolerance demands a positive "this run, an earlier attempt", and an unreadable attempt buys no exemption — so the worst a garbled field can do is restore the strict verdict, never a silent green. Discarding also grants no pass on its own: that branch only ever ADDED an error, and the green in the measured sequence comes from #4928's expected-0, which the discarded credential never enters.

Reverse verification (3 ablations, each restored by an EXIT INT TERM trap)

Mutations were applied with an exact-literal mutator that exits non-zero unless the anchor matches exactly once, and each was confirmed on disk by grepping for the injected marker AND for the deleted text before anything was read. This gate runs from source (node scripts/…), so no build/dist step stands between the edit and the measurement.

ablationpredictedobserved
A — tolerance removed (carriedOver always false)red on the measured-sequence limbred, 4 failures, all of them that limb
B — attempt comparison dropped (tolerate any same-run credential)red on the fail-closed limbred, 19 failures — including the pre-existing #4928/#6082 contradiction pins
C — run_id half of the tolerance droppedred on the foreign-run pinred, exactly 1 failure: the assertion written for it

Ablation A leaves the partial-re-run assertion green, which is the correct asymmetry: without the discard path those credentials are still on the roster and still counted. Ablation B is the one that matters for the strictness ruling — a blanket tolerance takes the pre-existing contradiction pins down with it.

Gates

Families derived at the final commit with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (no hand-written path list) — 9 matched, all run, all green at d288d27c6:

check:agent-test-spelling · check:cross-package-test-inputs · check:entry-guard · check:parse-guard · check:pnpm-filter-targets · check:shard-attestation · check-ci-filter-parity.mjs · check-cross-package-test-inputs.mjs · check-shard-attestation.mjs

check:shard-attestation verdict at that commit:

✓ check-shard-attestation --self-test: 129 assertions (dominance experiment + both #6082
counter-examples + the #4928 guard + the #6589 classifier pins + the #10889 quoting pins
+ the #11998 attempt-scoping sequence).
✓ check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s).

ESLint was run narrowed to the changed file, declared as a narrowing with its three measurements: the file is in ESLint's own linted population (its --format json result carries 0 warnings, so it was linted rather than ignored); the file count is 1, read from that same JSON; and this repo's single eslint.config.mjs enables type-aware linting for no file at all (no parserOptions.project, no typed rules — the config says so at its line 325 with a measured positive control), so a one-file diff cannot move the verdict on any untouched file. The repo-wide sweep is CI's run.

No changeset: CI plumbing under scripts/, nothing published — skip-changeset.

Note on body bytes: this PR body previously rendered needs. with an empty code span where needs. + an angle-bracketed placeholder + .result was written — GitHub's body sanitizer removed the angle-bracket token inside backticks. Rewritten without angle brackets.

Generated by Claude Code

Artifacts live on the RUN, not the attempt, so every attempt after the first
downloads whatever the earlier ones published. `judge()` compared only the run
id -- one level coarser than the artifact namespace it judges -- so after a
pre-judgement `filter` death (fail-open per THE FILTER CONTRACT #4928, which is
untouched here) `rerun_failed_jobs` could never converge: the legs correctly
stayed skipped on attempt 2, the roster adjusted to expected-0, and attempt 1's
own credentials were then refused one by one as "unexpected attestation ... that
no declared leg accounts for". Measured on run 32806633489.
The credential already carried `run_attempt` and the verdict already PRINTED it
-- it just never COMPARED it. So this is a verdict change only: no payload
change, and nothing in ci.yml moves.
The rule is "latest per shard WITHIN ONE RUN", not "only this attempt's":
- a credential from an earlier attempt of this run that no declared leg
accounts for is discarded with a log line, granting no pass (the green comes
from #4928's expected-0, which it never enters);
- a credential from an earlier attempt that a declared leg DOES account for
still counts. `overwrite: true` on every upload means a re-running leg
replaces its own artifact, so what survives is the credential of a leg that
did NOT re-run -- and that leg keeps its earlier conclusion in
`needs.<job>.result`. Refusing it would turn the ordinary rerun_failed_jobs
case (one flaky shard) permanently red: the same defect, one button over;
- everything else keeps its old verdict. The tolerance demands a positive "this
run, an earlier attempt": a foreign credential from the CURRENT attempt is
still refused in the same words, so is one from another run, and an absent or
unreadable attempt buys no exemption.
--self-test grows the measured sequence (both gates), the current-attempt
counter-limb, the partial-re-run pin that blocks the "current attempt only"
mis-fix, and the fail-closed cases on both sides of the comparison.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6
@yinlianghuiyinlianghui added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 25, 2026 — with Claude
@yinlianghui
yinlianghui marked this pull request as ready for review August 25, 2026 07:39
@yinlianghui
yinlianghui added this pull request to the merge queueAug 25, 2026
Merged via the queue into main with commit 402113dAug 25, 2026
35 checks passed
@yinlianghui
yinlianghui deleted the claude/issue-11998-attestation-run-attempt branch August 25, 2026 07:57
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@yinlianghui@claude