Skip to content

fix(tooling): join runner observations to guarded sites on (file, job, step) - #13245

Merged
hotlong merged 1 commit into
mainfrom
claude/issue-13121-headroom-join-triple
Aug 30, 2026
Merged

fix(tooling): join runner observations to guarded sites on (file, job, step)#13245
hotlong merged 1 commit into
mainfrom
claude/issue-13121-headroom-join-triple

Conversation

@os-elon

Copy link
Copy Markdown
Collaborator

Fixes#13121

scripts/measure-stall-guard-headroom.mjs joined GitHub's runner observations onto guarded sites by step name alone, while a site is identified by the triple (file, job, step) — the triple the tool already prints in every one of its own lines.

The premise, measured before implementing

Re-derived on origin/main at 5a9b7a0b10: 7 guard-wrapped steps, all names distinct, and (file, job, step) unique across the population — so the defect was LATENT and no reading on this tree was wrong. No duplicate name has arrived since the card was written, and no fourth join component is needed. The self-test now asserts both facts, so the day either stops holding is the day the suite says so.

The card's sharpest claim is TRUE

The card says the error direction "is not even consistently conservative". It is not, and the failure is worse than a wrong number. Driving the pre-fix code over two co-named guarded steps in different jobs, with only the faster one actually executed:

 .github/workflows/a.yml job `tight` step `Run the suite`
W 10m · C 20m · T 30m (job timeout-minutes)
worst observed: p 1m00s + s 1m00s = 2m00s (1 observation(s), worst on `Loose Job`)
deferred p+s+C = 22m00s vs T 30m -> COVERED, 8m00s to spare

The tight site never ran in that payload. The honest line is NOT OBSERVED. Instead the tool printed a confident COVERED for it, quoting a reading from a different job in its own evidence line, and exited 0. Reversing the observation sets flips the error the other way — a genuinely covered site reported UNCOVERED by 50m00s. Both outputs are identical in shape to a correct run.

What changed

Attribution resolves every observation against the identity the workflow file declares for it — the workflow's name:, and the job's name: template with each expression standing for one expansion, so a real matrix leg such as Test Core (3/6) matches. A component the payload does not carry is UNKNOWN: it can neither confirm nor exclude a candidate.

  • one survivor — attributed;
  • none — EXCLUDED and reported, never dropped in silence; its site reads NOT OBSERVED, the direction that cannot invent headroom;
  • more than one — REFUSES. Picking the worst, the first or the average is the same defect wearing a verdict. Two sites identical on the triple itself land here too: no identity separates them, a fourth component would be needed, and the refusal says so.

The narrowing is unconditional, not reserved for co-named steps. "The triple when the name happens to be ambiguous" is the same defect with the collision size fixed at one, and it is reachable with no duplicate name: anywhere — an unguarded step copying a guarded step's name.

On the collection end (const wanted = new Set(swept.sites.map((s) => s.step))), asked about on the card: it stays keyed on the name, deliberately. As a filter the name yields a superset of what any per-site join can use, so it cannot under-collect. What did under-collect was the observation record — it discarded workflow_name, leaving the join blind on two of the three axes even when the payload carried them. That is what changed.

--root DIR is added because the refusal branch cannot be reached from this tree, and a refusal nobody can trigger is decoration.

check-stall-guard-budget.mjs is not touched: its T - C >= W criterion is deliberately independent of p and s, and this card is the join in the measurement tool. Its own self-test still passes 61 assertions.

Verification — all at 39f19b2fe0

--self-test: 32 assertions passed (was 21). Reverse proof, predictions named before each run, each mutation proven on disk by decomposed before/after grep -cF counts on the removed and injected text plus git hash-object, each restore proven by blob equality against the HEAD blob and an empty git diff HEAD, under an absolute-path trap. No rebuild leg applies — node loads this .mjs from source, with no dist on the resolution path.

ablationpredictedobserved
re-key the bucket on the step name (the historical defect)red; the co-named fixture cases only; the real-tree case stays green4 of 32 failed, and the tight site printed COVERED, 8m00s to spare quoting worst on \Loose Job`` — the defect reproduced verbatim. The real-tree case stayed green: that is the latency
drop the identity narrowingred; cases 9/10/14; the already-ambiguous cases stay green8 of 32 failed, exactly those

Derived gate family — node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (12 families, derived from the real change set, not a hand-built diff): check:agent-test-spelling, check:bash32-floor, check:cli-command-ids, check:cross-package-test-inputs, check:entry-guard, check:parse-guard, check:pnpm-filter-targets, check:watch-hint-literal, check-ci-filter-parity.mjs, check-cross-package-test-inputs.mjs, check-shard-attestation.mjsall green, each read from its own printed verdict line, each exit code captured before any pipe. check-test-completeness.mjs argument-less is a PREREQUISITE NOT MET refusal (exit 3), NOT a red: it grades a saved turbo run test log CI tees for it, so it is NOT MEASURED locally by design.

Repo-wide pnpm lint (eslint . --no-inline-config, the whole farm, no narrowing): VERDICT command-exit 0, run through scripts/pm/os-verify-lock.sh. check-nul-bytes: OK over 7342 text files, plus a direct control-byte scan of the changed file.

No changeset: the diff is one internal scripts/ measurement tool that is not published and not wired into any workflow, so this PR releases nothing — skip-changeset.

Filed separately, not fixed here

This tool's --self-test is not wired into any check:* family, so its 32 assertions — the refusal branches included — never run in CI, unlike its sibling's (check:stall-guard-budget). Wiring it needs a new lint-farm entry, which is a new verification surface, so it is recorded rather than smuggled in. See the report.

Generated by Claude Code


Generated by Claude Code

…, step)
`measure-stall-guard-headroom` keyed its join on the step NAME alone, an
identifier that is unique only inside one job used as if it were global --
the same mistake `attachSiblings` documents one level up on the job axis.
Two guarded steps sharing a `name:` merged into one bucket and each site was
then judged against the worst reading of the union. The failure is silent:
the output is identical in shape to a correct run. It is also not
consistently conservative -- with asymmetric observation sets a site that
never ran is printed MEASURED and COVERED, quoting another job's reading.
Attribution now resolves every observation against the identity the workflow
file declares (workflow `name:`, the job's `name:` template with `${{ }}`
standing for one expansion). One survivor is attributed; none is EXCLUDED and
reported; more than one REFUSES rather than picking the worst, the first or
the average.
`--root <dir>` is what makes the refusal demonstrable: no duplicate step name
exists in this tree, and a refusal nobody can trigger is decoration.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
@os-elonos-elon added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 29, 2026 — with Claude
@os-elon
os-elon marked this pull request as ready for review August 29, 2026 14:53
@os-elon
os-elon enabled auto-merge August 29, 2026 14:53
@os-elon
os-elon added this pull request to the merge queueAug 29, 2026
@github-merge-queue
github-merge-queueBot removed this pull request from the merge queue due to no response for status checks Aug 29, 2026
@hotlong
hotlong added this pull request to the merge queueAug 30, 2026
Merged via the queue into main with commit f27e1c2Aug 30, 2026
34 checks passed
@hotlong
hotlong deleted the claude/issue-13121-headroom-join-triple branch August 30, 2026 03:39
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

3 participants

@os-elon@hotlong@claude