Skip to content

ci(devx): run measure-stall-guard-headroom's --self-test in the lint farm - #13358

Merged
os-elon merged 1 commit into
mainfrom
claude/issue-13246-stall-guard-selftest-wiring
Aug 30, 2026
Merged

ci(devx): run measure-stall-guard-headroom's --self-test in the lint farm#13358
os-elon merged 1 commit into
mainfrom
claude/issue-13246-stall-guard-selftest-wiring

Conversation

@os-elon

Copy link
Copy Markdown
Collaborator

Fixes#13246

The stall-guard family has three members and only two were wired. scripts/measure-stall-guard-headroom.mjs ships a --self-test that drives its real main() over fixture payloads and fixture workflow trees on disk, and no check:* family ran it, so its assertions never executed in CI.

Re-derived on this tree, not inherited from the card

32 assertions, re-measured at merge base 3ec8646f1b:

$ node scripts/measure-stall-guard-headroom.mjs --self-test
measure-stall-guard-headroom --self-test: 32 assertion(s) passed.
real 0m1.472s

The figure moved 21 to 32 inside #13245 and 32 is what the live tree carries.

The sibling's wiring shape is exactly two registration points and nothing else. Sweeping the whole tree for check:stall-guard-budget returns two hits, package.json:146 and .github/workflows/lint.yml:3471 (a run: step in the typecheck-source-gates job). No roster document, no needs: membership, no required-context entry. This PR copies that shape and adds nothing else.

--self-test is pure — measured, not reasoned. The flag never reaches fetchRunJobs: the entry point dispatches on it before main(), and main() only fetches for --run, which the self-test never passes. Measured with every outbound path armed to throw (net.Socket.prototype.connect, net.connect, net.createConnection, dns.lookup, dns.promises.lookup, globalThis.fetch) and with GITHUB_TOKEN / GH_TOKEN removed from the environment:

$ env -u GITHUB_TOKEN -u GH_TOKEN node --import ./no-net.mjs scripts/measure-stall-guard-headroom.mjs --self-test
measure-stall-guard-headroom --self-test: 32 assertion(s) passed.
EXIT=0

So the wiring is the two-line kind, not a design question. Its inputs are this repo's own .github/workflows directory plus temp fixtures it creates and removes; git status is clean after a run.

Why it was invisible to the gate that exists for this

check-self-test-wired.mjs enforces exactly this property, and it was right to stay silent: its population is scripts a workflow names, and no workflow named this one. Being unwired kept it out of the gate for being unwired. After this PR it is in the population and satisfied, confirmed by querying the gate's own collectInvocations rather than by reading its summary line:

named by CI : lint.yml (pnpm check:stall-guard-headroom)
self-tested by CI: lint.yml (pnpm check:stall-guard-headroom)

Only the --self-test leg

The two siblings at package.json:145-146 run --self-test && the gate. This one must not. The measurement path fetches GET /actions/runs/{id}/jobs and reads GITHUB_TOKEN / GH_TOKEN, so a bare invocation reds the farm on a token-less runner. The asymmetry is recorded in three places so the tidy-up that breaks it has to argue with something: the npm script, the workflow step comment, and the script header.

The script header was the actual cause, so it is corrected

The header said the tool "is NOT a CI gate and is deliberately not wired into any workflow: it needs the network and a token". That sentence is true of the tool and false of the flag, and it is what the exclusion was read off. It now separates the two and names the flag as the exception.

git diff for the script is comment-only: 0 changed lines outside the docblock, mechanically checked.

$ git diff 3ec8646f1b HEAD -- scripts/measure-stall-guard-headroom.mjs | grep -E '^[+-]' | grep -v '^[+-][+-][+-]' | grep -vcE '^[+-] \*'
0

Ablation: the new step can go RED, and the red names the assertion

A step wired into CI that cannot fail is the defect class this repo keeps finding (#4690, #13173), so this was measured rather than assumed. The gate runs node scripts/... from source, so there is no dist/ for a mutation to have to reach and no build leg to the ablation; landing the mutation on disk is the whole preflight, and it was proved before any reading was taken.

Mutation: restore the pre-#13121 defect in attribute() — when no candidate site survives identity matching, fall back to the first name-keyed candidate instead of excluding the observation.

Mutation confirmed on disk before reading, by marker count and by object hash:

injected marker present: 0 -> 1
hash now : a0e92470c96cc07e5db98c5683dc95b40483dc81
hash at HEAD : 73d8e93d057067891220052bc2eb4d94f0fe4f20
scripts/measure-stall-guard-headroom.mjs | 1 +

MUTATED, invoked exactly as the workflow invokes it:

$ pnpm -s check:stall-guard-headroom
measure-stall-guard-headroom --self-test: 2 of 32 assertion(s) FAILED
- an observation from a job that is not the guarded one is not attributed to it
- ...and the exclusion is reported, not swallowed
MUTATED EXIT=1

and the printed evidence shows the old defect wearing its old face — a guarded site quoting worst on 'Some Other Job'.

RESTORE leg, given the same treatment as the mutation leg rather than trusted:

marker after restore : 0
hash after restore : 73d8e93d057067891220052bc2eb4d94f0fe4f20 (byte-identical to the HEAD blob)
git diff HEAD for the file is empty: YES
$ pnpm -s check:stall-guard-headroom
measure-stall-guard-headroom --self-test: 32 assertion(s) passed.
RESTORED EXIT=0

So the step reds, it reds through the npm script the workflow runs, and the two assertions it reds on are precisely the #13121 refusal branches this card exists to stop being decoration.

Local verification

All at final commit 618163546a, exit codes captured before any pipe.

Gate family derived from the diff by node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (not a hand-written list) — 25 families, all exit 0: check:agent-test-spelling, check:bash32-floor, check:cli-command-ids, check:entry-guard, check:node-version, check:nul-bytes, check:parse-guard, check:pnpm-acquisition, check:pnpm-filter-targets, check:required-contexts, check:shard-attestation, check:stall-guard-budget, check:stall-guard-headroom, check:turbo-task-graph, check:type-check-coverage, check:watch-hint-literal, check:workflow-status-functions, check-aggregator-roster, check-ci-filter-parity, check-self-test-wired, check-self-test-workflow-commands, check-step-collectors, check-whole-set-label-write, check-drift-comment, pm/ci-failure --self-test.

Both convention-triggered obligations the derivation named for editing a gate script, green: scripts/pm/bare-root-worklist.mjs --self-test (50 live rows, 43 recorded verdicts, none stale, none missing, none contradicted) and pnpm check:pm-dispatch-gates (897 cases). No ledger row was needed: the new family's literal carries a separator, which that worklist refuses as already visible.

Repo-wide ESLint run in full rather than narrowed — pnpm lint = eslint . --no-inline-config, through the shared verify lock: VERDICT command-exit 0 · held the lock 58s · waited 0s.

.github/workflows/lint.yml re-parsed with yaml after editing; the job now carries three stall-guard steps and 33 steps in total.

Declarations

  • Contract accept/reject behaviour: UNCHANGED. No spec, schema, error code or envelope is touched. The diff adds a CI step and corrects a comment.
  • Published surface: UNCHANGED. Root package.json is private: true; scripts/ and .github/ publish nothing. No package version, export or type moves.
  • skills/ : NOT TOUCHED. No file under the published skills directory is in this diff, so the two line-count readings that clause asks for do not apply.
  • Script behaviour: UNCHANGED, mechanically: 0 changed lines outside the docblock, shown above. The wiring adds a caller, not a capability.
  • No changeset, and skip-changeset is applied: this PR releases nothing.

One observation, reported rather than fixed here

scripts/pm/dispatch-gates.mjs reads this script's DEFAULT_REPO = 'objectstack-ai/objectstack' as a path-population literal, so the newly discovered family prints as dead: 'objectstack-ai/objectstack' — no tracked path under its first segment; never was a repo path. That literal is an owner/repo slug, not a path. It is a pre-existing species rather than something this PR invents — check:driver-memory-census and check-test-completeness are on that list today for package names — and it is not CI dormancy, since typecheck-source-gates carries no paths filter and runs on every pull request. The cost is narrower: a card editing only .github/workflows/ is not told to run this gate locally, even though workflow text is what it reads. Left for triage rather than smuggled into a wiring PR.

Generated by Claude Code


Generated by Claude Code

…farm
The stall-guard family has three members and only two were wired. Its
measurement tool ships a `--self-test` that drives the real `main()` over
fixture payloads and fixture workflow trees, and no `check:*` family ran it,
so its 32 assertions never executed in CI.
The header's stated reason for keeping the tool out of the farm -- "it needs
the network and a token" -- is true of the TOOL and false of the FLAG. The
flag takes no `--run`, which makes `fetchRunJobs` unreachable from it.
Measured with net.Socket.connect, net.connect, dns.lookup and globalThis.fetch
all armed to throw and with GITHUB_TOKEN/GH_TOKEN unset: 32 assertions passed,
~1.5s, no build, no socket.
Wires ONLY the self-test leg. The two siblings run `--self-test && the gate`;
appending a bare invocation here would red the farm on a token-less runner,
and the header now records that asymmetry so the next reader does not tidy it
away.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
@github-actionsgithub-actionsBot added size/s ci/cd dependencies Pull requests that update a dependency file labels Aug 30, 2026
@os-elonos-elon added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed ci/cd dependencies Pull requests that update a dependency file labels Aug 30, 2026 — with Claude
@os-elon
os-elon marked this pull request as ready for review August 30, 2026 07:57
@os-elon
os-elon enabled auto-merge August 30, 2026 07:57
@os-elon
os-elon added this pull request to the merge queueAug 30, 2026
Merged via the queue into main with commit 0079e25Aug 30, 2026
43 checks passed
@os-elon
os-elon deleted the claude/issue-13246-stall-guard-selftest-wiring branch August 30, 2026 08:30
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/sskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] measure-stall-guard-headroom's --self-test is wired into no check:* family, so its 32 assertions never run in CI

2 participants

@os-elon@claude