Skip to content

ci(lint): run scripts/pm/ci-failure.mjs --self-test in Lint & Repo Gates - #10195

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-9898-ci-failure-selftest-in-ci
Aug 20, 2026
Merged

ci(lint): run scripts/pm/ci-failure.mjs --self-test in Lint & Repo Gates#10195
os-zhuang merged 1 commit into
mainfrom
claude/issue-9898-ci-failure-selftest-in-ci

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes#9898

scripts/pm/ci-failure.mjs shipped a --self-test that no job ran. Its four
sibling PM tools all run theirs as unconditional steps in the existing
Lint & Repo Gates job; this one was the odd one out, so it could rot until an
agent reached for it mid-round and found it broken.

This adds one step to the existing lint: job. No new job, no new check
name, no new required context.

Why this tool specifically

The rot it prevents is the exact defect the tool's own card was about. The file
was found near-complete but never run live: node 22's fetch ignores
HTTPS_PROXY, so behind an agent container's proxy every read answered 401 and
the tool's own transport probe reported PREREQUISITE NOT MET (exit 3) — and
looked right doing it. A retrieval tool that has stopped working presents as a
tool correctly declining to work.

The self-test is also materially larger than when the card was filed: #9966 /
PR #10157 (merged today) corrected the transport probe and added nine cases,
including the fourth measured container class. This card was deliberately
ordered after that fix — wiring a false green into a required job would have
pinned it as CI-enforced truth.

Falsification 1 — the step is in the right job (job key + line)

The card's own warning: a gate added to typecheck: shows green ticks while
blocking nothing, reproducing this card's failure mode as its fix. Read from
.github/workflows/lint.yml, not from the card:

job keylinename: (= check-run / required context)
targetlint:24Lint & Repo Gates (line 39)
traptypecheck:1509 (was 1462)TypeScript Type Check

Those are the only two jobs in the file. The trap is live: typecheck: really
does carry self-test steps of its own, so "this file has self-test steps" would
not have identified the right job.

Parsed rather than grepped — all five land in lint / Lint & Repo Gates, none
carries an if::

stepjob keyjob nameif:
PM dispatch-gates self-testlintLint & Repo Gatesnone
Part-of closing-keyword guard self-testlintLint & Repo Gatesnone
Single-claim path guard self-testlintLint & Repo Gatesnone
PM half-state sweeper self-testlintLint & Repo Gatesnone
PM ci-failure self-test (new)lintLint & Repo Gatesnone

The new step is index 23 of 74 in jobs.lint.steps, keys ['name', 'run'].

No required-context registry edit is needed, and none is possible by accident:
scripts/check-required-contexts.mjs assertion 10 explicitly forbids a registry
entry from embedding a step count, precisely so adding a step cannot desync it.

Falsification 2 — it needs no network (measured, not inferred)

A self-test that reached GitHub would put this required context at the mercy of
API availability. "It worked here" is not that claim, so it was measured three
ways, each with its instrument falsified against this same file's live path:

reading--self-testlive walk (control)
strace -f, network available0socket(), 0connect()3 socket(AF_INET), 3 connect()
empty network namespace (unshare -rn)exit 0exit 3, GET /rate_limit did not complete: fetch failed
in-process egress tripwire0 attempts(see note)

The strace reading is the load-bearing one: zero socket syscalls with full
egress available
, following forks, so nothing is hidden in a subprocess.

Note on the tripwire, recorded because it nearly produced a false reading: its
fetch arm fires correctly, but its child_process arm is blind to a named
ESM import binding
, which is exactly how this file imports spawnSync. Caught
by falsifying the instrument rather than trusting its "clean". strace and the
namespace have no such blind spot.

Structurally this is not accidental: the --self-test branch is selected before
the transport probe, it never re-execs, and its two readers are injected.

Falsification 3 — the step runs, rather than no-opping

"I added the step and CI is green" does not distinguish a step that ran from one
that no-opped, so both no-op shapes were driven:

  • Broken self-test reddens. Mutating isNewer in the shipped file
    (return at > btat < bt; confirmed on disk by grep count 1→0 / 0→1, not
    by an editor exit code) makes the step's exact command exit 1 with two
    named failures. Restored to byte-identity with HEAD and re-run green.
  • A wrong path reddens (exit 1, ERR_MODULE_NOT_FOUND) rather than exiting
    0 the way a zero-match pnpm --filter would.

Runtime

~65–170 ms, median ~0.10 s (8 runs), against a job that already runs ~70 gate
steps including a ~15 s one. It reads files and runs pure predicates.

Gates

node scripts/pm/dispatch-gates.mjs with no path args, derived from the real
diff at b3955be: check:node-version, check:required-contexts,
check:shard-attestation, check:workflow-status-functions,
check:type-check-coverage — all green.

Extra, because editing lint.yml moves gates no path derivation names (found by
grepping for readers of the file): check:changeset-gate-self-tests (its
wiring half slices the lint: job out by anchors and asserts the slice is
non-empty — green, so the insertion did not stale those anchors),
check:filter-alias-parity, check-merge-queue-triage-outcome --self-test,
check-cross-repo-closer-outcome --self-test, check:nul-bytes — all green.

Declared narrowing: check:type-check-debt (--re-measure) was not run. It
re-runs tsc across the debt ledger to re-measure error counts; this diff
contains no TypeScript. CI runs it regardless.

skip-changeset: this diff is one workflow file and publishes nothing.

Generated by Claude Code


Generated by Claude Code

`scripts/pm/ci-failure.mjs` shipped a `--self-test` that no job ran, unlike
the four sibling PM tools (dispatch-gates, the part-of guard, the single-claim
guard, the half-state sweeper) which all run theirs as unconditional steps in
this same job. It was the odd one out, so it could rot until the next agent
reached for it mid-round and found it broken.
The rot this prevents is the exact defect the tool's own card was about: the
file was found near-complete but never run live, because node 22's `fetch`
ignores HTTPS_PROXY, so every read answered 401 and the tool's own transport
probe reported PREREQUISITE NOT MET (exit 3) — and looked right doing it. A
retrieval tool that has stopped working presents as a tool correctly declining
to work.
One step in the existing `lint:` job, whose `name:` (`Lint & Repo Gates`) is
the required-status-check context and is unchanged — a required context is
matched by check-run name, so a step adds no context. No new job, no new check
name. Unconditional and un-`if:`-ed, like every self-test beside it.
Measured, not assumed: under `strace -f` with full egress available the
self-test makes zero socket() and zero connect() calls (the live walk, traced
identically, makes 3 and 3), and it exits 0 inside an empty network namespace
where the live walk exits 3. Runtime ~0.07-0.17 s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 20, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review August 20, 2026 13:53
@os-zhuang
os-zhuang added this pull request to the merge queueAug 20, 2026
Merged via the queue into main with commit 7cb2f9fAug 20, 2026
23 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-9898-ci-failure-selftest-in-ci branch August 20, 2026 14:21
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/sskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scripts/pm/ci-failure.mjs's --self-test runs nowhere in CI, unlike its four sibling PM tools

2 participants

@os-zhuang@claude