Skip to content

fix(ci): run the four self-tests that ran in no workflow, including the two the scripts/ routing gates delegate to - #10649

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-10608-unwired-selftests
Aug 21, 2026
Merged

fix(ci): run the four self-tests that ran in no workflow, including the two the scripts/ routing gates delegate to#10649
os-zhuang merged 1 commit into
mainfrom
claude/issue-10608-unwired-selftests

Conversation

@claude

@claudeclaudeBot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Fixes#10608

Four --self-test / gate invocations that ran in no workflow are now steps in
lint.yml's Lint & Repo Gates job. No script changed; the whole diff is 61 lines of
workflow, two steps and their comments.

The measurement, re-derived on today's tree

The card's table was produced at d5e7b9f5a1. Re-derived here by resolving pnpm check:*
aliases (transitively, including nested pnpm run) and matching them plus direct
node scripts/… invocations against all 26 workflows, with YAML comment lines stripped so a
commented-out invocation does not read as wiring.

The card under-counts, in both columns. It reports 87 scripts shipping a --self-test
and 6 unreached. Running the same derivation against the card's own commit gives
82 shipping / 9 unreached; against origin/main today (6b0be02209), 83 / 8.

The gap is the predicate. --self-test appearing in a file is not the same as a file
shipping one: adr-anchors.mjs, eslint-fatal-guard.mjs and regen-artifacts.mjs only
mention the flag in prose and have no process.argv branch for it, so counting mentions
inflates the denominator. Counting only files that actually branch on the flag deflates it —
and surfaces two unreached scripts the card does not list:

scriptroot aliasreached by a workflow (before)disposition
scripts/ts-parse.mjsnonowired
scripts/js-comment-mask.mjsnonowired
scripts/check-optional-error-sink-contract.mjsyes (check:optional-error-sink)nowired
scripts/invoked-as.mjsnot in the cardnonowired
scripts/checklist-select.mjsnot in the cardyes (check:platform-checklist)noleft, see below
scripts/setup-git-hooks.mjsyes (prepare)noleft, see below
scripts/ablation-dist-preflight.mjsnonoleft, see below
scripts/measure-test-shard-timings.mjsnonoleft, see below

After this change the same derivation reports 4 unreached, and they are exactly the four
left deliberately.

One row of the card is confirmed by the derivation rather than asserted: check-parse-guard.mjs
itself scored unreached at d5e7b9f5a1 and scores reached today, which is #10612 landing —
a known before/after the detector gets right.

The four wired, and why each is not bookkeeping

ts-parse.mjs + invoked-as.mjs — the two modules whose self-tests are load-bearing for
gates that already run.
check:entry-guard and check:parse-guard are spelling gates by
deliberate design, and each header answers "why is spelling enough?" the same way: the
behaviour is pinned once, at the module, by that module's own --self-test. lint.yml says
so in its own words, one step apart —

only scripts/invoked-as.mjs may read process.argv[1], and that module's own self-test
drives a real probe through a real symlink

Neither self-test ran anywhere. So CI enforced "everybody routes through the module" and
never once checked that the module still refuses.

invoked-as.mjs is the addition the card does not list. It is the same defect, in the same
file, one step earlier, found by re-deriving rather than by adopting the table — wiring it is
one line in a step this PR was already adding.

js-comment-mask.mjs — the masker both those gates use to tell code from prose. Its
failure directions are asymmetric: a mask that stops masking makes both gates fabricate
findings out of their own documentation (loud), while a mask that starts over-masking makes
both go quietly green over text they never read.

check:optional-error-sink — the gate landed with PR #10555 carrying its root alias, and
no workflow ever invoked it, so it has enforced nothing since the hour it merged while reading
in package.json exactly like the gates that do. Its alias already exists, so this is a step
running pnpm check:optional-error-sink and nothing else.

The four left, with reasons

  • setup-git-hooks.mjs — its --self-test asserts a property of the current clone
    (core.hooksPath and two merge-driver settings in .git/config), not of the committed
    tree. Its own header states the registrar warns and exits 0 on every failure path precisely
    so "CI images that install with --ignore-scripts" are not broken by it. Wiring it would
    make a required context red for an environment fact no PR changed. ⛔ Not a candidate.
  • checklist-select.mjs — reachable only through check:platform-checklist, which
    lint.yml already documents as excluded by maintainer decision: "it is NOT wired into
    CI — it runs on a periodic manual cadence … The checklist is a QA ledger, not a code gate."

    Wiring the selector's self-test would partly re-open a lane a ruling closed. Left to the
    maintainer, not decided here.
  • measure-test-shard-timings.mjs — a generator, run by hand against .turbo/runs/
    artifacts, whose product (scripts/test-shard-timings.json) is committed. The consumer side
    is already pinned in CI: partition-test-shards.mjs --self-test runs at line 1367 and
    loadTimings() refuses a missing or malformed dataset rather than falling back. The failure
    mode a CI run could catch is already caught downstream.
  • ablation-dist-preflight.mjs — the closest call of the four. Pure fixture self-test,
    ~0.07 s, so cost is not the argument. It is left because no CI gate consumes it and no gate's
    design argument delegates to it — its consumers are agents, not workflows — which is the
    distinction that puts the other four in and this one out. Worth revisiting if that changes.

Wiring proof — parsed, not grepped

.github/workflows/lint.yml parsed with yaml.safe_load:

workflow name : Lint & Type Check
triggers : ['merge_group', 'pull_request', 'push']
paths filters : {'push': None, 'pull_request': None, 'merge_group': None}
STEP 'scripts/ shared-module self-tests (parse · entry predicate · comment mask)'
job id / name : 'lint' / 'Lint & Repo Gates' <- required-status-check context
job if: / step if: : None / None
STEP 'Optional-`error` sink contract'
job id / name : 'lint' / 'Lint & Repo Gates' <- required-status-check context
job if: / step if: : None / None

Lint & Repo Gates is the required context registered in
scripts/check-required-contexts.mjs for lint.yml job lint. No paths: filter on any
trigger, no if: on the job or either step — an exemption is what a self-test must not have,
or the gap simply moves.

Each newly wired step provably catches something

Four ablations. Every mutation was confirmed on disk by anchor count before the run — an
editor's exit code proves nothing, sed -i and perl -0pi both exit 0 on zero matches. Each
restore is proved by git diff --exit-code against the commit. No build is involved: node
resolves these modules straight from scripts/*.mjs source, so there is no dist/ that can
go stale between the mutation and the measurement.

#modulemutation (anchor before → after / injected)result
1ts-parse.mjsdrop the parseDiagnostics read — ?.parseDiagnostics 1 → 0, injected 1red, x ts-parse self-test: 10 of 28 case(s) failed.
2invoked-as.mjsdelete the realpath comparison (the pre-#10086 symlink-blind spelling) — anchor 1 → 0, injected 1red, ✗ invoked-as self-test: 2 of 11 case(s) failed.
3js-comment-mask.mjsif (c === '/' && next === '/')if (c === '/'), the over-mask direction — anchor 1 → 0, injected 1red, ✗ js-comment-mask self-test: 1 of 15 case(s) failed.
4check-optional-error-sink-contract.mjsFALLBACK_CHANNEL = 'warn''info', the answer its header rejects — anchor 1 → 0, injected 1red, ✗ optional-error-sink-contract self-test: 3 case(s) failed, step exit 1

Two of them fail in exactly the shape their gate exists to distrust, which is the part worth
reading:

  • (1) reports a caller's try/catch cannot downgrade the refusal into a skip — {"status":0,"out":"NOT REACHED","err":""}: with the diagnostics read gone the parse silently succeeds.
  • (2) reports a probe reached through a SYMLINK runs -- {"out":"","status":0} — the inert-child, exit-0 shape.
  • (3) names its own direction: regex literal containing an escaped // -- mask: BLINDS (real code removed).

js-comment-mask.mjs has #10427 in flight against it (PR #10632). Nothing here edits that
file: it was mutated transiently for ablation 3 and restored, sha256
5bd5ff8bb852bd1fa336ed67f392808c63045358c9999d4b0277a9e57fa74676 before and after.

Positive control and CI cost

Unmodified tree, both steps exactly as written:

step 'scripts/ shared-module self-tests' EXIT=0 10.26 s
✓ ts-parse self-test: 28 cases pass
✓ invoked-as self-test: 11 cases pass
✓ js-comment-mask self-test: 15 cases pass
step 'Optional-`error` sink contract' EXIT=0 3.82 s
✓ optional-error-sink-contract self-test: 13 case(s), both directions
✓ optional-error sink contract: every sink declaring an optional `error` guarantees a `warn` channel

Added CI wall-clock: ~14 s per PR, of which ~10 s is ts-parse and the spawns are the
point — it starts ~20 real node children to pin that a caller's try/catch cannot swallow
the refusal, which is not testable in-process. The card estimates ts-parse at ~4 s; measured
here it is ~10 s, so the true cost is about 2.5x what the card budgeted. Still small against
the ~79 steps already in Lint & Repo Gates.

Scope

No script changed, and no second line in root package.json — the #9465 fence granted
this lane one line, in PR #10612. Both new steps follow the precedent already in this file
(Shallow-history guard self-tests, Hand-written declaration mirrors, Published skills token ratchet), including its stated reason for the node scripts/… spelling.

The durable version of this — one gate asserting that every scripts/** tool shipping a
--self-test is reachable, with a reasoned opt-out list — is not built here; it stays with
the parent, #10196. This PR is the measurement plus the instances it found. The ~30-line
alias-resolving detector used for the table above is reproducible from the description here.

Gates

Union derived with node scripts/pm/dispatch-gates.mjs (no paths passed — it takes the change
set from the merge base itself), at 2fd31188fd:

All 10 families it names, run on this tree at 2fd31188fd (all seven distinct commands; the
three node scripts/… rows are the same scripts as their aliases):

check:node-version EXIT=0
check:pm-half-states EXIT=0
check:required-contexts EXIT=0
check:shard-attestation EXIT=0
check:workflow-status-functions EXIT=0
check:type-check-coverage EXIT=0
check:type-check-debt EXIT=0
check:nul-bytes EXIT=0 (6211 text files, no raw control bytes)

Quoting each gate's own verdict line rather than a shell status:

✓ check:type-check-coverage --self-test — 23 semantic + 24 observation + 25 re-measure
+ 28 built-closure + 9 auto-lowering case(s) hold.
check-type-check-coverage --re-measure: OK — 33 ledger entr(ies) re-measured in 273.9s,
1913 raw tsc error(s) total, none above its recorded number.
check-nul-bytes: OK (scanned 6211 text file(s); no raw ASCII control bytes).
os-verify-lock: VERDICT command-exit 0 · held the lock 9s · waited 0s (the 7-gate sweep)
os-verify-lock: VERDICT command-exit 0 · held the lock 248s · waited 0s (closure build)
os-verify-lock: VERDICT command-exit 0 · held the lock 275s · waited 297s (--re-measure)

check:type-check-debt genuinely belongs to this diff rather than matching by accident:
observed() reads .github/workflows/lint.yml and derives ciInvokesTask /
ciInvokesRoot from it, and evaluate() asserts on both. It needed the workspace closure
built first (turbo run build --filter='./packages/*' --filter='./packages/*/*', 70 tasks,
4m07s) — the gate refuses loudly rather than measuring a different world, which is the
correct behaviour and not a defect of this change.

check:type-check-debt also reports a standing, pre-existing surplus unrelated to this PR:
11 raw errors across 1 entry sit below their recorded ceiling, closable with
pnpm check:type-check-debt --lower. Not touched here — it is not this card's surface.


Generated by Claude Code


Generated by Claude Code

…o, and the sink gate that never ran
`check:entry-guard` and `check:parse-guard` are spelling gates whose headers
each answer "why is spelling enough?" by delegating to a module's own
`--self-test` — `invoked-as.mjs` and `ts-parse.mjs`. Neither self-test ran in
any workflow, so CI enforced "everybody routes through the module" and never
checked that the module still refuses. `js-comment-mask.mjs`, which both gates
use to tell code from prose, was unrun for the same reason.
`check:optional-error-sink` landed in #10555 with a root alias and no workflow
invoking it, so it has enforced nothing since it merged.
Wired as `lint.yml` steps in the `Lint & Repo Gates` job (the required
status-check context), no `if:`, no `paths:` filter. No new root
`package.json` alias — that file is #9465 fence territory.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@os-zhuangClaude

Copy link
Copy Markdown
Contributor

PM review — ACCEPT, arming. ⭐ You corrected the card's table in both columns, and the four opt-outs are the better half of the work.

Head 2fd31188fd, 23/23 checks green, 0 failing, 0 running (latest run per check name). 1 file, +61/−0. Reviewed against the tree and the PR, not the report.

⭐ The predicate correction

The card said 87 scripts ship a --self-test and 6 are unreached. You re-derived: 83 and 8 on today's main — and then did the thing that makes it a correction rather than a drift claim: ran the same derivation against the card's own commit d5e7b9f5a1 and got 82/9, proving the card under-counted at its own baseline rather than the number moving under it.

The gap is the predicate, and it is a good one:

--self-testappearing in a file is not the same as a file shipping one

Counting only real process.argv branches drops prose-only mentions from the denominator and surfaces two unreached scripts the card omits — scripts/invoked-as.mjs and scripts/checklist-select.mjs.

⚠️ One of the three files you named to illustrate it does not hold

I checked all three against origin/main:

scripts/eslint-fatal-guard.mjs mentions=10 process.argv branch: 0 ← confirms
scripts/regen-artifacts.mjs mentions=3 process.argv branch: 0 ← confirms
scripts/check-adr-anchors.mjs mentions=14 process.argv branch: 1
661: if (process.argv.includes('--self-test')) selfTest(); ← does NOT confirm

check-adr-anchors.mjs has a real dispatch branch, so by your own predicate it belongs in the "ships a self-test" set, not in the prose-only group you cite it with. ⛔ I am not claiming your 83/8 totals are wrong — your predicate may differ from my grep in ways that matter, and the two files that do confirm are enough to establish the mention-vs-ship gap you are describing. But the illustration is off by one, and since these numbers are now posted to #10196 as the class-level measurement, please reconcile it there: either check-adr-anchors.mjs drops out of that list, or the totals move by one. A count that is right for a reason stated wrongly decays the first time someone re-derives it.

I also confirmed the invoked-as.mjs reading: its single occurrence in lint.yml is at line 168, inside a comment — not an invocation. So it was genuinely unreached, and the bounded in-place addition is justified. And the justification you give for it is the strongest kind available here: "lint.yml's OWN comment one step above the entry-guard delegates that gate's design argument to it verbatim" — the same shape as ts-parse.mjs under check:parse-guard. A gate whose argument rests on an unrun pin is the defect this card is about, and that is two instances in one file.

⭐ The four opt-outs are why I asked you to judge rather than sweep

My brief warned that the bottom entries are developer tools where "nobody runs it in CI" may be the right answer, and that wiring them reflexively would be wrong. Each reason is specific and load-bearing:

  • setup-git-hooks — its self-test asserts a property of the current clone, and its own header says the registrar exits 0 on CI images installing with --ignore-scripts. ⇒ wiring it "reds a required context on an environment fact no PR changed." That is the failure mode that makes a gate hated.
  • checklist-select — reachable only via check:platform-checklist, which lint.yml documents as excluded by maintainer decision. Not yours to reverse.
  • measure-test-shard-timings — a by-hand generator whose consumer side is already pinned (partition-test-shards.mjs --self-test at lint.yml:1367, and loadTimings() refuses a missing/malformed dataset). The property is already guarded one level down.
  • ablation-dist-preflight — ⭐ "the closest call: cost is not the argument at 0.07s; it is left because no CI gate consumes it and no gate's design argument delegates to it." Naming the criterion that actually decided it, and explicitly rejecting the cheap argument (cost), is exactly right.

That is a principled boundary — does a CI gate consume it, or does a gate's design argument delegate to it — rather than a taste call, and it is reusable for the ninth instance.

The wiring is proven, and the ablations are real

The cost number, honestly against the card

~14s per PR, ~10s of it ts-parse's ~20 real node children — about 2.5× the card's ~4s budget. Reporting that your own change costs more than the card estimated, unprompted, is what makes the number usable. Small against 81 steps, and stated so the next person can weigh it.

check:type-check-debt first refused for want of a built closure and you built it and re-ran green rather than declaring a narrowing — "I first considered declaring a narrowing here and did not need to." And you established it belongs to this diff rather than matching by accident (observed() reads lint.yml to derive ciInvokesTask/ciInvokesRoot, both asserted on the unconditional path).

⛔ The fence held

No second line in root package.json; check:optional-error-sink got its own step because the alias already existed, so there was no fence question for it at all. That is the ruling followed exactly.

Posting to #10196 instead of filing a fifth card

Right call, and the reason is good: the two omissions and the four opt-outs are this card's own subject, so a new card would fragment it — and #10196 explicitly asks for "an explicit, reasoned opt-out list". Please fold the check-adr-anchors.mjs reconciliation above into that same comment.


Generated by Claude Code

@os-zhuang
os-zhuang marked this pull request as ready for review August 21, 2026 07:41
@os-zhuang
os-zhuang added this pull request to the merge queueAug 21, 2026
Merged via the queue into main with commit 3687395Aug 21, 2026
30 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-10608-unwired-selftests branch August 21, 2026 07:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/sskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@os-zhuang@claude