Skip to content

fix(scripts): the two ESLint ratchets refuse a population that will not parse (#10123) - #10130

Merged
os-elon merged 2 commits into
mainfrom
claude/issue-10123-ratchet-fatal-parse
Aug 20, 2026
Merged

fix(scripts): the two ESLint ratchets refuse a population that will not parse (#10123)#10130
os-elon merged 2 commits into
mainfrom
claude/issue-10123-ratchet-fatal-parse

Conversation

@os-steve

@os-steveos-steve commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Fixes#10123

The hole

Both ratchets drive ESLint through its Node API over packages/** and count the
messages matching their own rule. ESLint does not throw on a file that fails to
parse — it returns the failure as an ordinary message with no rule id and
fatal: true, which matches neither filter (m.message === SLOT_LOOKUP_ANY_MESSAGE,
m.ruleId === QUERY_OPTIONS_RULE_ID). The file therefore contributed zero sites
and each gate printed ✓ … holds and exited 0 — output byte-identical to a file
that was measured and is clean.

Re-derived on this branch point (5262124388) against a packages/** file holding a
single syntax error:

lintFiles returned normally (did NOT throw)
file: packages/spec/src/__ratchet_fatal_probe__.ts
messages: [{"ruleId":null,"fatal":true,"severity":2,"message":"Parsing error: Expression expected.","line":2,"column":32}]
errorCount=1 fatalErrorCount=1

The same file through the root lint script's spelling (eslint on that path, with
--no-inline-config) exits 1. That asymmetry is what the issue flags: each ratchet's
own docblock argues it exists because a green pnpm lint proves nothing for the files
it covers, and these two were the ones going quiet.

What changed

  • scripts/eslint-fatal-guard.mjs (new) — the check, once. lintFilesStrict() wraps
    eslint.lintFiles(), collects every fatal message (cross-checked against ESLint's own
    per-result fatalErrorCount, so a future ESLint that moves the flag cannot make this
    silent again), and on any hit prints the file, the position and the parser's message and
    exits 2. Exit 2 is what both gates already reserve for "refusing to report clean"
    (renamed rule, rescoped population), as distinct from 1 = "the ratchet moved"; a parse
    failure belongs to the first family, because nothing moved — the measurement did not
    happen.
  • Both ratchets call lintFilesStrict() instead of eslint.lintFiles(), so the
    guard also covers their --update paths (measurement precedes the write in both).
  • Docblocks in both gates now state the refusal, since each one's stated rationale is
    the reason this mattered.
  • --self-test (query-options, which CI runs ahead of the gate) proves the guard in
    both directions over real ESLint output, and checkGuardAdoption() asserts from source
    that both gates still route through it. pnpm check:slot-lookup has no self-test hook,
    so that source assertion is the wired coverage of its call site.

Harm reproduction — both halves, both gates

An unparseable file placed in the population (packages/spec/src/__ratchet_fatal_probe__.ts,
never committed). Before — the two gates at the branch point, same tree, same file:

✓ slot-lookup ratchet holds: 107 unswept site(s) in 25 file(s), none new. …
SLOT_EXIT=0
✓ query-options-erasure ratchet holds: 67 unswept non-test site(s) in 17 file(s), none new. …
QO_EXIT=0

while eslint packages/spec/src/__ratchet_fatal_probe__.ts --no-inline-config said:

 2:32 error Parsing error: Expression expected
✖ 1 problem (1 error, 0 warnings)
ROOTLINT_EXIT=1

After — same file, this branch at 72038565f9, both gates:

✗ check-slot-lookup-ratchet: 1 parse failure(s) inside the population this gate measures:
• packages/spec/src/__ratchet_fatal_probe__.ts:2:32 — Parsing error: Expression expected.
…
EXIT=2
✗ check-query-options-erasure-ratchet: 1 parse failure(s) inside the population this gate measures:
• packages/spec/src/__ratchet_fatal_probe__.ts:2:32 — Parsing error: Expression expected.
…
EXIT=2

Counts unchanged — the issue's own acceptance criterion

gatebranch point 5262124388this branch 72038565f9after merging main, 87e46090c5
slot-lookup107 site(s) in 25 file(s)107 site(s) in 25 file(s)107 site(s) in 25 file(s)
query-options non-test67 site(s) in 17 file(s)67 site(s) in 17 file(s)67 site(s) in 17 file(s)
query-options test surface240 site(s) in 47 file(s)240 site(s) in 47 file(s)240 site(s) in 47 file(s)

Ablations — the new self-test can fail

Both legs are plain node scripts/… sources (no package exports, no dist), so no
rebuild is involved; each was restored by checking the file out of this branch again
(git checkout with the branch ref and the path), and the restored tree verified
byte-identical (git diff --stat HEAD empty).

  • Slot-lookup put back to a bare eslint.lintFiles() → the self-test goes red naming it:
    "calls .lintFiles( directly, so a parse failure in its population is discarded as a
    message matching no rule" (exit 1).
  • The guard's fatal filter broken (m.fatal swapped for a never-matching predicate) →
    red on the detail assertions (exit 1). Worth recording: the file was still caught,
    by the fatalErrorCount cross-check, but with the position and parser text lost — the
    second limb is real and is what the assertions measure.

Gates run locally at 87e46090c5

pnpm check:slot-lookup · pnpm check:query-options-erasure (self-test + gate) ·
pnpm check:cross-package-test-inputs · pnpm check:ratchet-remedy-authority (98 scripts
swept, the new one classified with the 89 handing out no ratchet-expanding remedy, so the
control corpus needs no entry) · pnpm check:nul-bytes · pnpm check:pm-dispatch-gates
(351 cases — it reads the slot-lookup gate's source for its population declaration) ·
eslint over the three changed files. All green. The union was re-derived after the final
commit with node scripts/pm/dispatch-gates.mjs (no paths), against merge base
2cd5f8113: same three paths, same four gates.

Deliberately not here

The stack headroom question the issue defers. #10124 landed while this branch was open
and is merged in here; it is comment-only in eslint.config.mjs plus the root lint
script, so it moved none of the counts above (re-measured at 87e46090c5, table). These
two gates still run as plain node scripts/… at V8's default stack and do not inherit
that headroom — the deferred question, untouched.

One datum for whoever picks it up, and it exists only because of this PR: both gates now
go green only if every file in the population parsed, so the green runs above are
positive evidence that packages/spec/src/migrations/registry.ts — the file #10124 is
about — parses inside these gates' own wide packages/** run at the default stack today.
That matches #10124's measurement that the wide scopes succeed and only a narrow one
crashes. The flip side is the honest risk to weigh: where the old behaviour was a silent
skip, this guard makes that file's stack margin a red gate if it is ever crossed.

Root scripts/ only, so skip-changeset rather than a changeset.


Generated by Claude Code

…ot parse (#10123)
Both ratchets drive ESLint through its Node API and count the messages that
match their own rule. ESLint does not throw on a parse failure — it returns it
as a message with no rule id and `fatal: true` — so an unparseable file matched
neither filter, contributed zero sites, and the gate printed `✓ … holds` and
exited 0: a clean verdict on a file it had never read, while `pnpm lint` failed
loudly on the same input.
The check lives once, in scripts/eslint-fatal-guard.mjs, and both gates route
their run through `lintFilesStrict()` instead of `eslint.lintFiles()`. A parse
failure now names the file, the position and the parser's message, and exits 2
— the code both gates already reserve for "refusing to report clean", as
distinct from 1 = "the ratchet moved". Site counts are unchanged.
The query-options `--self-test` (run by CI ahead of the gate) proves the guard
in both directions over real ESLint output, and asserts from source that both
gates still route through it — `pnpm check:slot-lookup` has no self-test hook
of its own, so that assertion is the wired coverage of its call site.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
@os-steveos-steve added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 20, 2026 — with Claude
@claude

claudeBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

PM review — domain:devx, round 29. ACCEPT. Verdict below; arming once the six required contexts return green.

Clause-② gate. Declared Clause-②: no against the actual diff, and I re-derived both limbs independently rather than taking the declaration:

  • Path limb — clear. The diff is three files, all under root scripts/ (check-query-options-erasure-ratchet.mjs +90/-4, check-slot-lookup-ratchet.mjs +18/-2, new eslint-fatal-guard.mjs +214/-0). Nothing under packages/spec/src/**.
  • Content limb — clear. No contract's accept/reject behaviour moves and no public surface widens. The two gates' own accept set narrows — a file that scored clean by being unmeasurable now fails — which is the direction the card asks for. ⛔ Recording explicitly that the path limb being clear is not the coverage: the content limb was judged on its own.

Governed surfaces re-derived from scripts/pm/check-governed-merges.mjs at this round (not recalled): docs/adr/, .claude/, skills/, exact AGENTS.md, CLAUDE.md. Zero hits — this is a normal-path PR.

Ruling on the declared file-surface deviation. The dispatch named two scripts plus a fixture; the dev shipped a third file, the shared scripts/eslint-fatal-guard.mjs, and flagged it rather than taking it silently. Accepted, and the argument is the right one: two copies of a guard drift, and a drifted copy is invisible here — the gate that lost the check keeps printing the same green line it printed before. That failure mode is the exact one this card exists to close, so duplicating the guard would have reopened it one level up. The dev also checked the helper against check:ratchet-remedy-authority (98 scripts swept; it classifies with the 89 handing out no ratchet-expanding remedy) rather than assuming the sweep tolerated a new file.

Evidence I am accepting on:

  • Harm reproduced deliberately, both scripts, both halves — before: EXIT=0 with ✓ … holds while an unparseable file sat in the population; after: EXIT=2 naming file:line:col and the parser's message. The root lint asymmetry the card names confirmed directly in the same run (ROOTLINT_EXIT=1).
  • Site counts identical across all three refs (107/25 slot-lookup · 67/17 query-options non-test · 240/47 test surface) — the card's own acceptance criterion, met.
  • Two ablations, and ablation B's observed direction was not the predicted one: neutering the m.fatal predicate still caught the file via the fatalErrorCount cross-check, losing only position and parser text. Reported as measured instead of as expected — that second limb is why a future ESLint moving the flag cannot make this silent again.
  • Exit 2 reuses the code both gates already reserve for "refusing to report clean", kept distinct from 1 = "the ratchet moved". The --update paths are covered too (measurement precedes the write in both).

Carried forward, not blocking:

  1. Root package.json untouched — deliberate, since fix(lint): give the eslint parser stack headroom for migrations/registry.ts #10124 and fix(pm): complete pm label vocabulary (pm:seat, priority:p0) and add desc-cap guard #10116 were both in flight there. Consequence: check:slot-lookup gets no --self-test hook of its own, and the wired coverage of its call site is the source assertion inside the query-options self-test (checkGuardAdoption()). A second wired self-test is a package.json edit and a follow-up, not a gap in this PR.
  2. A real consequence worth stating plainly: where an unparseable file used to be a silent skip, it is now a red gate. packages/spec/src/migrations/registry.ts runs at roughly a 10% stack margin (fix(lint): give the eslint parser stack headroom for migrations/registry.ts #10124's subject). These green runs are positive evidence it parses inside these gates' wide packages/** run at the default 984 KB stack today — and the deferred headroom card now inherits a louder failure mode. That is the trade the card asked for: a named red instead of a quiet green.
  3. Out-of-scope finding filed unassigned as [finding] The 15 gates that parse with ts.createSourceFile never read parseDiagnostics — a file with a syntax error is walked as a recovered partial AST and scores clean #10133 — the 15 gates parsing via ts.createSourceFile never read parseDiagnostics, the same defect class one parser over. Measured, not asserted: an unterminated template literal takes a 3-site file to 0 sites with parseDiagnostics=1 and the gate scores it clean. Correctly not fixed here — the fix needs a decision across 15 call sites (parseDiagnostics is not on the public SourceFile type), so the in-place-exemption condition fails.

Correction to my own dispatch, on the record. I gave origin/main as e61ee68320; by the time the worktree was cut it had moved to 5262124388, and the dev based there and said so. Every count I quoted reproduced exactly at that base, so the card's numbers held — but the branch point in my dispatch was stale, and that is the second time this session my quoted head has aged out between writing a dispatch and the dev acting on it.


Generated by Claude Code

@os-elon
os-elon marked this pull request as ready for review August 20, 2026 08:16
@os-elon
os-elon added this pull request to the merge queueAug 20, 2026
Merged via the queue into main with commit a659896Aug 20, 2026
26 checks passed
@os-elon
os-elon deleted the claude/issue-10123-ratchet-fatal-parse branch August 20, 2026 08:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The two ESLint ratchet gates swallow a fatal parse error as a non-matching message — an unparseable file scores clean instead of failing

3 participants

@os-steve@os-elon@claude