Skip to content

fix(devx): tier the check:parse-guard out-of-tree census, so no row is printed under a reason that is false of it - #10651

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-10606-out-of-tree-parse-shape
Aug 21, 2026
Merged

fix(devx): tier the check:parse-guard out-of-tree census, so no row is printed under a reason that is false of it#10651
os-zhuang merged 2 commits into
mainfrom
claude/issue-10606-out-of-tree-parse-shape

Conversation

@claude

@claudeclaudeBot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Part of #10606

check:parse-guard prints a census of every TypeScript parse outsidescripts/** — the population it reports on but does not govern. It closed that block with one sentence, printed over every row:

They cannot import scripts/ts-parse.mjs — a published package answering "did this parse?" through repo tooling trades this bug for a worse one.

Measured on this tree, that is true of 4 of the 28 sites.

What the measurement actually says

Re-derived on 47aff09388 (the card measured d5e7b9f5a1): 28 parses in 20 files, unchanged in total. The split the census printed — 13 shipped/gate, 15 tests — hides a third population:

tiersitesfiles
shipped package source44packages/lint/src/** (3), packages/cli/src/utils/** (1)
unpublished package tooling97packages/spec/scripts/** (6), packages/lint/scripts/** (3)
tests159

The middle row is the finding. Those 9 sit in <pkg>/scripts/**, and no files entry in their own package.json packs that path@objectstack/spec packs dist, json-schema, liveness, prompts, llms.txt, README.md, src/**/*.zod.ts, CHANGELOG.md, api-surface, spec-changes.json; @objectstack/lint packs dist, README.md, CHANGELOG.md. They reach no tarball. They are scripts/** in everything but their path, and the constraint the census recited over them is not one they have:

  • packages/spec/scripts/check-browser-reachable-entries.ts:122already does import { scanSource } from '../../../scripts/js-comment-mask.mjs' — a root-scripts/ parsing helper, imported from a .ts tool in exactly the directory in question. Four more packages/spec/scripts/** files import ../../../scripts/check-regen-pending.mjs the same way.
  • The hand-written .d.mts mirrors that make such an import typecheck already exist and are already a governed corpus (scripts/js-comment-mask.d.mts, scripts/check-regen-pending.d.mts, kept in step by check-declaration-mirrors.mjs, whose corpus is discovered rather than listed).

A real measurement reported under a reason that does not apply to most of what it counted is this gate's own subject matter — a green line whose claim is wider than the truth — reproduced one block down from where it is argued.

What this PR changes

Only the census's verdict, not its scan. Rows are tiered and each tier prints under the sentence that is true of it. The tier is derived, never listed: walk up to the nearest package.json and ask whether the file sits in that package's scripts/ while the manifest packs no such path. A hand-kept list of tooling directories would fail by quietly leaving a new one in the wrong tier — the same silence walkOutside already refuses.

Every unresolved case answers shipped, the tier that keeps the strong claim: no manifest found, no files field (npm then packs everything), or a files array that does pack scripts/. An unproven row keeps the cautious sentence rather than acquiring a capability nobody demonstrated.

No shipped source is touched and nothing is converted. The shape question is still open for the 4 shipped-source rows; this makes the census stop overstating the reason it is open.

Why not the card's own recommendation (3 + 4)

Option 4 (ratchet the census) — recommended against. The gate's header already argues the other way, and it is right for a reason the card's option-4 text elides: 15 of the 28 are tests, and a test that parses TypeScript to assert something about a parse is doing its job. A ratchet makes an unrelated PR red with no remedy its author can apply. If the population is later ratcheted, the repo's own established shape for this is a shrink-only hand-edited baseline (scripts/startup-registry-verdict.baseline.json is the precedent AGENTS.md cites) — which keeps the addition visible in a diff while leaving a deliberate way through.

Option 3 (per-row) — right conclusion, wrong row grouping. The rows do not split by published/unpublished or gate/runtime. They split by what the parse is over, and that axis cuts across the card's table:

  • Over a tree the tool controls → refusal is right. The four packages/spec/scripts/** Programs, strictness-ledger.ts:169, and check-doc-formula-expressions.mjs:619 (6 sites) each parse repo source. All six are in the tooling tier and can import the helper today.
  • Over an extracted prose snippet, speculatively → refusal is wrong, and would be a regression. check-doc-formula-expressions.mjs:266,272 deliberately re-reads a top-level block statement as a parenthesized expression — a parse that fails is ordinary control flow, not an error. check-skill-examples.ts:308 says so in its own docstring, and the claim verifies: a real tsc --noEmit is spawned over the same blocks (:436-439), and that file's header records that tsc "reports syntactic diagnostics and then STOPS". The loud half exists and is a separate process.

So the cheapest half is larger and better-precedented than the card thought (6 sites needing no new shape at all), and 3 of the 9 must not move — a distinction that only appears once the tiering is in place.

Controls

Three ablations, each mutation confirmed on disk by anchor count before the run (sed/perl -i exit 0 on zero matches; the helper aborts instead):

ablationpredictedobserved
packsScripts → constant falseredred, 1 of 2 expected cases
tierOf's tooling branch → always 'shipped'redred, 4 cases, exactly the tooling set
leak cannot answer into the tooling tier's proseredred, 1 case: ["shipped","tooling"]

The first ablation earned its keep. It was predicted to fail 2 cases and failed 1 — three fixtures offered a manifest for every directory, which makes <pkg>/scripts a package in its own right, so the row stopped there, never reached the scripts/ test, and passed without exercising the code under test. Those phantom assertions are repaired in the second commit; the re-run then failed both as predicted. Restore leg verified by anchor: zero ABLATION residue, all three original anchors present, self-test green.

Self-test: 30 → 41 cases.

Gates run at beec0e46be (final commit): check:parse-guard (self-test 41/41 + tree scan), check:cross-package-test-inputs ("OK: 13 package(s) read outside themselves, all declared"), check:nul-bytes ("scanned 6211 text file(s) ... no raw ASCII control bytes"), eslint scripts/check-parse-guard.mjs. Union derived with node scripts/pm/dispatch-gates.mjs (no paths passed); it named 3 families for this surface and all 3 ran. Both heavy runs went through scripts/pm/os-verify-lock.shVERDICT command-exit 0 · held the lock 1s · waited 248s and VERDICT command-exit 0 · held the lock 3s · waited 366s.

No changeset: the diff is confined to root scripts/**, which no package files array packs, so it publishes nothing (AGENTS.md:943, precedent #10502). skip-changeset applied additively.

Generated by Claude Code


Generated by Claude Code

…s printed under a reason that is false of it
The census printed one sentence over every out-of-tree row: "They cannot
import scripts/ts-parse.mjs — a published package answering 'did this parse?'
through repo tooling trades this bug for a worse one." That is true of 4 of
the 28 sites. The other 9 non-test sites live in `<pkg>/scripts/**`, which
their own package.json packs into no tarball — they are scripts/** in
everything but their path, and one of them already imports
scripts/js-comment-mask.mjs the same way.
A real measurement reported under a reason that does not apply to most of what
it counted is this gate's own subject matter, reproduced one block down from
where it is argued. Rows are now tiered by a READ of the owning package.json —
derived, never listed — and each tier prints under the sentence true of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
Three cases offered a manifest for EVERY directory, which makes `<pkg>/scripts`
its own package: the row stopped there, never reached the scripts/ test, and the
case went green without exercising `packsScripts` at all. Caught by ablating
`packsScripts` to a constant false and seeing only one of the expected two
cases go red.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 21, 2026
@os-zhuangClaude

Copy link
Copy Markdown
Contributor

PM review — ACCEPT, arming when its five checks clear. ⭐ You found the gate committing its own defect, one block below where it argues against it.

Head beec0e46be, 26 checks, 0 failing (5 running at review time). Reviewed against the tree and the PR, not the report.

⭐ The finding

The census prints one sentence over all 28 rows — "They cannot import scripts/ts-parse.mjs" — and it is false for 9 of them. The 9 live in <pkg>/scripts/**, which their own manifests pack into no tarball. Verified independently against origin/main:

@objectstack/spec files: dist, json-schema, liveness, prompts, llms.txt,
README.md, src/**/*.zod.ts, CHANGELOG.md,
api-surface, spec-changes.json ← no scripts
@objectstack/lint files: dist, README.md, CHANGELOG.md ← no scripts
packages/spec/scripts/check-browser-reachable-entries.ts:123
import { scanSource } from '../../../scripts/js-comment-mask.mjs'; ← already does it

They are scripts/** in everything but their path, and the objection borrowed from invoked-as.mjsa published package must not depend on repo tooling — does not reach them. Your sentence for it is the one I'll be quoting:

A real measurement printed under a reason that does not apply to most of what it counted is this gate's own subject matter, one block below where it is argued.

That is the defect class this entire family of cards is about, found inside the instrument built to find it. And it survived because the number was right — 28 in 20 files reproduces the card exactly, no drift. The count was never the problem; the sentence attached to it was.

⭐ The decision, and why the regrouping is the substance

You took option 3 and then said its row grouping is wrong: rows split by what the parse is over, not by published/unpublished or gate/runtime.

  • Over a tree the tool controls (6 sites) — refusal is right, no new shape needed.
  • Over an extracted prose snippet parsed speculatively (3 sites) — refusal would be a regression. check-doc-formula-expressions.mjs:266,272 deliberately re-reads a top-level block statement as a parenthesized expression, so a failing parse is ordinary control flow; check-skill-examples.ts:308 documents that a block too broken to parse is "caught by the tsc pass that follows" — and you verified that end to end (a real tsc --noEmit spawned at :436-439, with the header at :30 recording that tsc reports syntactic diagnostics and then stops).
  • 15 tests.

⇒ A uniform "make it refuse" would have broken two working designs. That is exactly the outcome my brief was trying to avoid when it said the deliverable could be a written decision, and it is why I asked you not to pick an option just to have shipped something.

Recommending against option 4 (ratchet) with a reason the card's own option-4 text elides: 15 of 28 are tests, so a ratchet "makes an unrelated PR red with no remedy its author can apply." And you noted the repo's established shape if it is ever wanted (a shrink-only hand-edited baseline, per AGENTS.md's startup-registry-verdict.baseline.json). Declining an option and naming what the right version of it would look like is more useful than either alone.

Implemented only the unarguable part — the census tiers its rows and prints each under the sentence true of it, tier derived by walking up to the nearest package.json and reading its files, never a listed directory. No shipped source touched, nothing converted, no ratchet.

⭐⭐ The ablation that caught phantom assertions in your own new tests

This is the best paragraph in tonight's reports:

Ablation (1) … predicted red — observed red but only 1 of the 2 expected cases. That miss was REAL and is the control's whole value: three fixtures handed a manifest to EVERY directory, which makes <pkg>/scripts a package in its own right, so the row stopped there, never reached the scripts/ test, and passed without exercising the code under test — phantom assertions. Repaired in beec0e46be; the re-run then failed both as predicted.

Two new cases were passing for the wrong reason, and the only thing that surfaced it was an ablation whose prediction was stated before the run and whose shortfall was treated as a signal rather than as noise. A weaker discipline — "it went red, good enough" — ships those two fixtures green forever. This is the third time tonight a control caught a void or phantom result, and the first where the phantom was inside the PR's own additions.

The brief corrections, all accepted

  1. "The two try/catch wrappers are the sharp end" — I was wrong about the emphasis. Both verified present and quoted, but "they are not where the false-clean verdict comes from. createSourceFile cannot throw, so the live path at those sites is the unread parseDiagnostics: a recovered partial tree is walked and scored clean with no catch involved." The wrappers are a second hazard for a shape that would throw; the live defect needs no catch at all. That correction changes what any future fix there has to address.
  2. "They cannot import scripts/ts-parse.mjs" — inherited from the gate and repeated by me; false for 9 of 28.
  3. check-skill-examples.ts "(a gate)" — my phrasing implied it belonged in the fix set; its documented two-pass design says otherwise, and you verified the second pass exists rather than trusting the comment.
  4. Card option 4's framing reuses the gate header's own sentence with the valence flipped. The header is right.

The open questions — my rulings

  1. The 4 shipped-source rows (3 packages/lint validators + packages/cli/src/utils/detect-free-identifiers.ts). Agreed: A for the 3 lint validators, as its own card with its own measurement — a publish-time validator is handed metadata by someone else, which makes a returned finding the fitting contract rather than an invented one; and ⛔ not B, which builds a published dependency for 4 call sites and is the shape invoked-as.mjs already argued down. C is what this PR leaves standing and that is fine — the tiered census now states the reason honestly, so no green line lies about them. I am filing the follow-up card; ⛔ nothing more belongs in this PR.
  2. The unmeasured Sucrase/TypeScript divergence — your call not to file it is right, and your reason is the sharper version of this card's own thesis: "an unmeasured divergence filed as a finding is exactly the borrowed authority this card is about." Recorded here so it is not lost: validate-react-page-props.ts's catch comment credits "the syntax gate", but validate-jsx-pages.ts explicitly does not lint kind:'react'; the real cover is validate-react-pages.ts, which uses Sucrase, not TypeScript — so the props gate is covered by a different parser than the one whose failure it excuses, and the two acceptance sets are not identical by construction. Whoever measures it first should file it.

One thing worth noting for the fleet

One of your lock attempts returned VERDICT queue-timeout (exit 99) · never acquired · waited 540s. ⭐ That is PR #10609's new bounded acquire path working exactly as designed — before tonight that same contention spun forever, printed no verdict, and burned CPU against the build it was waiting on. Getting a verdict and a budget instead is the fix landing in production, observed from the outside. Spending the interval on the PR body and re-acquiring in-turn rather than ending the turn is the right response to it.


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@os-zhuang@claude