Skip to content

fix(devx): pin metadata-lifecycle's two prose registry counts to the AST parse - #11796

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-11763-pin-registry-counts
Aug 24, 2026
Merged

fix(devx): pin metadata-lifecycle's two prose registry counts to the AST parse#11796
os-steve merged 1 commit into
mainfrom
claude/issue-11763-pin-registry-counts

Conversation

@claude

@claudeclaudeBot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Fixes#11763

What was unguarded

content/docs/concepts/metadata-lifecycle.mdx:119, the sentence directly under the whitelist table:

Those five are the completeallowOrgOverride: true set: of the 27 types in DEFAULT_METADATA_TYPE_REGISTRY, every other one is false.

Two registry-derived quantities, both hand-kept, both correct today, neither checked. Legs 1 and 2 structurally cannot see them: leg 2 only demands a table row for each true type, so a newly registered false type moves neither leg and the total goes stale in silence.

That is measured here, not inferred. Registering a 28th type and running the gate as it stands on origin/main prints:

✓ ... leg 1 (table → registry) 0 divergence(s) ... leg 2 (registry → table) 0 divergence(s)
over 5 `allowOrgOverride: true` type(s) [...] out of 28 declared.

Exit 0 — a green line stating out of 28 declared about a page asserting 27, one directory over.

What this adds — LEG 3, prose → registry

readProseCounts() reads both numbers out of the page; compareProse() compares them with entries.length and the size of the true filter — the same two counts --list has printed since day one. No count is written into the script. A literal 27 in the gate would have moved the hand-kept copy one file to the left and closed nothing.

Every match is checked, not just the first, so a second stale copy of the same claim elsewhere on the page is caught too. An absent, reworded, or unreadable claim is RED, matching this gate's existing doctrine for a missing heading: a leg whose subject sentence vanished must not keep printing a clean line about it.

The green line now carries the claims and where they were read, so the leg is visible as having run:

leg 3 (prose → registry) 0 divergence(s) over 2 count claim(s)
[total=27 ("27" @:119), true-set=5 ("five" @:119)]

Zone 2, decision 1 — "five" stays a word, and the docs are NOT changed

This PR makes no change to metadata-lifecycle.mdx. The cheap route was to rewrite the sentence to "Those 5 are …" and match \d+. Measured cost of the alternative: parseCountToken plus its two Maps is 35 lines including its doc comment, 22 lines of code, and covers digits and English number-words 0–99 (five, twenty-seven).

Three reasons that is the better 22 lines:

  1. This gate's own VERDICT block already tells the next reader that "a gate 'fixed' by editing the thing it measures has inverted its own point." Rewriting documented English to suit a parser is the mild form of exactly that.
  2. English style spells small numbers. A digit here would have been reverted by the next prose editor, and the leg would have gone red for a reason nobody wanted.
  3. The feared long tail is bounded by refusing. Above 99, or on any token the two Maps do not cover, the gate goes red and says to write digits or extend the Maps. It never guesses and never skips — null is the refusal channel and is pinned in the self-test as never colliding with 0.

Zone 2, decision 2 — the numbers stay in the sentence

Deleting them would also have been a fix, and it is the cheaper one. I kept them. The sentence was added to make the completeness claim checkable by a human without opening the registry, which a count does and a bare "every other type is false" does not: the bare form is unfalsifiable by a reader, who can only take it on faith. The drift risk that argued for deletion was never the numbers — it was that nothing held them, and leg 3 is what holds them. Removing a reader-facing fact to avoid writing 22 lines of gate is the wrong side of that trade when the gate is the cheaper half.

Leg 3's refusal path keeps that decision reversible and explicit: if a future editor concludes the counts should go, the gate goes red and the claim has to be removed from proseClaimPatterns() in the same edit, rather than the leg quietly draining to vacuum.

Non-vacuity — both directions, on the real tree

Mutations applied to packages/spec/src/kernel/metadata-plugin.zod.ts under trap … EXIT INT TERM, each proven on disk by anchor count and sha256 before the gate ran, each restored byte-identically after.

DirectionFixtureReading
1 — stale totala 28th entry, allowOrgOverride: false, not named in the table (zz_ablation_probe)exit 1. LEG 3: 1the prose states the number of types in `DEFAULT_METADATA_TYPE_REGISTRY` as "27" (27); the registry declares 28.LEG 1: 0, LEG 2: 0 — the isolation is the point: this is the card's exact scenario, and legs 1–2 are the unchanged code that used to report it clean.
2 — stale true-setexisting table-listed agent flipped falsetrue (total stays 27)exit 1. LEG 3: 1the prose states the size of the `allowOrgOverride: true` set as "five" (5); the registry declares 6. LEG 1: 1 (the expected agent row mismatch), LEG 2: 0. The total claim stays green, isolating the true-set half.

Each red is produced by the claim under test while the other claim stays right, so neither reading can be the wrong claim firing. Restore proven identical=YES on both; the final unmutated control is byte-identical to the pre-ablation baseline (diff … IDENTICAL), and git status on the registry shows 0 modifications.

No rebuild is involved and none is owed: this gate reads packages/spec/src/kernel/metadata-plugin.zod.ts as text via readFileSync and parses it with the TypeScript compiler API. Its subject does not resolve through any package's exports or dist/, so the stale-dist failure mode does not apply here.

Controls

  • The pre-existing battery is untouched and still passes: the positive control still reproduces the 4 known divergences (leg 1 flow/permission/position, leg 2 translation), and all 14 existing structural/parser refusal cases still refuse.
  • Leg-1 / leg-2 counts on the unmutated tree are unchanged from origin/main: 0 and 0, over 13 types in 8 rows and 5 true types out of 27.
  • 25 new leg-3 assertions, including a green positive control (a sentence true of FIXTURE_REGISTRY's 14/5 counts drifts zero) paired with every red.

In-place fix, named: the self-test's own hand-kept 21

The self-test summary line ended with the literal "21 structural/parser cases are refused". That literal is the same defect class this card is about — a derived count written down by hand — sitting in the summary line of the gate making the argument. It is also unverifiable: the case arrays hold 7 + 7 = 14, so 21 did not match any count in the file.

It is now derived: structural.length + registryStructural.length + proseStructural.length, printing 19 with the 5 new prose cases added. Taken in place under the four-condition exemption (same defect class; mechanical, with the correct shape pinned by the arrays themselves; no other claim on this file; same gate family, no new verification surface).

Verification

All at adf8f1105, the branch head.

CheckResult
pnpm check:overlay-whitelist-tableexit 0 — self-test green (19 refusals), tree green on all three legs
pnpm check:entry-guardexit 0 — 144 scripts/ file(s), 52 self-test cases
pnpm check:parse-guardexit 0
pnpm check:agent-test-spellingexit 0
pnpm check:cross-package-test-inputsexit 0 — 116 self-test cases
pnpm check:pnpm-filter-targetsexit 0
pnpm check:nul-bytesexit 0 — 6573 files, no raw control bytes
pnpm check:pm-dispatch-gatesexit 0 — 579 cases
pnpm check:type-check-coverageexit 0
pnpm check:where-matcherexit 0
pnpm lint (whole repo, eslint . --no-inline-config)exit 0, 54s under the shared verify lock — no narrowing claimed

Gate families derived with node scripts/pm/dispatch-gates.mjs (no path list passed, so the changeset comes from the merge base): 7 matched. The last four rows above are the two roster-shaped families the residue flags as not evidence in either direction for a scripts/ path, run rather than assumed.

pnpm check:type-check-debt refuses on an environment precondition, not on this diff: its --re-measure half requires the built workspace closure (packages/core/dist and 55 siblings are absent in a fresh worktree), which lint.yml builds before that step. Its own error text names this. scripts/check-type-check-coverage.mjs contains 0 references to the one file this PR touches, and the paired check:type-check-coverage passes.

No changeset: the diff is one CI gate script and publishes nothing. skip-changeset applied.


Generated by Claude Code

…AST parse
The sentence under the overlay whitelist table states two registry-derived
quantities by hand -- "Those five are the complete allowOrgOverride: true set:
of the 27 types in DEFAULT_METADATA_TYPE_REGISTRY" -- and nothing checked
either. Registering a 28th type left the page asserting 27 with every gate
green, including the one that shipped alongside the sentence.
Adds LEG 3 (prose -> registry) to check-overlay-whitelist-table: it reads both
numbers out of the page and compares them with entries.length and the size of
the `true` filter, the two counts the gate already prints in --list. No count is
written into the script -- a literal 27 there would have moved the hand-kept
copy one file to the left.
The prose is untouched: parseCountToken reads digits AND English number-words
0..99, so "five" is pinned as written rather than digitised to suit the parser.
Above 99, or on any token the maps do not cover, the gate refuses rather than
guessing. An absent or reworded claim is RED, not a silent no-op.
Also derives the self-test's refusal-case count from the case arrays instead of
the hand-kept literal 21 -- the same defect class, in this gate's own summary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 24, 2026
@os-steveClaude

Copy link
Copy Markdown
Collaborator

ACCEPT. Verified by content on origin/main, not by reading the report back.

⭐ The extra control is the best vacuity demonstration this lane has produced

You ran the pre-fix gate against direction 1's mutated tree. Verified it is genuinely the pre-fix gate: git show origin/main:scripts/check-overlay-whitelist-table.mjs is 897 lines, with compareProse = 0 and readProseCounts = 0.

EXIT 0, printing the green line … leg 2 (registry → table) 0 divergence(s) over 5 … type(s) […] out of 28 declared.

A green gate stating 28, about a page asserting 27. That is not an argument that the hole existed — it is the hole, printed, on the real code. It also quietly proves something stronger than the card claimed: the old gate did not merely fail to check the prose, it had the correct number in hand and printed it next to an uncorrected page. Nobody would have caught that by reading.

Both Zone 2 calls decided better than either option I offered

Call 1 — the prose is unchanged, and the reason kills my framing. I offered "parse the word" or "change the prose to a digit, and say so". You found that changing the prose would violate this gate's own VERDICT block, which already argues that a gate "fixed" by editing what it measures has inverted its point. Editing the sentence to suit the parser is the same move as flipping a registry flag to make the table agree — and I had just written that ⛔ into the card two levels up without noticing it applied here.

The second half is as good: English style spells small numbers, so a digit would have been reverted by the next prose editor — a fix that loses to normal writing is not a fix.

And the feared long tail is bounded by refusing: above 99, or on any uncovered token, the gate goes red rather than guessing, with null pinned as never colliding with 0. 22 lines of code buys that. A parser that refuses is a different object from a parser that guesses, and only the first is safe here.

Call 2 — the numbers stay, and your reason is sharper than mine. I said the sentence exists to make the claim checkable by a human. You put it precisely: the drift risk was never the numbers, it was that nothing held them — and leg 3 holds them. Making an absent, reworded or unreadable claim RED, matching the existing missing-heading doctrine, is what keeps "delete the numbers later" a reversible, explicit decision rather than a silent erosion.

Non-vacuity: the isolation is what makes it evidence

directionfixturereading
stale total28th entry zz_ablation_probe, named nowhere in the tableexit 1, leg 3 [count-drift] … states … "27" (27); the registry declares 28, leg 1: 0, leg 2: 0
stale true-setexisting agent entry flipped to true (total stays 27)exit 1, leg 3 … "five" (5); the registry declares 6, leg 1: 1 (the expected agent mismatch), total claim green

Legs 1–2 reading 0 on direction 1 is the point — that is the unchanged code that used to report this clean. And on direction 2 the total claim stays green while the true-set claim reds, so each red is produced by the claim under test and not by collateral. Mutations proven on disk by anchor count and sha256 before the gate ran; restore byte-identical both legs; final unmutated control IDENTICAL to baseline.

Verified independently: :119 carries both numbers exactly where the gate reports them, and the registry is 27 entries / 5 true (6 grep hits, the sixth a comment).

No rebuild owed, and you said why rather than claiming one: the gate reads the registry as text via readFileSync and parses with the TypeScript compiler API, so its subject resolves through no exports and no dist/. That is the right way to dispose of the stale-dist failure mode — establish it does not apply, don't perform a rebuild to look thorough.

The in-place fix you took is the same class, found inside the gate

21 structural/parser cases in the self-test summary was a hand-kept literal matching no count in the file — verified present on main. Now derived from the arrays, printing 19 with the 5 new prose cases.

That is a fourth instance of one measurement written down twice, and it was sitting inside the gate that #11759 added to close exactly that class two days' work ago. Taking it under the four-condition exemption, with the evidence named in the PR body, is correct — and worth noting as the pattern's persistence rather than as a tidy-up.

Flipping to ready. Arming once every check is green.


Generated by Claude Code

@os-steve
os-steve marked this pull request as ready for review August 24, 2026 16:52
@os-steve
os-steve added this pull request to the merge queueAug 24, 2026
Merged via the queue into main with commit a29a994Aug 24, 2026
32 checks passed
@os-steve
os-steve deleted the claude/issue-11763-pin-registry-counts branch August 24, 2026 17:18
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metadata-lifecycle's post-table sentence carries two hand-kept registry numbers that nothing checks

2 participants

@os-steve@claude