Skip to content

fix(docs-audit): a PARTIAL ledger read is a verdict, not a smaller number - #10501

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-9896-docs-audit-partial-ledger-parse
Aug 21, 2026
Merged

fix(docs-audit): a PARTIAL ledger read is a verdict, not a smaller number#10501
os-zhuang merged 1 commit into
mainfrom
claude/issue-9896-docs-audit-partial-ledger-parse

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes#9896

The defect

parseLedgerSource in scripts/docs-audit/affected-docs.mjs recognizes a route-ledger row through a single-quote-only regex. The pre-existing broken-scan guard is rowsParsed === 0, so it catches only the all-or-nothing case. A ledger where some rows are spelled in a quote the recognizer declines parsed fine and reported a smaller population with no verdict and exit 0 — a partial result that renders exactly like a complete one.

1. The spelling census, with its positive control

Every route ledger on a718ee3dd, route: / client: declarations by quote spelling vs. what the recognizer read:

ledgerroute: declared'"`parsedΔ
plugins/plugin-auth/src/auth-route-ledger.ts565600560
rest/src/rest-route-ledger.ts969600960
runtime/src/route-ledger.ts797900790
services/service-datasource/.../datasource-route-ledger.ts111100110
services/service-i18n/.../i18n-route-ledger.ts330030
services/service-settings/.../settings-route-ledger.ts440040
services/service-storage/.../storage-route-ledger.ts101000100
total259259002590

client:: 221 declared, 221 bound, Δ 0 — the same 221 the bridge reports.

⛔ That is a zero-hit, so it is claimed only with a positive control. Two of them:

  • On the real tree. Backtick-quoting exactly one row of i18n-route-ledger.ts moves the census to 258 of 259 and the gate to exit 1 — the counter demonstrably fires (§4).
  • In --self-test. The all-single-quoted fixture is pinned to declined = 0and to routesDeclared === rows.length; the partial fixture beside it is pinned to declined = 3. A zero from a counter that cannot fire is not evidence, so the pair is written as a pair.

The census also finds the delta is not only about route:. Three distinct drops, all measured, all exit 0 before this PR:

ledger written asrowsParsedclient-bound rowsverdictexit
all single-quoted (today)3221none0
one backtick-quoted route:2220none0
one backtick-quoted client: (route left single-quoted)3220none0
one backtick-quoted route: in rest-route-ledger.ts95221⚠️none0

Row 3 is the shape the rowsParsed === 0 guard cannot see by construction: the row keeps its seat and loses its binding.

Row 4 is worse, and it is the one that decided the design. The row window is delimited by the same single-quote-only lead, so a declined row does not close the previous row's window — the row before it inherits the declined row's client. Backtick-quoting GET /api/v1/meta moved meta.getTypes onto the server-only GET /api/v1/docs, and clientRows stayed 221. A count comparison is blind to it.

2. What "loud" is here, and why

A brokenScan verdict — exit non-zero — not a warning and not a new verdict word. Argued from what consumes the output, which is three things:

  1. scripts/docs-audit/check-affected-docs.mjs, run as a step in .github/workflows/docs-drift-check.yml. Its exit code is the verdict; a warning here is a line in a job log that nothing reads.
  2. The advisory run's summary line, which already renders brokenScan as ⛔ N broken-scan verdict(s): ….
  3. The drift PR comment, which already renders bridge.brokenScan outside its collapsed what this run could not see section.

All three paths already existed and already carried brokenScan, so joining that array is the change that reaches every consumer at once — no new field to half-wire (#9433's failure mode).

Verdict and not report on the test bridgeCoverageFrom already applies to itself: a verdict "cannot fire on a tree where the scan works at all". Today's tree is 259 of 259, Δ 0 — this cannot fire. It is a break in the scan, not a property of the route surface, which is exactly what separates it from the 45-of-221 reach ratio: that ratio is a fact about the repo and stays reported, because failing on it would be widening-by-CI, which the #9747 family declines. Respelling a ledger row is neither — it is cheap to fix in either direction (respell the row, or widen the recognizer with the before/after numbers #9432 asks for).

Keyed on the declined spelling, not on the shortfall, because of row 4 above: the shortfall is zero in the case that produces a wrong binding. The numerator is reported beside it, not instead of it.

Concretely:

  • parseLedgerSource returns { rows, declined, routesDeclared, clientsDeclared }. rows comes out of the loop it always did, byte for byte — the recognizer is not widened, and --self-test pins the population at its old value so a silent widening fails there (verified by ablation, §4).

  • bridgeCoverageFrom raises one verdict per ledger that declined anything, naming file, line and literal, capped at three named entries plus a count so a wholly respelled 96-row ledger cannot flood a job log:

    ✗ broken scan: packages/services/service-i18n/src/i18n-route-ledger.ts is a PARTIAL read of that
    ledger, not its shape — the row recognizer reads single-quoted values only and read 2 of 3
    declared string-literal `route:` value(s) and 2 of 2 declared string-literal `client:` value(s);
    declined 1: line 96: route: `GET /api/v1/i18n/locales`
    
  • Both halves of the fraction print on every run, so the denominator no longer has to be known by heart:

     ledger rows read ........... 259 of 259 declared
    client-bound ledger rows ... 221 of 221 declared
    
  • .github/workflows/docs-drift-check.yml: the broken-scan banner used to promise the scan "came back structurally empty". That wording would let a reader skip exactly the new case, so it now reads "is broken, not clean".

3. Blast radius — what else reads this ledger

Measured, not assumed.

  • Regex readers of ledger source text in the repo: exactly one.grep -rn "route\s*:" --include='*.mjs' --include='*.mts' --include='*.ts' over the tree returns scripts/docs-audit/affected-docs.mjs and nothing else.
  • Value readers are unaffected. 30+ files import the ledgers (*.conformance.test.ts, packages/client/src/*-route-ledger-coverage.test.ts, …); none reads them as text (grep readFileSync over that set: zero ledger reads). They see real TypeScript strings, so a quote respelling is invisible to them by construction.
  • So the radius is the one file's two output paths — the --bridge-coverage gate, and the advisory run's bridgeCoverage rendered in the drift PR comment. Both now carry the verdict.
  • What a silently dropped row costs on the advisory path. Of the 45 reachable client-bound rows, 6 name a client method that at least one hand-written page carries; those 6 carry 7 (row, page) nominations across 4 distinct pagesapi/client-sdk.mdx, kernel/runtime-services/data-service.mdx, and two release-owned pages. Dropping one of those 6 removes a real doc nomination, including from the page the bridge exists to put back on the list. Dropping any of the other 215 removes no nomination today but silently shrinks the denominator both the gate and the PR comment report — the "smaller number reported as complete" this card is about.

One boundary this does not close, filed rather than absorbed as #10500: a route: whose value is not a string literal at all (route: ROUTES.health) is invisible to the recognizer and to the counter. The only exact discriminator against the route: string; member every ledger's own entry interface declares is the opening quote — measured: 8 non-quoted route: occurrences across the seven ledgers, 7 of them that interface member and 1 an English sentence in a comment — so any heuristic separating them is a guess that would red on accurate files today. The boundary is pinned in --self-test and stated in the README instead.

4. Ablations

Every mutation was confirmed on disk by grepping for both the injected and the removed text — never by an editor's exit code — and every restore was confirmed byte-clean with git diff --quiet. No rebuild is involved: affected-docs.mjs is executed directly by node, with no dist/ between the edit and the run.

A. Reintroduce the dropping spelling (real tree). Backtick-quote one route: in i18n-route-ledger.ts — on-disk: injected spelling ×1, removed spelling ×0:

EXIT=1
ledger rows read ........... 258 of 259 declared
✗ broken scan: …i18n-route-ledger.ts is a PARTIAL read … read 2 of 3 declared string-literal
`route:` value(s) … declined 1: line 96: route: `GET /api/v1/i18n/locales`

Before this PR the same input printed 220, no verdict, exit 0.

B. The client: case. Backtick-quote one client:, leave its route: single-quoted:

EXIT=1
client-bound ledger rows ... 220 of 221 declared
✗ broken scan: … read 3 of 3 declared string-literal `route:` value(s) and 2 of 3 declared
string-literal `client:` value(s); declined 1: line 96: client: `i18n.getLocales`

rowsParsed is still 3 — the old guard stays blind, as designed — yet the verdict fires and names the entry.

C. The window-inheritance case. Backtick-quote GET /api/v1/meta in rest-route-ledger.ts:

EXIT=1
ledger rows read ........... 258 of 259 declared
client-bound ledger rows ... 221 of 221 declared ← the client count is perfectly clean
✗ broken scan: …rest-route-ledger.ts is a PARTIAL read … read 95 of 96 … declined 1:
line 131: route: `GET /api/v1/meta`

D. Ablate the new verdict itself (if (false && (l.declined.length || unreadRows > 0)); on-disk: ablated form ×1, original ×0). The self-test reds on 4 pins, each naming what it lost:

✗ self-test "a partial read is a VERDICT, not a smaller number": brokenScan → expected true, got false
✗ self-test "and the verdict carries the numerator": 2 of 4 → expected true, got false
✗ self-test "and NAMES the entry it could not read": i18n/locales → expected true, got false
✗ self-test "yet the declined spelling still carries the verdict": brokenScan → expected true, got false
✗ affected-docs self-test failed (4 case(s)).

Restored: original form ×1, ablation residue ×0, ✓ 281 cases pass.

E. Ablate in the opposite direction — silently widen the recognizer (routeRe accepts all three quotes). 5 pins red, including the two that exist to catch exactly this:

✗ self-test "the narrow population is UNCHANGED — this reports, it does not widen": row count → expected 2, got 4
✗ self-test "read + declined accounts for every declared `route:`": partition → expected 4, got 6
✗ self-test "the client-bound COUNT is identical either way — the number is blind": clientRows → expected 1, got 2

Restored byte-clean: ✓ 281 cases pass.

5. Gates

All run at the final commit 4e7229a6 (working tree clean), verdict lines quoted as each gate printed them. Family derived by node scripts/pm/dispatch-gates.mjs with no path arguments — 3 paths, 10 matched families:

gateits own verdict line
affected-docs.mjs --self-test✓ affected-docs self-test: 281 cases pass.
check-audit-scope.mjs --self-test✓ check-audit-scope self-test: 24 cases pass.
check-audit-scope.mjs✓ docs-accuracy-audit scope is in sync with content/docs/: 181 hand-written doc(s).
check-affected-docs.mjs (check:docs-audit-scope + the drift gate step)exit 0, ledger rows read 259 of 259 declared
check-node-version.mjscheck-node-version: OK (29 setup-node step(s) across 26 workflow(s), all on Node 22).
check-workflow-status-functions.mjs --self-test + run✓ … 34 assertions; OK (scanned 26 workflow file(s), 45 job(s) …)
check-required-contexts.mjs --self-test + run✓ … 124 assertions; exit 0
check-shard-attestation.mjs --self-test + run✓ … 92 assertions; ✓ check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) …
check-cross-package-test-inputs.mjs --self-test + runAll 60 self-test cases passed.; OK: 12 package(s) read outside themselves, all declared …
check-nul-bytes.mjsOK (scanned 6146 text file(s) … no raw ASCII control bytes).
npx eslint scripts/docs-audit/affected-docs.mjsexit 0

Exit codes were captured before any pipe (cmd > file 2>&1; EXIT=$?), never after a tail.

No changeset: the diff is scripts/docs-audit/** and .github/workflows/** — repo tooling, no publish surface at diff time. skip-changeset applied.


Generated by Claude Code

…mber
`parseLedgerSource` recognizes a route-ledger row through a single-quote-only
regex. The pre-existing broken-scan guard is `rowsParsed === 0`, so it catches
only the all-or-nothing case; a ledger where SOME rows are spelled in a quote
the recognizer declines parsed fine and reported a smaller population with no
verdict and exit 0.
Measured on a718ee3, respelling one row of `i18n-route-ledger.ts`:
- backtick-quoted `route:` -> 221 -> 220 client-bound rows, no verdict, exit 0
- backtick-quoted `client:` -> 221 -> 220, `rowsParsed` still 3, exit 0
- backtick-quoted `route:` in `rest-route-ledger.ts` -> clientRows still 221,
but `meta.getTypes` moved onto the server-only `GET /api/v1/docs`: the row
window is delimited by the same single-quote-only lead, so a declined row
does not close the previous row's window
The last case is why the verdict keys on the DECLINED SPELLING rather than on a
count shortfall - the count is blind to it.
`parseLedgerSource` now returns what it declined beside what it read, and
`bridgeCoverageFrom` turns any declined declaration (or any declared row value
it did not read) into a `brokenScan` verdict naming the file, the line and the
literal. Both halves of the fraction print on every run. The recognizer is
untouched: today's seven ledgers are wholly single-quoted (259 route, 221
client, delta 0), so no verdict can fire on this tree, and `--self-test` pins
the population at its old value so a silent widening fails there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@os-zhuangos-zhuang added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed size/m labels Aug 21, 2026 — with Claude
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation ci/cd labels Aug 21, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review August 21, 2026 02:09
@os-zhuang
os-zhuang added this pull request to the merge queueAug 21, 2026
Merged via the queue into main with commit 6156951Aug 21, 2026
32 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-9896-docs-audit-partial-ledger-parse branch August 21, 2026 02:46
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddocumentationImprovements or additions to documentationskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs-audit: a PARTIAL ledger parse is silent — one backtick-quoted route: drops a row from the bridge population with no verdict and exit 0

1 participant

@os-zhuang