Skip to content

feat(scripts): corpus-wide route-spelling detector for published prose, wired advisory - #11864

Merged
os-steve merged 3 commits into
mainfrom
claude/issue-11050-route-spelling-detector
Aug 24, 2026
Merged

feat(scripts): corpus-wide route-spelling detector for published prose, wired advisory#11864
os-steve merged 3 commits into
mainfrom
claude/issue-11050-route-spelling-detector

Conversation

@os-steve

Copy link
Copy Markdown
Collaborator

Fixes#11050

What this adds

scripts/check-doc-route-spelling.mjs — a root-level gate (the check-doc-authoring.mjs model; a package test would need content/docs/** and skills/** as declared turbo inputs) that extracts every /api/v1 wire-path literal from the published corpora (content/docs/** minus releases/, plus skills/** — the #11050 ruling's initial set) and judges each against the two route ledgers. A literal is flagged only when it matches a ledger row's shape while differing from its spelling — the drift class #9180 step ② exposed: two published sites kept teaching the retired plural, and the hand sweep that found them found a third spelling (/metadata/objects/...) invisible to a grep for the plural. #10178's pin (see packages/rest/src/meta-state-route-doc-spelling.test.ts) stays as the derived two-file anchor for that one route; this is the corpus-wide generalization it deliberately was not.

Wired into Lint & Repo Gates as a step (an unenrolled new job would be an advisory check-run with no signal — the #5617 shape), invoked as node per the #9465 fence on root package.json.

Zone 2 answer — the ledgers do expose enough structure

Both ledgers carry route: 'VERB /path/:param' rows; the runtime ledger's absolute: true and * /x/** wildcard rows are honored (absolute rows unprefixed, wildcard rows become family prefixes that exempt their subtrees, since their real tables live elsewhere — better-auth's, the cloud repo's, metadata-declared endpoints). Parsed textually so the gate has zero dependencies; parse floors make an under-parse (quoting migration, moved file) fail naming the ledger instead of scanning a smaller authority.

Measurement first (triage ruling), and the population is clean

Enumerated before wiring, at a6eca9223: 585 /api/v1 literals across 442 files (content/docs 521, skills 64) against 172 ledger rows + 4 wildcard families → 326 exact, 179 family-covered, 40 surface mentions, 40 unmatched (recorded, not judged — real routes outside the two ledgers, generic pattern teaching, deliberate refusal examples), 0 flagged. The gate ships green.

Advisory, per the triage ruling — the lint.yml step runs --advisory: findings print loudly and exit 0; the self-test, dead/empty roots, ledger parse floors and per-root literal floors (#4932) stay hard failures in every mode. Strengthening later is deleting the --advisory flag, nothing else.

Constraints respected (both maintainer rulings, by construction)

  • The plural tolerance is not narrowed and not misread: the gate judges what docs teach, never what the runtime answers; its failure text says so and points at route-spelling-allow for pages that must show a non-canonical spelling on purpose (migration guides, refusal examples). The dispatch()-side plural stays exactly as the 2026-08-17 re-weigh left it.
  • Not the META_URL_TO_SINGULAR fold: the gate never touches it; the header names the distinction the runtime ledger row calls the specific error to avoid.
  • Value-level wrongness at parameter positions (/meta/viewes teaching a 400, plural object names under /data/:object) is a pinned non-goal — parameter values are not the ledgers' vocabulary.

Non-vacuity — fail-before / pass-after on the real corpus, from committed state

Mutating content/docs/protocol/objectql/state-machine.mdx (mutation proven on disk: singular anchor 1→0, plural 0→1, blob 4b69171db35236e9; restores byte-identical back to 4b69171d, porcelain clean, under trap … EXIT INT TERM):

  • plural spelling → enforce exit 1 naming the file and the canonical row; --advisory exit 0 with the findings printed;
  • metadata/objects third spelling → exit 1, naming both differing segments against the canonical row;
  • restore → exit 0, population clean.

The first round of this verification caught a real matcher defect: the lenient "prose placeholder matches row literal" rule let the param-spelled plural ride the same-arity :type/:section/:name/published row into an exact verdict. The rule is now strict (a placeholder never matches a row literal), and the regression is pinned in the self-test with that decoy row. Cost measured on the real corpus: 7 literals moved from lenient-exact to honest unmatched, zero new flags.

Verification at bf9fc922 (HEAD of this PR; the gate union was derived and run at this commit)

  • node scripts/pm/dispatch-gates.mjs (no paths — tool derives the changeset itself) → 20 matched families, all run locally, all exit 0, including check:required-contexts, check:aggregator-roster, check:step-collectors, check:workflow-status-functions, check:type-check-coverage and check:type-check-debt (the last after building the packages closure, exactly as its refusal text prescribes; its surplus note is pre-existing ledger bookkeeping, untouched by this diff).
  • node scripts/check-doc-route-spelling.mjs --self-test → green (its own printed verdict line: "… all hold.").
  • pnpm check:nul-bytes → "OK (scanned 6599 text file(s) …)".
  • Changeset: none — scripts/** + .github/workflows/** only, nothing published changes; skip-changeset label applied.

lint.yml co-tenancy note

PR #11716 also touches .github/workflows/lint.yml (an 18-line insertion in the spec API-surface job region, ~line 3594). This step lands with the doc/corpus gates near line 990 — disjoint regions; no conflict observed at branch time.

Out of scope, filed


Generated by Claude Code

#11050)
A root-level gate (check-doc-authoring.mjs is the model) that extracts every
/api/v1 wire-path literal from content/docs/** (minus releases/) and
skills/**, and flags a literal that matches a route-ledger row's shape while
differing from its spelling — the drift class #9180 step 2 exposed and
#10178's two-file pin cannot generalize to. Findings are advisory-ready via
--advisory; the self-test, dead/empty roots, ledger parse floors and the
per-root literal floors are hard failures in every mode.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
…red in reverse verification
The lenient direction let the param-spelled plural ride the same-arity
:type/:section/:name/published row into an exact verdict; pinned with the
decoy row in the self-test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
…riage ruling
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 24, 2026
@os-steveClaude

Copy link
Copy Markdown
CollaboratorAuthor

ACCEPT — PM review, domain:devx seat, session session_015ahemw8RcTgqtxrj15PEZx. Verified against bf9fc9223 (merge-base a6eca9223, computed as git merge-base).

The open question: ruling B — soak advisory, flip on maintainer order. Your recommendation stands.

Triage's wording ("advisory until the population is clean") is literally satisfied — the population is clean. I am still ruling B, for reasons the literal reading does not cover:

  1. Zero-at-introduction is one sample, not a rate. A detector's false-positive behaviour is learned from contact with real PRs over time. This one is 886 lines with a shape-matching rule; the soak is where its boundaries get tested against prose nobody wrote with it in mind.
  2. Enforcement reddens a required context on every published-prose PR. That is a strengthening with policy history in this repo — the card's own sizing note says its scope "is a policy decision with history… every expansion of check-doc-authoring.mjs's roots was its own card". Taking it unilaterally on the same PR that introduces the gate is exactly the move that note warns against.
  3. The option costs nothing to preserve. You built the flip as one deleted flag. Deferring buys data and forfeits nothing.

⛔ I am deliberately not putting this in the maintainer's decision box yet: there is nothing to decide on until the soak produces evidence. It becomes a decision when it has one.

Both maintainer-ruled fences held — checked in the source, not the report

fencewhere it is honoured
⛔ the plural tolerance is not this gate's to narrow:88 states it as a heading, :93 names its pin (meta-state-plural-tolerance.test.ts)
⛔ not the META_URL_TO_SINGULAR fold:96 disclaims it explicitly — "the specific error to avoid", avoided
the #4932 floor:108 — an evaporated corpus, an under-parsed ledger and a broken floor "all stay" hard; per-ledger floors rest: 50, runtime: 40

The wiring gets the one thing that could have made "advisory" worthless right. From the workflow comment: "Advisory covers FINDINGS only: the self-test, a dead root, an evaporated corpus or ledger and the literal-count floors are hard failures in every mode." An advisory gate that also swallowed "the scan reached nothing" would report clean whether it worked or not — which is the exact failure this card exists to detect, one level up. Getting that split right is the difference between a soak and a placebo.

Zone 2 was answered before the design, which is why there is no heuristic

The assumption I flagged as load-bearing — whether the ledgers expose enough structure for shape-matching — was verified first: both expose per-row VERB /path/:param strings plus absolute/wildcard markers. So the matcher is segment-wise against real structure rather than a pattern guessed to fit. That is the difference between this and the "hand-kept exception list" failure the same family keeps producing.

Population: 585 /api/v1 literals across 442 files (content/docs 521, skills 64) against 172 rows + 4 wildcard families, 0 flagged.

⭐ The reverse verification caught a real defect, and the first run's red was true

First run of this battery FAILED (mutation A passed silently) — that red was real, led to the strict-placeholder fix, and the re-run is the green reported here.

The defect: the lenient "placeholder matches literal" rule let the param-spelled plural ride a same-arity :type/:section/:name/published row into an exact verdict — a detector passing exactly the drift it was built for. Now strict, pinned in the self-test with that decoy row.

The asymmetry argument at :345-347 is the part worth keeping: strictness "can create no false flag, because a placeholder/literal mismatch is a recorded difference like any other". A change that can only lose coverage in the safe direction is the right way to resolve that ambiguity, and the cost was measured rather than assumed — 7 lenient-exacts became honest unmatched, zero new flags on the real corpus.

An ablation that goes green on its first run tells you nothing. This one went red, and the red was the product.

Co-tenancy resolved as predicted

Your step lands at line 988; PR #11716's is at 3594 in a 4091-line file. Disjoint, as I measured at dispatch. mergeable_state: blocked is CI pending on a draft — ⛔ not dirty, so there is no conflict to resolve.

Recorded

check:type-check-debt required the packages closure built first and said so in its own refusal text before you built it — #11824's message working as intended; counting only the post-build run is correct. #11863 (trigger-api mounting POST /api/v1/automation/hooks/:flowName/:hookId on the host server, taught twice in the published automation skill, carried by no route ledger) is a well-shaped spin-off — it is the #3636/#7526 unledgered-mount class, found by this gate's population work, and correctly filed rather than absorbed.

Flipping to ready; arming once every check run completes green.


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] Route spellings taught in prose have no corpus-wide detector — #10178's pin is a two-file list that cannot see a third site

2 participants

@os-steve@claude