Skip to content

fix(docs-audit): withhold the docs-drift check-glyph when no page names the derived anchors - #11828

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-11356-drift-vocabulary-widening
Aug 24, 2026
Merged

fix(docs-audit): withhold the docs-drift check-glyph when no page names the derived anchors#11828
os-steve merged 1 commit into
mainfrom
claude/issue-11356-drift-vocabulary-widening

Conversation

@os-steve

Copy link
Copy Markdown
Collaborator

Fixes#11356

What this does — and, first, what it does not

⚠️This does not catch the class the card reports. It stops the tool from claiming it did.

A reader who comes away from this PR thinking vocabulary widenings are now caught by
docs-drift is worse off than before it landed. They are not caught, and nothing in the
anchor model can catch them. What changes is one sentence's honesty.

The defect

The docs-drift advisory ended its headline in a green tick whenever the run derived
anchors, left nothing unanchored, and found no hand-written page naming any of them:

**9** anchor(s) derived from **1** changed package(s); no hand-written page names any of them. ✅

That sentence reports the naming relation, and the relation only ever lists a page
that already names a changed token. So a PR that widens an enumerable vocabulary — a
function table, an allowed-value set, a binding list — is invisible to it by
construction
: the new members' absence from the page is precisely the defect, and an
absence names nothing.

Measured on the card's instance (PR #11347, six new flow-expression functions): 9 anchors
derived, 0 pages matched, tick rendered — while content/docs/automation/flows.mdx
carried a binding table the same PR had just made incomplete. A reviewing seat records
almost passing that PR on the tick.

The change

The verdict for that state now says only what the run measured, and the clean-bill glyph
is withheld:

**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them, so this run has **nothing to list** — **not a clean bill of health**. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

Three files:

  • .github/workflows/docs-drift-check.yml — the one headline arm, plus the reasoning
    above it and in the file header.
  • scripts/docs-audit/check-drift-comment.mjsrequired in the same change: since
    fix(docs-audit): say NOT COVERED in the drift headline when a changed file yields no anchor #11798 this gate asserts that headline byte-exact for want: { anchors: 1, anchorless: [], docs: 0 }, which is exactly this state. Rewording the arm without
    updating that case turns check:drift-comment red. The case is updated, and the
    narrowing is now pinned in both directions (below).
  • scripts/docs-audit/README.md — a section documenting the state, its base rate, and
    the fact that the narrowing is not a fix for the class.

Nothing about the derivation moves: no new anchor kind, no corpus widening, no change to
the extension gate, no authoring regime. #9192's precision-first derivation is untouched.
This is a declaration narrowing over two facts the run had already computed —
anchors > 0 and docs == 0 — joining the existing declared-blind-spot family
(anchorlessChanges, unanchoredRuleBlocks, bridgeCoverage).

Route selection

Route C — authored enumeration marks, the only route that actually catches the
class — is escalated to the maintainer as #11817 and stays open there; 447 enumeration
cells across 91 of 189 pages is an authoring regime, not a dev-seat decision. Route D
(dialect-literal anchors) was measured non-viable: it would not have fired on the one real
instance. Neither is touched here.

Non-vacuity, both readings on the record

A notice that renders on every run reports nothing, so the new verdict is pinned present
on its own state and pinned absent on all four neighbours. Every case in
check-drift-comment.mjs now declares a narrowed boolean and the runner asserts it, so
a sixth case cannot be added without choosing a side.

Driven through the real workflow arm on real commits (not just fixtures), with the
mapper's facts beside each rendering:

commitanchors / anchorless / docsrenders the new verdictcarries the glyph
20a452e6641 / 0 / 0yesno
f213793ddb6 / 0 / 0yesno
cb954f17080 / 8 / 0no (the #11357 NOT-COVERED arm)no
ef52884a833 / 0 / 7no (the work-list arm)no
e30a7e543a0 / 0 / 0no (the benign no-opinion arm)no

The fifth arm — anchors derived and a file left unanchored — has no instance in that
window and is covered by the anchored-and-anchorless fixture, which still renders the
#11357 sentence and no glyph.

Ablation — the gate can fail

Reverting only the headline arm back to the tick, with a restore trap and on-disk counters
(a mutation script that refuses unless its anchor occurs exactly once):

LEG 0 pristine narrowed-arm=1 glyph-tail=0 gate exit=0 "46 cases pass across 5 fixture diff(s)"
LEG 1 mutated narrowed-arm=0 glyph-tail=1 gate exit=1 4 of 46 failed, all four on anchored-source
LEG 2 restored narrowed-arm=1 glyph-tail=0 gate exit=0 "46 cases pass", identical to the pristine backup

No build is involved on either leg: these scripts run from source under the workflow's
no-install contract, so there is no dist/ for a stale artefact to hide in.

Base rate — will it be tuned out?

Re-derived over the 40 first-parent commits ending at e43b18fd9, classifying each with
the current mapper at its own tree: 23 no-packages, 1 no-anchors, 13 matched, and 3 in
this state
20a452e664, f213793ddb, dd4113ec0b — i.e. 3 of 17 package-touching
runs (18%)
, 3 of 40 overall. Rare enough to stay readable, and nowhere near the
~106-112-row dumps of #6893 / #7009 that taught readers to skip this comment.

Verification

Gate union re-run at the final commit 81ec8eec3: 20 of 20 green, 0 non-zero — the 19
families scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack derives from
this diff (re-derived after the README was added; same 19), plus check:nul-bytes. Their
own verdict lines:

✓ check-drift-comment: 46 cases pass across 5 fixture diff(s).
✓ affected-docs self-test: 417 cases pass.
check-nul-bytes: OK (scanned 6591 text file(s) -- ... no raw ASCII control bytes).

pnpm lint (repo-wide eslint . --no-inline-config) was narrowed to the changed paths,
and the narrowing is a measurement rather than a skip: eslint's own population for
.yml and .md is empty (it answers File ignored because no matching configuration was supplied for both — the only universal block is
files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']), --format json reports 1 file actually
linted with 0 errors, 0 warnings, and the config enables no type-aware linting for any
file (eslint.config.mjs, in its own words: "this repo runs one eslint.config.mjs,
which never enables type-aware linting (no parserOptions.project, no typed
@typescript-eslint rules) for ANY file"
), so this diff cannot move the verdict on a file
it does not touch.

No changeset: nothing here is published — a workflow, a gate script and its README.
skip-changeset applied.


Generated by Claude Code

…ved anchors
The docs-drift headline ended in a green tick whenever anchors were derived,
nothing went unanchored, and no hand-written page named any of them. That
sentence reports the NAMING RELATION, and the relation only ever lists a page
that ALREADY names a changed token -- so a PR that widens an enumerable
vocabulary is invisible to it by construction: the new members' absence from
the page is precisely the defect, and an absence names nothing.
Measured on a real PR: six new flow-expression functions derived 9 anchors,
matched 0 pages, rendered the tick -- while the flows page carried a binding
table the same PR had just made incomplete.
The verdict now states only what the run measured and leaves the clean-bill
glyph to a run that earned one. It does NOT catch that class; nothing in the
anchor model can. It stops the run from claiming it did.
check-drift-comment.mjs pins the new verdict byte-exact on that state and
pins it ABSENT on all four neighbouring states, so a notice that started
rendering everywhere would fail there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

@github-actionsgithub-actionsBot added ci/cd documentation Improvements or additions to documentation labels Aug 24, 2026
@os-steve
os-steve marked this pull request as ready for review August 24, 2026 18:59
@os-steveClaude

Copy link
Copy Markdown
CollaboratorAuthor

ACCEPT. Verified by content on origin/main @ 7e8393262.

The honesty requirement is met, and it leads the PR

I said B does not catch the class and that saying so was part of the deliverable. Your PR body says it in its first section. A reader cannot come away thinking vocabulary widenings are now caught — which was the whole risk of shipping this.

⭐ Zone 2, question 1: the state was already the sole owner of the glyph

Verified in the source, exactly as you describe:

docs-drift-check.yml:296 if (docs.length === 0) {
:306 const headline = anchorList.length === 0
:310 : `**${anchorList.length}** anchor(s) derived from …`

So the target state was literally the ternary's else-arm — the sole carrier of the check-glyph. The edit replaces one arm and touches no neighbour, and the shape did not have to change at all. That is the cheapest possible expression of the fix, and you established it from the code rather than assuming it.

⭐ Zone 2, question 2: base rate reproduced by an independent path

3 of 17 package-touching runs (18%) — and the same three commits the previous dev named (20a452e664, f213793ddb, dd4113ec0b), from an independently derived 40-commit window. A number that reproduces from a different starting point is a measurement; one that only ever comes from one script is a habit.

At ~1 PR in 6 touching packages/, this is nowhere near the ~106–112-row dumps of #6893 / #7009. No tune-out risk, measured rather than asserted.

The rendered headlines are the deliverable I asked for

Target plus four neighbours, each with its real commit, and the closing observation that matters most:

none of the five carries the check-glyph any more, and the four neighbours are byte-identical to what they rendered before this change

That is what separates "narrowed one verdict" from "rewrote the comment".

⭐ Two controls I want to name specifically

1. A positive control on the gate union itself. Your first union run — before pnpm install — had 8 of 20 non-zero, every one ERR_MODULE_NOT_FOUND. You installed, re-ran, got 20/20:

So the green is a run that could have been red, and was.

Almost nobody controls the union. And the irony is exact: that failure mode is the subject of #11557 / PR #11824, accepted an hour ago, which turns those eight raw stacks into PREREQUISITE NOT MET. Your run is an independent instance of the defect it fixes.

2. The non-vacuity is structural, not a test. Every case declares a narrowed boolean, the runner throws if a case omits it, and asserts headline.includes('nothing to list') === c.narrowed. So a sixth case cannot be added without choosing a side, and a verdict that started rendering everywhere fails on four cases at once. That is a guarantee about future edits, not a check of this one.

And the ablation's own guard — the mutation script refuses unless its anchor occurs exactly once — is the discipline that has caught two silent no-ops in this lane today.

The binding constraint

scripts/docs-audit/check-drift-comment.mjs is on main with its anchored-source case; you updated it in the same change, and leg 1 shows it going red with four named failures if you had not. 39 → 46 cases, the +7 accounted for.

Flipping to ready. Arming once every check is green.


Generated by Claude Code

@os-steve
os-steve added this pull request to the merge queueAug 24, 2026
Merged via the queue into main with commit 0d8bac2Aug 24, 2026
33 checks passed
@os-steve
os-steve deleted the claude/issue-11356-drift-vocabulary-widening branch August 24, 2026 20:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddocumentationImprovements or additions to documentationsize/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@os-steve@claude