fix(docs-audit): say NOT COVERED in the drift headline when a changed file yields no anchor

[claude]

Fixes#11357

A README carries no @docs-rule block, exports no symbol, mounts no route and declares no
SDK method, so a README-only diff derives no anchor at all and the docs-drift advisory
answers it with "this run has no opinion about the docs" — which a reviewer reads as
"nothing to check". Two real README defects landed inside that gap on one day: #11180
(packages/cli/README.md advertising os studio, a command the CLI does not ship) and
#11262 (packages/console/README.md asserting a @object-ui/console fallback the code no
longer performs). Neither was detectable by this check on any run.

This is #9282's option 4 — record the blind spot in the check's own output. It is a
reporting change: anchor derivation is untouched, and no run's verdict moves (this
check never fails a build in the first place).

The defect is in the headline, not in the honesty

The run was already honest — the limits line names the file it could not anchor, and the
card is precise about that. But limits renders inside a collapsed details fold that GitHub
renders collapsed, under the "no opinion" headline. A reviewer who does not expand the
fold reads the whole comment as a clean bill of health. So the fix states the blind spot in
the headline, where it cannot be missed; the fold keeps the detail, unchanged.

Whenever anchorlessChanges is non-empty, all three headline arms now name the count and
the files, say those pages are NOT COVERED by this run, and withhold the ✅ — a
green tick is the clean-bill glyph, and a partial look is not a clean bill.

Before / after, both directions, verbatim

Rendered end to end: fixture git diff → the real affected-docs.mjs --json → the real
inline github-script block → these bytes. Commit shas in the fixtures are pinned by fixed
commit dates so the two runs are byte-comparable.

A README-only diff — the specimen class:

 « the dedup marker comment »
### 📓 Docs Drift Check
- Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **1** changed package(s)), so **this run has no opinion** about the docs.+ ⚠️ **1** changed file(s) yielded no anchor (`packages/demo/README.md`), so the pages documenting them are **NOT COVERED by this run** — this is not a clean bill of health for those files. Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **1** changed package(s)).
« collapsed fold: What this run could not see »
- **1** changed file(s) yielded no anchor (`packages/demo/README.md`) — pages documenting those are invisible to this run
…

That one line is the entire diff between the two comment bodies.

A diff touching an anchorable source file — the whole body is byte-identical, headline
and all:

« the dedup marker comment »
### 📓 Docs Drift Check
**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them. ✅
« collapsed fold: Which tree this was computed on »
…

The new gate renders the workflow instead of grepping it

scripts/docs-audit/check-drift-comment.mjs extracts the Comment on PR step's
github-script body and runs it the way the action does — over stub github /
context / core / require — against real affected-docs.mjs --json output from
throwaway git repos. A source grep would pass just as happily on text that never renders
and on text that renders unconditionally, and neither of those is a report.

Five fixture diffs, 39 assertions. Each case asserts the mapper facts it rides on
(anchors / anchorlessChanges / docs) before it asserts any text, so a case that
silently stopped exercising its branch fails there instead of passing here.

fixture diff	anchors	anchorless	docs	headline must
README only	0	1	0	say NOT COVERED, name the file, drop "no opinion"
anchorable source, unnamed	1	0	0	be the untouched ✅ text, with no notice
both	1	1	0	report the anchors and the blind spot, withhold ✅
anchorable source, page names it	1	0	1	be unchanged; the row list still renders
nothing under packages/	0	0	0	be preserved byte-for-byte

Zero dependencies, like the mapper: this job runs no pnpm install.

Non-vacuity — the gate observed RED in both directions

Each leg: mutate → prove the mutation reached disk → run → restore → prove the restore.
(The first attempt at leg 2 was a silent no-op — a shell-quoting error killed the edit
script while the gate stayed green. The disk-landing proof is what caught it; without it,
"green" would have read as "the pin does not fire", which is the opposite of the truth.)

• Revert the headline change (workflow restored to 56630b7ee; NOT COVERED
  occurrences 1 → 0, 110-line diff): 6 of 39 fail, all on the two anchorless cases —
  readme-only (4) and anchored-and-anchorless (2). The three no-anchorless controls stay
  green.
• Make the sentence unconditional (4 conditional sites → true; injected sites 4/4,
  remaining anchorless-gated arms 0): 5 of 39 fail, all on the three cases that must
  not carry it — anchored-source (2), docs-listed (1), no-package-change (2).

Restored both times: git status --porcelain empty, both gates green again.

Scope

• ⛔ Not option 3. The coarse package-mention fallback was measured 0-for-3 on recall on
  its own specimen ([finding] docs-drift is structurally blind to declared-fields.ts — the file that CARRIES a rule several pages restate yields no anchor, so changing the rule produces "no opinion" #9282), and would regrow the ~106–112-row lists [finding][devx] Docs Drift Check is package-granular, so any PR touching @objectstack/spec lists ~112 docs — an advisory that large is one every reader learns to skip #6893 / [devx] Docs Drift Check edges are still package-granular: a one-line JSDoc edit in @objectstack/client lists 11 docs, a spec src change 106 #7009 measured
  and readers learned to skip; docs-drift-check: cap the PR comment above a threshold — a hub-package PR posts an invariant ~106-row list into every subscribed agent session on every push #9037 had to cap the comment for the same reason.
• ⛔ Not option 1 for READMEs. Whether a README could carry a @docs-rule block and opt
  in under the existing mechanism is a larger question the card leaves unmeasured, and this
  PR does not touch it.
• ⛔ No anchor logic touched.affected-docs.mjs is not modified at all in this PR; its
  self-test (417 cases) is a control here, not a subject.
• Judgment call worth a reviewer's eye: the card quotes only the "no opinion" headline, and
  the fix applies the same conditional to all three headline arms. One rule — if a file
  went unanchored, the headline says so — rather than a notice that appears on one branch
  and not its neighbours. With nothing unanchored, all three arms are byte-identical to what
  they were.

Verification

Run on b56d379e5 (the final commit):

• node scripts/docs-audit/check-drift-comment.mjs → ✓ 39 cases pass across 5 fixture diff(s)
• node scripts/docs-audit/check-affected-docs.mjs → ✓ affected-docs self-test: 417 cases pass. (control — unchanged)
• The 17 further families node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack derives for these paths: all green (19/19 including the two above).
• ESLint, narrowed and measured: of the three changed paths, ESLint's own config supplies no
  configuration for the .yml and the .md (both reported File ignored because no matching configuration was supplied), and the one file in its population,
  check-drift-comment.mjs, lints clean — 0 errors, 0 warnings, counted from --format json.
  This repo runs one eslint.config.mjs which never enables type-aware linting for any file
  (eslint.config.mjs:326-335, stated there with a positive control), so this diff cannot
  move the verdict on any untouched file.

No changeset: workflow + tooling script + internal README only, nothing published changes —
skip-changeset.

---

Generated by Claude Code

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

[os-steve]

ACCEPT. Verified by content on origin/main, not by reading the report back.

You strengthened the card, and the addition is decisive

The card's case was that the headline reads as "nothing to check". You found the structural reason: the detail line is inside a collapsed fold.

203 if (anchorless.length) limits.push(`**${anchorless.length}** changed file(s) yielded no anchor …`)
226 ? ['', '<details><summary>What this run could not see</summary>', '', ...limits.map(…)]
252 ? `Nothing in this diff resolved to a documentable surface …`

So the honest half of the report is real but hidden behind a disclosure triangle, while the misleading half is the headline. That is not "a reviewer might misread it" — it is "a reviewer sees only the misleading half by default". The fix belongs exactly where you put it.

My scoping held, and you checked rather than assumed it

I ruled option 4 as a reporting fix and flagged in Zone 2 that if the run could not tell which files were anchorless at headline-composition time, the fix reached further than I thought. Verified: anchorless is read at :103, headline composed at :252. And you established the separability I asked for — "this file produced no anchor" vs "this diff had nothing to document" (the latter being anchorless = 0 and anchors = 0, e.g. a tooling-only PR). Option 4 did not reach further. affected-docs.mjs untouched, as ⛔'d.

⭐ The silent no-op you caught, and published

my first attempt at leg 2 was a silent no-op — a shell quoting error killed the edit script while the gate reported green. The disk-landing proof caught it; the exit-0 edit tool plus a green gate would otherwise have read as 'the pin does not fire', the exact inversion the rule exists to prevent.

That is the whole reason mutations must be proven on disk rather than by an editor's exit code, and it is the third variant of the swallow-your-own-failure trap this lane has logged today (| tail eating an exit code; gh returning empty; a stripped-comment regex eating its own subject). Publishing the miss is what makes the eventual reds believable.

The new gate's design carries the same instinct: 39 assertions running real git diff → real affected-docs.mjs --json → the workflow's real inline github-script block extracted from the YAML, and each case asserting its mapper facts (anchors / anchorlessChanges / docs) before any text — so a case that stopped exercising its branch fails there rather than passing here vacuously. Both ablations red on exactly the cases that should fail and green on the controls (6/39 on revert, all on the two anchorless cases; 5/39 on unconditional, all on the three that must not carry it).

Ruling on question 2: A, keep it uniform. As shipped.

Narrowing to the quoted arm would leave the ✅ arm — a literal clean bill of health — as the one place an unanchored file goes unmentioned. That is the same misreading one branch over, and strictly worse: "no opinion" at least signals absence, while "✅" asserts presence of a check that did not happen. The work-list arm has the same problem in a milder form: it otherwise reads as the list.

It is the same conditional at the same composition point, no anchor logic moves, no verdict moves, and you measured that with anchorless = 0 all three arms render byte-identical to before (anchored-source fixture, commit dates pinned). One rule beats a notice on one branch and not its neighbours.

Ruling on question 1: A, and I am filing it — you were right to leave it to me.

Your reasoning is the deciding one: #9632 (README links) is already closed, and #11357 is the only open row naming this corpus. Once it closes, "READMEs are invisible to every docs gate" is recorded nowhere open — and the next README defect gets re-discovered instead of recalled. Two such defects landed on one lane in one day (#11180's phantom os studio, #11262's dead @object-ui/console fallback); a third will happen.

Option 4 reports the blind spot. It does not close it. Those are different facts and only one of them is now true in the repo.

I am filing the residual as a new unassigned finding carrying the card's two unmeasured questions — option 3's noise cost on a hub package (#6893 / #7009 measured ~106–112 rows, learned-to-skip, #9037 capped the comment), and whether any README could reasonably carry a @docs-rule block and opt in under #9394's existing mechanism. ⛔ Not re-opening option 3, which I declined on the measurement; the new row records the coverage gap, not a proposed fix.

Keep Fixes #11357 — C would leave a card open for work its own PR does not do, which is how "open" stops meaning anything.

Flipping to ready. Arming once every check is green.

---

Generated by Claude Code