Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 28 additions & 1 deletion .github/workflows/docs-drift-check.yml
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,6 +25,13 @@ name: Docs Drift Check
# on one day (#11180, #11262). The fold was right and stays; what changed is that a run
# with an unanchored file now says so where a reviewer reads it, and withholds its ✅.
#
# Since #11356 the ✅ is withheld from the last arm that carried it: "N anchor(s) derived,
# no hand-written page names any of them". That sentence reports the NAMING RELATION, and
# a page is listed only when it already names a changed token — so a PR that widens an
# enumerable vocabulary (a function table, an allowed-value set) is structurally invisible
# to it, because the new members' absence from the page is precisely the defect. ⚠️ The
# narrowing does NOT catch that class; it stops the run from claiming it did.
#
# And since #9519 it states WHICH TREE it read. On `pull_request`, actions/checkout gives
# the MERGE of base and head, so the row set is a fact about a commit that exists on no
# branch a reader can name — while the comment's own re-derivation command sent them to
Expand DownExpand Up@@ -303,11 +310,31 @@ jobs:
// ⛔ The ✅ is DROPPED whenever a file went unanchored, deliberately: a green
// tick is the clean-bill glyph, and "no page names the anchors I derived" is
// not a clean bill when some of the diff derived no anchor to name.
//
// ⛔ And since #11356 the ✅ is gone from this arm ENTIRELY — including the run
// where nothing went unanchored, which is where it used to live. "No page names
// the anchors I derived" is a fact about the NAMING RELATION, not about the
// docs: a page is listed because it ALREADY names a changed token, so a change
// that WIDENS an enumerable vocabulary can never be caught here — the new
// members' absence from the page IS the defect, and an absence names nothing.
// Measured on #11347 (six new flow-expression functions): 9 anchors derived, 0
// pages named them, ✅ — while `content/docs/automation/flows.mdx` carried a
// table enumerating the available bindings that the PR had just made
// incomplete, and a reviewing seat almost passed the PR on that tick.
//
// ⚠️ This does NOT catch that class, and must not be read as doing so. Nothing
// in the anchor model can (three in-model routes were measured to fail on the
// one real instance; the authoring-mark route that would catch it is escalated
// as #11817). What it stops is the run CLAIMING it did: the verdict now says
// only what it measured — anchors were derived, no page named them — and leaves
// the clean-bill glyph to a run that earned one.
const headline = anchorList.length === 0
? (anchorless.length
? `${notCovered} Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **${pkgs.length}** changed package(s)).`
: `Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **${pkgs.length}** changed package(s)), so **this run has no opinion** about the docs.`)
: `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them.${anchorless.length ? ` ${notCovered}` : ' ✅'}`;
: (anchorless.length
? `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them. ${notCovered}`
: `**${anchorList.length}** anchor(s) derived from **${pkgs.length}** changed package(s); no hand-written page names any of them, so this run has **nothing to list** — **not a clean bill of health**. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.`);
// The tree identity is rendered here only when anchors WERE derived: that
// is the run with an opinion ("no page names any of them"), and a reader
// whose own tree does yield a row is owed the reason. With no anchors the
Expand Down
30 changes: 30 additions & 0 deletions scripts/docs-audit/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -209,6 +209,9 @@ reader has to expand a fold to find is one a reader does not find. The failure #
records was never the tool lying; it was the tool never signalling its own limits at the
point of use.

Since #11356 the same posture reaches the ✅ itself — see **The verdict when nothing names
the anchors** below.

### Measured, before and after

Ten real PRs, each re-derived at its own merge base with its own docs corpus. `docs` rows:
Expand DownExpand Up@@ -241,6 +244,33 @@ run says so and points at the coarse set — where the old tool's 22 rows were e
mentioning `@objectstack/cli`. A CLI **command name** (`os meta resync`) is exactly the
recall class the shape guard costs us: it is a lowercase word, so it cannot anchor.

### The verdict when nothing names the anchors

That state — anchors derived, nothing left unanchored, no page naming any of them — used
to end the headline in a ✅. It no longer does (#11356). The sentence reports the **naming
relation**, and the relation only ever lists a page that ALREADY names a changed token, so
a PR that **widens an enumerable vocabulary** is invisible to it by construction: the new
members' absence from the page is precisely the defect, and an absence names nothing.

Measured on #11347 — six new flow-expression functions (`round`/`floor`/`ceil`/`abs`/
`min`/`max`) — the run derived 9 anchors, matched 0 pages, and rendered the ✅, while
`content/docs/automation/flows.mdx` carried a table enumerating the available bindings
that the same PR had just made incomplete. A reviewing seat almost passed the PR on that
tick.

⚠️ **The narrowing does not catch that class.** Nothing in the anchor model can: anchoring
the new members is impossible by construction, the siblings live in carriers the diff
never touches, and the container symbol is named by no page. The authoring-mark route
that would catch it is escalated as #11817. What the narrowing changes is that the run no
longer **claims** it did — the verdict states only what it measured, and the clean-bill
glyph is left to a run that earned one. The rendering is pinned in both directions by
`check-drift-comment.mjs`, which asserts the verdict byte-exact on that state and asserts
it ABSENT on all four neighbouring states.

How often it renders, re-derived over the 40 first-parent commits ending at `e43b18fd9`:
3 of the 17 package-touching runs (18%) — `20a452e664`, `f213793ddb`, `dd4113ec0b` — so it
is a rare notice rather than a per-PR banner, which is what keeps it readable.

**Cost** (the card's open question): the anchor derivation reads the same 178-page corpus
the old one did, plus the 18 route-registrar/ledger sources (~875 KB) and one `git show`
per changed file per side. Measured end-to-end on the ten PRs above, `node affected-docs.mjs`
Expand Down
41 changes: 38 additions & 3 deletions scripts/docs-audit/check-drift-comment.mjs
Original file line numberDiff line numberDiff line change
Expand Up@@ -35,6 +35,16 @@
*
* diff → the real affected-docs.mjs → its real --json → the real comment script → bytes
*
* Since #11356 it pins one more verdict the same way: the `anchors-derived-none-named`
* state — anchors derived, nothing unanchored, no page naming any of them — used to end
* in a ✅. That glyph read as "docs verified" on a change whose own page had just gone
* incomplete, because a page is listed only when it ALREADY names a changed token, so a
* PR that widens an enumerable vocabulary can never be caught by the relation (the new
* members' absence from the page IS the defect). The narrowing does not catch that class;
* it stops the run from claiming it did. Both readings are on the record below: the
* verdict is pinned byte-exact on that state and pinned ABSENT on all four neighbours,
* because a notice that renders everywhere reports nothing.
*
* Each case declares the mapper facts it depends on (`anchors` / `anchorless` / `docs`)
* and those are asserted BEFORE the rendered text is. Without that, a change in how the
* mapper classifies a `.md` file would quietly move every case onto the same branch and
Expand DownExpand Up@@ -182,6 +192,15 @@ const NAMES_THE_ANCHOR = '# Guide\n\nThe cap is `MAX_WIDGETS`, and it bounds the
const ADDS_ANCHOR = 'export function renderWidget(x: number) {\n return x + 1;\n}\n\nexport const MAX_WIDGETS = 5;\n';
const README_EDITED = '# @objectstack/demo\n\nRun `os demo` to start. Then run `os demo studio` for the UI.\n';

/** The #11356 verdict, byte-exact — rendered by exactly one of the five cases below. */
const ANCHORS_DERIVED_NONE_NAMED =
'**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them,'
+ ' so this run has **nothing to list** — **not a clean bill of health**. This check sees only'
+ ' pages that NAME a derived anchor: one that documents this change in prose, or enumerates it'
+ ' in an authoring dialect, names none and stays invisible to it on every run.';
/** The phrase that tells that verdict apart from every neighbouring one. */
const NARROWED_PHRASE = 'nothing to list';

/**
* `want` is the mapper contract each case rides on — asserted before any text is, so a
* case that silently stopped exercising its branch fails here instead of passing there.
Expand All@@ -193,6 +212,7 @@ const CASES = [
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
change: { 'packages/demo/README.md': README_EDITED },
want: { anchors: 0, anchorless: ['packages/demo/README.md'], docs: 0 },
narrowed: false,
expect: (headline, body) => {
check('readme-only', 'the headline names the blind spot', true, headline.includes('NOT COVERED by this run'));
check('readme-only', 'the headline names the file that went unanchored', true, headline.includes('packages/demo/README.md'));
Expand All@@ -203,15 +223,22 @@ const CASES = [
},
{
id: 'anchored-source',
what: 'an anchorable source file, no page naming it — the arm',
what: 'an anchorable source file, no page naming it — the anchors-derived-none-named arm (#11356)',
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR },
want: { anchors: 1, anchorless: [], docs: 0 },
narrowed: true,
expect: (headline) => {
check('anchored-source', 'the blind-spot sentence does NOT render — it is a report, not boilerplate',
false, headline.includes('NOT COVERED'));
check('anchored-source', 'the headline is the untouched ✅ text', true,
headline === '**1** anchor(s) derived from **1** changed package(s); no hand-written page names any of them. ✅');
check('anchored-source', 'the headline is the narrowed #11356 verdict, byte-exact', true,
headline === ANCHORS_DERIVED_NONE_NAMED);
// The point of the narrowing, asserted as its own reading: this state no longer
// carries the clean-bill glyph. Byte equality above would catch a ✅ appended to
// the end, but not one moved elsewhere in the line.
check('anchored-source', 'the clean-bill glyph is withheld', false, headline.includes('✅'));
check('anchored-source', 'and the verdict says what it could not see, not just what it found', true,
headline.includes('not a clean bill of health'));
},
},
{
Expand All@@ -220,6 +247,7 @@ const CASES = [
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR, 'packages/demo/README.md': README_EDITED },
want: { anchors: 1, anchorless: ['packages/demo/README.md'], docs: 0 },
narrowed: false,
expect: (headline) => {
check('anchored-and-anchorless', 'the derived anchors are still reported', true, headline.startsWith('**1** anchor(s) derived'));
check('anchored-and-anchorless', 'and the unanchored file is reported beside them', true, headline.includes('NOT COVERED by this run'));
Expand All@@ -232,6 +260,7 @@ const CASES = [
base: { 'content/docs/guide.mdx': NAMES_THE_ANCHOR },
change: { 'packages/demo/src/widget.ts': ADDS_ANCHOR },
want: { anchors: 1, anchorless: [], docs: 1 },
narrowed: false,
expect: (headline, body) => {
check('docs-listed', 'the work-list headline is unchanged', true,
headline === 'This PR changes **1** package(s): `@objectstack/demo`, touching **1** documentable anchor(s).');
Expand All@@ -244,6 +273,7 @@ const CASES = [
base: { 'content/docs/guide.mdx': NAMES_NOTHING },
change: { 'content/docs/guide.mdx': `${NAMES_NOTHING}\nOne more line.\n` },
want: { anchors: 0, anchorless: [], docs: 0 },
narrowed: false,
expect: (headline) => {
check('no-package-change', 'the original headline is preserved byte-for-byte', true,
headline === 'Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from **0** changed package(s)), so **this run has no opinion** about the docs.');
Expand DownExpand Up@@ -275,6 +305,11 @@ try {
check(c.id, 'the comment keeps its heading', '### 📓 Docs Drift Check', lines[1]);
const headline = lines[2];
if (PRINT) console.log(`\n── ${c.id} — ${c.what}\n${headline}\n`);
// Non-vacuity for the #11356 verdict, asserted in the LOOP rather than per case, so a
// sixth case cannot be added without declaring which side of the line it sits on.
if (typeof c.narrowed !== 'boolean') throw new Error(`case ${c.id} does not declare \`narrowed\``);
check(c.id, `the #11356 narrowed verdict ${c.narrowed ? 'renders here' : 'does NOT render here'}`,
c.narrowed, headline.includes(NARROWED_PHRASE));
c.expect(headline, body);
}
} finally {
Expand Down
Loading