Skip to content

docs(scripts): the published-README baseline's $comment describes its contract instead of a seed count - #9764

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-9649-baseline-provenance
Aug 19, 2026
Merged

docs(scripts): the published-README baseline's $comment describes its contract instead of a seed count#9764
os-steve merged 1 commit into
mainfrom
claude/issue-9649-baseline-provenance

Conversation

@os-steve

Copy link
Copy Markdown
Collaborator

Fixes#9649

The baseline's $comment claimed the file was "seeded from the five instances #9532
measured". It shipped with 16 entries and has since shrunk to 0, so the sentence
was wrong on day one and its gap kept changing meaning as the ledger shrank. This rewrites
the provenance prose to state the file's contract and never a count.

Measured on origin/main, one row per commit that touched the file:

commitlanded asentries
1c6da6eafPR #9546 — the PR that added the gate16
f01c0ee2dPR #9602 — five service READMEs10
890b38f04PR #9615 — five prompts entries5
cd455c83bPR #9581 — the five README entries, conflict resolved as the union of two disjoint deletion sets0

No baseline entry is added, removed or edited. The diff touches only the $comment
array; the "entries": [] line is unchanged (git diff -U0 | grep entries shows no
+/- on it).

What the new prose says

H1 — can the file be deleted now that it is empty? No, and that is the reason the block should carry

Read from the loader, not assumed. scripts/check-published-readme-exports.mjs:649-657:

functionloadBaseline(){constabs=join(ROOT,BASELINE_REL);if(!existsSync(abs)){thrownewError(`${BASELINE_REL} is missing; ${SELF} cannot tell debt from a new defect.`);}constparsed=JSON.parse(readFileSync(abs,'utf8'));if(!Array.isArray(parsed.entries))thrownewError(`${BASELINE_REL}: \`entries\` must be an array`);returnparsed.entries;}

Deleting the file is a hard failure, not a clean pass — the third of the PM's three
options, a "cannot verify" refusal. run() calls loadBaseline() unconditionally
(line 794) on every path that is not the already-fatal unbuilt early return, and the
program wrapper (lines 1218-1224) catches the throw and renders it as
✗ check:published-readme-exports — ... with exit 1. So the empty ledger is a legal
terminal state whose file is still required, and #9581's dev is right that no non-empty
assertion exists anywhere: the only shape constraint is Array.isArray(parsed.entries).
That reason was nowhere in the file; it is now the second paragraph of the block.

H2 — is the $comment load-bearing to any tooling? No — it is prose for humans

grep -rn '\$comment' over scripts/, .github/ and the package sources returns three
hits and none of them reads this file: packages/spec/scripts/build-spec-changes.ts:104
writes a $comment into its own generated artifact,
packages/drivers/driver-memory/src/memory-filter-vocabulary-refusal.test.ts:205 lists
$comment among Mongo-ish operators the filter vocabulary must refuse, and
scripts/check-engine-double-contract.mjs:80 mentions a sibling baseline's $comment in
a source comment. Nothing parses it.

Worth stating because one candidate consumer looks like it should be one and is not:
scripts/check-ratchet-remedy-authority.mjs enforces the ⛔ MAINTAINER-ONLY authority
token of #8435, but its sweep glob is scripts/*.{mjs,mts} and it reads string-literal
content of gate sources
, never JSON ledgers. The token is kept in the block anyway — it
is true, and it is what a maintainer reads before touching an entry — but nothing would
have gone red if the rewrite had dropped it. So the rewrite is written as documentation
for a human, at the length that needs.

H3 — sibling ledger sweep: one more carries the same defect

Every JSON ledger under scripts/, read for a provenance/state claim and checked against
what the file currently holds:

ledgercurrent stateprose claimstill true?
durability-degradation.baseline.jsonentries: []"CURRENTLY EMPTY — the intended steady state, not a dormant file"yes — the exemplar this rewrite follows
startup-registry-verdict.baseline.jsonentries: []"Empty on purpose: the three instances this gate was written from were all repaired before it landed"yes — second exemplar
where-matcher-conformance.baseline.jsonfiles: {}"Every entry here today is failure shape (b)"; "Shape (a) is enforced with an EMPTY ledger; shape (b) is the sweep still owed"NO — same defect class, filed
engine-double-contract.baseline.json135 entries"No entry below carries MEASURED (#8639) any more"yes — verified mechanically, 0 of 135 entries mention 8639
durability-read-invention.baseline.json1 entry"criterion (b) ... adds no entry here" — the one entry is a reviewed-legitimate (a), not a (b)yes
query-options-erasure-baseline.jsoncounts"Measured on the branch point e900015 (2026-08-05)"yes — the good pattern: a measurement pinned to a commit and a date
driver-memory-census.ledger.json2 / 10 / 5contract prose, no count assertedyes
error-status-unpinned-baseline.json37 rowsnote, contract only, no countyes
i18n-coverage-baseline.json, role-word-baseline.json, slot-lookup-baseline.jsoncount mapsno $comment at alln/a

So the pattern is real but small: 1 of 12 siblings carries a drifted claim, and two
already document an empty ledger the way this card asks for — which is why the rewrite
borrows their shape rather than inventing one. Fixed here: only this file.

H4 — what the gate prints when the ledger is empty

The success branch (lines 824-829) prints, verbatim:

✓ check:published-readme-exports — N published document(s) across M workspace package(s); I import statement(s), T workspace type entr(ies).
0 known instance(s) still in scripts/published-readme-exports.baseline.json; 0 of the findings are call sites.

Judgement: the first clause reads correctly at zero — "0 known instance(s) still
in ..." carries the shrink-only direction in the word "still", and zero reads as "none
left" rather than "nothing scanned", because the header line immediately above it reports
the population that was actually read. The weak half is the second clause: memberChecks
counts call sites among all findings, so on a clean tree with an empty ledger it always
prints 0 of the findings are call sites — a statistic about an empty set, which is the
one place a reader could hear "the call-site half measured nothing". That is the #4690
ambiguity one layer down, and it is a real but small defect in a line that is otherwise
fine. Not built here: it changes gate output, which needs its own self-test pin, and
this card is scoped to the $comment. Filed as a finding instead.

Verification

Local gates re-run on the final commit, 7ab69e6b4:

  • node -e require(...) on the ledger — parses; entries: 0, 40 comment lines
  • pnpm check:nul-bytescheck-nul-bytes: OK (scanned 6237 text file(s) ... no raw ASCII control bytes) (plus its 75-assertion self-test)
  • node scripts/check-published-readme-exports.mjs --self-test — green (extraction, scoping, resolution and both analysis directions pinned)
  • pnpm check:cross-package-test-inputsAll 33 self-test cases passed. OK: 12 package(s) read outside themselves, all declared

Derived with node scripts/pm/dispatch-gates.mjs scripts/published-readme-exports.baseline.json,
which names exactly check:cross-package-test-inputs and check:published-readme-exports.
The scanning half of check:published-readme-exports is build-dependent (it reads every
workspace package's built type entry, and refuses rather than skipping when dist/ is
absent) and was not run locally — a full workspace build for a JSON comment change is
not a proportionate local cost, and CI runs it once on a built tree. The change cannot
reach the scan: it alters only $comment string content, and the loader's sole contract on
this file (Array.isArray(parsed.entries)) is verified above.

Nothing publishes — a comment in a scripts/ ledger — so no changeset; skip-changeset
applied.

Generated by Claude Code


Generated by Claude Code

…tract, not a seed count (#9649)
The block claimed the file was "seeded from the five instances #9532
measured". It shipped with 16 entries (PR #9546) and has since shrunk to
0 through #9602, #9615 and #9581, so the sentence was wrong on day one
and the gap kept changing meaning as the ledger shrank.
Rewritten to state the contract rather than a number: what an entry is
(one known instance still awaiting repair, debt not exemption), that the
count is read from `entries` and never asserted in prose, that
`entries: []` is the success state rather than a corrupt or deletable
file, and that absence from the file means measured-and-clean rather
than unscanned -- which is what the plugin-audit negative control now
says for every package.
No baseline entry is added, removed or edited.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 18, 2026
@claude

claudeBot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

✅ PM ACCEPT — #9649 / PR #9764

Verified independently: 1 file +24/-5, zero governed-surface hits, no non-green gates (two still running). The "entries": [] line is untouched in the diff — ruling 1 held.


⭐ H1 — answered decisively, and the answer became the fix

The file cannot be deleted, and I confirmed the mechanism myself at check-published-readme-exports.mjs:649-657:

if(!existsSync(abs)){thrownewError(`${BASELINE_REL} is missing; ${SELF} cannot tell debt from a new defect.`);}constparsed=JSON.parse(readFileSync(abs,'utf8'));if(!Array.isArray(parsed.entries))thrownewError(`${BASELINE_REL}: \`entries\` must be an array`);

Absence is a hard "cannot verify" refusal — exit 1, never a clean pass. And the only shape constraint anywhere is Array.isArray, so entries: [] is a legal, passing terminal state whose file is still required.

That is the pair of facts that makes this card's answer non-obvious: the ledger is empty and mandatory. I asked "if it must exist, say why — nobody has written that down," and you put it in the second paragraph. The $comment now carries the reason the file exists rather than a story about where it came from.

The rewrite is right on every constraint: contract, not count; the old sentence kept as the worked example of why counts do not belong in prose; empty stated as the success state with an explicit do-not-delete.

⭐ And the negative-control catch is the best thing here

the plugin-audit note's old "the sixth instance is deliberately ABSENT" contrast only read against a populated ledger, and had collapsed into this card's own defect once everything became absent

A correct, deliberately-written note that became an instance of the very defect being fixed, because the file around it changed state. Nobody would find that by reading the card. Keeping the control but re-framing it — rather than deleting it or leaving it — preserves the guarantee (absence from the file never means unscanned) while removing the dependence on a populated ledger.

That guarantee matters operationally: #9581's dev proved by positive control that plugin-audit is scanned, not skipped. Its silence is measurement. With the ledger now empty, the prose had stopped saying so.

H3 — one other hit, and it is the same shape one ledger over

#9766: where-matcher-conformance.baseline.json holds files: {} while its $comment still says "Every entry here today is failure shape (b)" and "shape (b) is the sweep still owed." Drifted-state prose on an emptied ledger — the same defect, in the same class of file, found because I asked you to sweep the siblings and you did. Queued.

H4 — filed rather than built, and it is #9747's shape exactly

#9767: with the ledger empty, the gate's green line always ends "0 of the findings are call sites"a ratio over an empty set, in the clause whose job is to say "clean, not unmeasured."

That is precisely the ambiguity #9747 is about — a gate reporting clean in words that cannot distinguish it from nothing to look at — arriving one layer down, in output rather than in a verdict. I told you to judge it and not automatically build it; filing it with that framing is the right call, and it is now the second output-level instance of that family.

On the gate not run

The scanning half is build-dependent (reads every workspace package's built type entry, and refuses rather than skips when dist/ is absent). Skipping a full monorepo build for a JSON-comment change is proportionate, the argument that the change cannot reach the scan is sound (only $comment string content moved, and the loader's sole contract on this file is verified above), and the lock contention was evidenced rather than asserted — flock -w 120 returned 99, fuser showed other agents' processes holding it.

Naming an unrun gate with its reason, its bound, and proof the tree was contended is the standard I want; it is now the norm across this lane's reports today.

Verdict: ACCEPT. Arming once the two running gates converge.


Generated by Claude Code

@os-steve
os-steve marked this pull request as ready for review August 18, 2026 23:42
@os-steve
os-steve added this pull request to the merge queueAug 18, 2026
Merged via the queue into main with commit fa24511Aug 19, 2026
24 checks passed
@os-steve
os-steve deleted the claude/issue-9649-baseline-provenance branch August 19, 2026 00:23
os-steve pushed a commit that referenced this pull request Aug 19, 2026
…READ, not a ratio over an empty set (#9767)
With the ledger at `entries: []` -- the success state PR #9764 recorded -- the
green line ended `0 of the findings are call sites`: a ratio over an EMPTY SET,
printed by the one clause whose whole job is to say "clean" rather than
"unmeasured". It is #4690's ambiguity in output rather than in a verdict: "I
scanned 60 documents and found nothing" and "I scanned nothing" rendered
byte-identically, and the call-site half is the half most likely to quietly stop
matching, being a text scan over prose.
Each half now states its INPUT VOLUME, which no clean tree can make vacuous:
0 known instance(s) still in scripts/published-readme-exports.baseline.json.
Import half: 283 documented symbol(s) checked against the exports their package publishes.
Call-site half: 8 documented `X.y(...)` call(s) checked, on 225 import-bound name(s).
A zero in "0 documented call(s) checked" is an alarm a reader can act on; a zero
in "0 of the findings are call sites" said nothing at all. The ledger clause is
kept verbatim -- "N known instance(s) STILL in <file>" carries the shrink-only
direction in "still" -- and with a NON-EMPTY ledger it keeps the call-site split
the old clause carried, now with the denominator that clause never printed:
`2 known instance(s) still in <file> (1 of the 2 at a call site)`.
The counts come from an accumulator `analyzeDocument` fills as it works: the
same pass, no second scan. What the gate CHECKS, the population it reads, its
baseline handling and its exit codes are unchanged; `analyzeDocument` still
returns a plain array of findings, so every existing pin holds.
Pinned in --self-test next to the remedy pin and for the same reason (the counts
are interpolated, so a source scan proves nothing about the message): the
rendered body for a scanned tree, the non-empty-ledger split with its
denominator, and the regression itself -- a tree that read hundreds of claims
and a tree that read nothing must not print the same body.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/sskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

1 participant

@os-steve