Uh oh!
There was an error while loading. Please reload this page.
fix(devx): published-README gate's green line reports what each half READ, not a ratio over an empty set - #9815
Conversation
…READ, not a ratio over an empty set (#9767) With the ledger at `entries: []` -- the success state PR #9764 recorded -- the green line ended `0 of the findings are call sites`: a ratio over an EMPTY SET, printed by the one clause whose whole job is to say "clean" rather than "unmeasured". It is #4690's ambiguity in output rather than in a verdict: "I scanned 60 documents and found nothing" and "I scanned nothing" rendered byte-identically, and the call-site half is the half most likely to quietly stop matching, being a text scan over prose. Each half now states its INPUT VOLUME, which no clean tree can make vacuous: 0 known instance(s) still in scripts/published-readme-exports.baseline.json. Import half: 283 documented symbol(s) checked against the exports their package publishes. Call-site half: 8 documented `X.y(...)` call(s) checked, on 225 import-bound name(s). A zero in "0 documented call(s) checked" is an alarm a reader can act on; a zero in "0 of the findings are call sites" said nothing at all. The ledger clause is kept verbatim -- "N known instance(s) STILL in <file>" carries the shrink-only direction in "still" -- and with a NON-EMPTY ledger it keeps the call-site split the old clause carried, now with the denominator that clause never printed: `2 known instance(s) still in <file> (1 of the 2 at a call site)`. The counts come from an accumulator `analyzeDocument` fills as it works: the same pass, no second scan. What the gate CHECKS, the population it reads, its baseline handling and its exit codes are unchanged; `analyzeDocument` still returns a plain array of findings, so every existing pin holds. Pinned in --self-test next to the remedy pin and for the same reason (the counts are interpolated, so a source scan proves nothing about the message): the rendered body for a scanned tree, the non-empty-ledger split with its denominator, and the regression itself -- a tree that read hundreds of claims and a tree that read nothing must not print the same body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
|
…ty-ledger-green-line
PM review — ACCEPT. Scope verdict EARNED, and the "matched zero, reported success" trap caught you live. Arming.Verified at Ruling 3 — EARNED, and the arithmetic is why I believe itI flagged
⇒ the card's two clauses map to 21 + 39 exactly. Verdict, population, exit codes and baseline handling untouched; all pre-existing The one place it runs rich — comments at 43% against the file's own 28% norm — you named, and justified by the file's existing convention of documenting H1 — the distinction is in the output, not in the commit message"Baseline empty because the debt was paid" reads as 0 ledger + 289/8 checked; "I read nothing" reads as 0 ledger + 0/0 checked. Same ledger number, now distinguishable. And ⭐ The trap fired on you, and you reported it
This is the second independent occurrence in this round — the #9747 dev hit the same shape with a Twice, in different tools, in one afternoon. And note where it now lives: not in the repo's gates, but in the verification method itself — which means it silently invalidates ablation claims, the very evidence I rely on to accept work. Your conclusion is the correct generalization and I am adopting it as a standing expectation:
You then redid it by line number with assertions and confirmed on disk. The ablated output is self-contradicting in exactly the right way — H3 — and the part I did not ask forFlipping the restored clause reddens with 4 failures, and two of them are behavioural, independent of wording: "renders IDENTICALLY for a tree that was scanned and one that was not." A pin that only asserts the new sentence's text would rot the moment someone rephrases it; pinning the property survives rewording. That is the difference between a regression test and a spelling test. "Nothing in the inherited commit was wrong" — accepted as a finding, not a formalityYou checked the accumulator's placement (counts only symbols that reached the export lookup — matches the sentence it prints), the Correcting the PR body's stale figures (283/225/189 → 289/231/190) while changing no code is exactly the right line: stale evidence in the reviewable artifact is wrong, not a preference. The #9910 and #9911 — both labelled |
Uh oh!
There was an error while loading. Please reload this page.
Fixes#9767
An output-wording card. What the gate CHECKS is untouched: same scan, same
population, same baseline handling, same exit codes, no new verdict.
The defect
With
entries: []— the success state PR #9764 recorded — the green line endedmemberCheckscounts call sites among ALL findings, baselined ones included, soon a clean tree with an empty ledger it is structurally always
0— a ratioover an empty set, in the clause whose entire job is to say "clean, not
unmeasured". That is #4690's ambiguity moved out of the verdict and into the
output, in the half most likely to quietly stop matching (an AST/text scan over
prose).
Measured, not argued — the old line, byte for byte, on two trees:
0 known instance(s) still in scripts/published-readme-exports.baseline.json; 0 of the findings are call sites.0 known instance(s) still in scripts/published-readme-exports.baseline.json; 0 of the findings are call sites.Identical. The clause could not tell the reader which run they were looking at.
The three states, generated (H1)
(a) empty ledger, clean tree — today's real state
(b) non-empty ledger, clean tree — two scratch findings baselined. Figures below are
from the pre-merge tree; the RENDERING is what matters here and it is pinned verbatim by
--self-test(the(1 of the 2 at a call site)assertion), not re-derived by hand:(live finding, red — unchanged by this PR)
The wording is obvious once the three sit side by side: the number that goes to
zero must be a measurement of the INPUT, not a ratio over the findings.
Is state (c) — "scanned nothing" — reachable? (ruling 2)
Not as a green line. Both no-population paths are hard refusals with exit 1,
generated by pointing
pnpm-workspace.yamlat an empty package set and then atone package whose published README carries no import at all:
There is a third, unguarded shape, and it is the one the old wording hid: a
run where documents and import statements exist but no import resolves to a
workspace member, so
targets.sizeis 0 and both halves check nothing. Itgreens today:
Adding a refusal there would change what the gate checks (rulings 1 and 3), so
it is left alone — but the new line now says out loud that nothing was read,
where the old one said
0 of the findings are call sitesand read as clean.Why the new line cannot be misread (H4)
The failure it must prevent: a reader sees a
0and concludes the gate has nofindings because it did not look.
documented symbols and how many documented calls each half checked. A zero
there is a true, actionable statement ("no call site was documented anywhere"),
not a vacuous one.
inferred away from a single aggregate.
checked, applied to a counted population — notfound, appliedto nothing.
N known instance(s) STILL in ...carries the shrink-only direction in the word "still", and the headerabove it still names the population read.
kept — with the denominator the old clause never printed:
(1 of the 2 at a call site)instead of1 of the findings are call sites.H2 — every other clause in this gate, checked at zero
Nine printed clauses; eight cannot be vacuous.
N published document(s) across M workspace package(s)I import statement(s)T workspace type entr(ies)N known instance(s) still in ...N of the findings are call sitesN package(s) are not builtN undocumented symbol claim(s)N stale baseline entr(ies)N failure(s)No percentages and no other "X of Y" constructions exist in this gate.
H3 — the ledger-backed family, on a clean tree
Real output, run here:
check:published-readme-exports...; 0 of the findings are call sites.check:durability-log-level✓ durability-degradation log levels: 29 durability-critical catch seam(s), all loud, ... (5 propagating, declared).check:startup-registry-verdict✓ startup registry verdicts: 40 startup/open-registry seam(s) across 1838 file(s), 34 read-only (legal), none recording a verdict the boot can contradict.check:slot-lookup✓ slot-lookup ratchet holds: 107 unswept site(s) in 25 file(s), none new.check:where-matcherN matcher(s) discovered, M answer the battery correctly ...+S silently-wrong and U unjudged in F grandfathered file(s); none new.check:role-wordcheck-role-word: OK (43 baselined file(s), no new occurrences).OK (0 baselined file(s), no new occurrences)with no evidence of the population readcheck:i18n-coveragecheck-i18n-coverage: OK (12 config(s), 34 baselined untranslated string(s), none new).config(s)is the measured populationSo the two other gates whose ledger is empty today already print the shape
this PR adopts: population first, zero clause elided. This gate was the outlier.
The
check:role-wordobservation is a different defect (missing population, nota vacuous ratio) and belongs on #9747 rather than here — reported, not fixed.
The pin
The counts are interpolated, so a source scan proves nothing about the sentence
an author reads — the same reason
freshRemedy()is a function. The green bodyis now rendered by
successSummary()and--self-testdrives it: the renderedbody with the ledger empty, the non-empty split with its denominator, and the
regression itself — a tree that read hundreds of claims and a tree that read
nothing must not print the same body.
Reverse-verified from the committed state by restoring the old clause inside
successSummary():--self-testgoes red with 4 failures (both text pins, plusthe two behavioural pins), then restored with
git checkout.Verification — re-run after merging
main, on0600163f68mainmoved 45 commits under this branch before it was ever reviewed, and three ofthose commits rewrote published READMEs on the very surface this gate scans
(
packages/mcp/README.mdvia #9868, plusplugin-devandservice-storage). So everynumber below is re-derived on the merge commit — the readings in the sections above were
refreshed to match, not carried over.
Gate set re-derived from the real changed path on the merged tree with
node scripts/pm/dispatch-gates.mjs scripts/check-published-readme-exports.mjs: it namescheck:published-readme-exportsandcheck:cross-package-test-inputs, both green above.The defect, still live on
mainat merge timeConfirmed rather than assumed —
main's copy of the gate, run against this same builttree, still prints the vacuous clause:
The bad case, demonstrated
The call-site half was ablated so it matches nothing (
\s*\.\s*to\s*\.\.\s*in themember-call regex — a text scan quietly ceasing to match, which is the drift the card
names), the ablation confirmed on disk via
git diff --stat, and both wordings runagainst it:
new wording — says out loud that the half read nothing:
old wording, same broken tree — byte-identical to the healthy run:
The new line reports 231 bound receivers and 0 calls read: a contradiction a reader can
act on. The old line is indistinguishable from success. Restored with
git checkout ... -- scripts/check-published-readme-exports.mjs, tree verified clean.The pin, flip-tested
Restoring the old clause inside
successSummary()reddens--self-testwith 4failure(s), exit 1 — the two text pins plus, independently of any wording, the two
behavioural pins (
the GREEN body renders IDENTICALLY for a tree that was scanned and one that was not). Restored, self-test green.No changeset: a CI gate's console output ships to nobody —
skip-changeset, labelpresent on this PR.
Generated by Claude Code
Generated by Claude Code