Filed by the dev on #13173 (PR #13487), session session_01Pk26oZ12t5N1hwGW1m1MgC.
Filed unassigned and ungraded — this is a survey request, not a measured defect.
Why
#13173 recorded, under its own "Not established" heading, that nobody had checked
whether other --self-test implementations in scripts/ share the shape it
measured. That survey is still not done. #13173 itself is now closed out by
PR #13487, so the question would otherwise leave with it.
The shape to survey
Two distinct holes were measured in scripts/check-doc-authoring.mjs while
fixing #13173. A survey wants both, because the second is invisible to a check
for the first:
- No assertion floor. Success decided as
failures.length === 0, so a
battery that never ran is indistinguishable from a battery that passed. An
early return at the top of one battery function left it entirely unrun and
the script still exited 0, printing its full success line. main() doing return selfTest(). A return that leaves the self-test
before its verdict prints nothing at all and still exits 0. A gate can have
a perfectly good assertion floor and still be defeated this way, because the
floor never runs either.
The discriminator, cheaply
For each scripts/check-*.{mjs,mts} with a --self-test mode:
grep -n "failures.length === 0\|return selfTest()" scripts/check-*.mjs
⚠️ A verdict line that already prints a case count (872 cases pass and the
like) is evidence but not proof — #13173 warns that a merely non-zero count
still passes a battery dropping from 40 cases to 3, and a pinned TOTAL rots as
soon as a sibling battery grows.
The repair shape that worked, for whichever instances the survey finds
PR #13487 pinned the registered names, not a number: a frozen registry of
battery names each with its own case floor, every battery opening with
battery('...'), every assertion attributed to the battery most recently
opened, and a refusal when the opened set differs from the declared set. A set
difference names which battery stopped running; a count says only that
something did. The registry's own size is pinned too, since deleting an entry
silences its floor as effectively as zeroing it. Five ablations are recorded on
that PR.
Related
Generated by Claude Code
Filed by the dev on #13173 (PR #13487), session
session_01Pk26oZ12t5N1hwGW1m1MgC.Filed unassigned and ungraded — this is a survey request, not a measured defect.
Why
#13173 recorded, under its own "Not established" heading, that nobody had checked
whether other
--self-testimplementations inscripts/share the shape itmeasured. That survey is still not done. #13173 itself is now closed out by
PR #13487, so the question would otherwise leave with it.
The shape to survey
Two distinct holes were measured in
scripts/check-doc-authoring.mjswhilefixing #13173. A survey wants both, because the second is invisible to a check
for the first:
failures.length === 0, so abattery that never ran is indistinguishable from a battery that passed. An
early
returnat the top of one battery function left it entirely unrun andthe script still exited 0, printing its full success line.
main()doingreturn selfTest(). Areturnthat leaves the self-testbefore its verdict prints nothing at all and still exits 0. A gate can have
a perfectly good assertion floor and still be defeated this way, because the
floor never runs either.
The discriminator, cheaply
For each
scripts/check-*.{mjs,mts}with a--self-testmode:872 cases passand thelike) is evidence but not proof — #13173 warns that a merely non-zero count
still passes a battery dropping from 40 cases to 3, and a pinned TOTAL rots as
soon as a sibling battery grows.
The repair shape that worked, for whichever instances the survey finds
PR #13487 pinned the registered names, not a number: a frozen registry of
battery names each with its own case floor, every battery opening with
battery('...'), every assertion attributed to the battery most recentlyopened, and a refusal when the opened set differs from the declared set. A set
difference names which battery stopped running; a count says only that
something did. The registry's own size is pinned too, since deleting an entry
silences its floor as effectively as zeroing it. Five ablations are recorded on
that PR.
Related
check-doc-authoring.mjs --self-testexits 0 while registering ZERO cases — and prints its full success line asserting they all hold #13173 — the instance, in the gate that exists to catch prose claiming morethan the code delivers.
this is about a self-test asserting nothing. A gate can be clean under one
and defeated by the other.
scripts/check-dispatcher-error-vocabulary.mjs— the sibling that landed thenamed-set form for its own
SHAPES/samplespair.Generated by Claude Code