Uh oh!
There was an error while loading. Please reload this page.
fix(docs-audit): exclude benchmarks and non-underscore test/ dirs from isTestFile - #12965
Conversation
…m isTestFile `isTestFile` covered `__tests__/` / `__mocks__/` / `__fixtures__/` and the `.test.` / `.spec.` infixes, but neither the non-underscore `test/` / `tests/` spelling of the same directory convention nor `*.bench.*`. Six packages use the non-underscore spelling, so `test/fixtures/*.ts` and benchmarks were reaching both call sites: the PHASE-1 change classifier, which treated them as implementation changes, and `walkSourceFiles`, which fed them into the `--bridge-coverage` ceiling population. Fixed as a predicate over the convention, not as a path list: two arms, plus self-test cases pinning the classes deliberately left admitted (`*.testkit.ts`, `*.fixture(s).ts`, `*.pin.ts`, `testkit/`) so the exclusion cannot widen by drift. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
📓 Docs Drift CheckNothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs. What this run could not see
Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
os-elon
commented
Aug 28, 2026
ACCEPT — armed ( Entry qualification: every check. 33 runs at ⭐ Scope held exactly. Both of my Zone 2 assumptions were falsified, and the second one inverts the result1. ⛔ The card's control does not reproduce, and that is a finding, not a nuisance. Today's ⇒ The published 2. "Verdict-neutral" was right for the convention route and wrong for the ceiling — and the difference is a correctness gain. The full The fix is over the convention, not over the two paths that exposed it
The population sweep carries both controls, which is what makes its numbers readings: the predicate was sliced verbatim out of the shipped source rather than re-implemented; a positive control agrees with a known test file and a known implementation file; and a deliberately over-wide stand-in predicate newly excludes 1906, proving the diff arithmetic can return a large non-zero. Result: old excludes 2844, new excludes 2872, newly excluded 28, newly admitted 0. Only 2 of the 28 declare a path, and they are exactly the 3 tails the ceiling lost. Ablation, one arm at a time, from the committed state, each mutation proved on disk by marker count before any result was read and each restore proved by state (
All 18 derived gates green (15 path-matched + 2 convention-triggered +
Generated by Claude Code |
Part of #11857 — this is the card's cheaper half (B) only. #11857 stays open: its half A
(the evidence-based registrar route) is a contract question and is not implemented here.
Scope
The card names one self-contained defect: "
isTestFileexcludes neithertest/fixtures/*.tsnor
*.bench.ts." That is what this PR fixes, as a predicate over the convention ratherthan as the two path literals that exposed it.
⛔ Not in this PR, deliberately: admitting any non-test
packages/**/*.tswhose maskedsource yields a route tail (half A). That redefines "registrar" to include
packages/specZodcontract declarations, and it moves a published figure other records cite. Every number half A
quotes — 45 to 59 client-bound rows, the storage ledger closing, 12 to 20 registrar files,
43 to 82 tails — belongs to half A and none of it is claimed here.
The control does NOT reproduce, and that is the first thing to read
The card's recipe requires reproducing 12 registrar files / 43 route tails on the unmodified
tree before any delta means anything. On
origin/mainatb1a987e4ait does not:a6eca9223)b1a987e4a)Cause, measured not guessed. Two files landed since
a6eca9223whose basenames containroutes, soREGISTRAR_FILE_REadmits them. Both are confirmed absent at the card's ref bygit cat-file -e:packages/spec/src/migrations/entries/retired-keys/18.kernel__Manifest__contributes.routes.tspackages/spec/src/migrations/entries/semantic/18.plugin-manifest-contributes-routes-retired.tsThey are ADR-0049 retirement entries, not registration sites. The exact split of the +1 tail and
+2 reachable between them was not measured — isolating it needs a probe outside this half's
dispatched scope. Filed as a separate
findingissue rather than pursued here.What the predicate now excludes
Two arms, both written over the convention:
.bench.joins.test./.spec.in the infix arm.test/andtests/directories join__tests__/__mocks__/__fixtures__— the same convention, spelled without underscores. Six packages use it, andtest/fixtures/falls out of it by shape rather than by being named.Swept across the tree with the predicate sliced verbatim out of the working-tree source (not
re-implemented), against 4907
packages/**/*.tsfiles:The 28 are 2 benchmarks and 26 files under a non-underscore
test/directory — 13 of thempackages/qa/dogfood/test/fixtures/*.ts, which is the card's first instance, reached by shape.Both controls fired in the same run: the slice agrees with a known test file and a known
implementation file, and a deliberately over-wide stand-in predicate newly excludes 1906 —
so the diff arithmetic is demonstrably able to return a large non-zero.
Did the live numbers move? Partly — and the published figure did not
The dispatching assumption was that this half is verdict-neutral on today's tree. That is
right for the convention route and wrong for the ceiling. Full
--bridge-coveragediff,before against after, is one line:
reachable 47, unreachable 172, and the whole per-ledger table — byte-identical.
The 3 ceiling tails removed are attributable in full, and they are exactly the two files the
card named as false positives:
packages/qa/dogfood/test/fixtures/endpoint-policy-fixture.ts/api/v1/apps/e8policy/public-notes,/api/v1/apps/e8policy/private-notes)packages/spec/src/benchmark.bench.ts/api/benchmark)No other file among the 28 declares a
path:. This is a correctness gain, not a loss: theceiling is meant to measure the headroom a better recognizer could reach, and a benchmark and a
test fixture were inflating it by 3 with paths no registrar serves.
The published
45is untouched by this PR. It reads 47 both before and after — this changemoves it by zero. Note for #10534 and #9572 separately that the figure they quote drifted 45 to
47 on
mainon its own, through the two migration entries above and not through anything here.packages/spec/src/conversions/registry.ts— the card's third false positive — is notexcluded by this predicate, which is the correct outcome: its cause is different (a
connector-action input inside an automation fixture, not a route), no
isTestFilechangereaches it, and it is not admitted today anyway.
Reverse verification
Both arms were ablated from the committed state, one at a time, each confirming the mutation
landed on disk by marker count before reading any result, each restored under a
trap ... EXIT INT TERMwith restoration proved bygit diff HEADempty plus a blob-hash matchagainst
HEAD(81bb2690a41c3798650213841db0e196e5a5b8c3), never by an exit code.benchalternative dropped from the infix armtest/+tests/directory arm deleted outrightThe load-bearing red is this one, which is the production failure mode itself:
The two walk fixtures are non-vacuous by construction, in the same way the existing #11866
block documents: neither carries a
.test./.spec.infix, so under the previous predicateboth were walked, and
stub-route.tsthen matchedREGISTRAR_FILE_REand let a test fixturecontribute production route tails.
Over-widening is pinned too
An exclusion that silently widens drops real implementation changes — the failure
selfTest'sown docblock names first, and
isTestFilefeeds the PHASE-1 change classifier as well as thewalk. So the classes deliberately left admitted are pinned admitted:
*.testkit.ts(8 files,shipped harness code, the same class as
packages/qa/src/testing.tswhich was already pinned asimplementation),
*.fixture.ts/*.fixtures.ts,*.pin.ts, thetestkit/andtesting/directories, and a bare
fixtures/ormocks/directory outsidetest/(zero populationtoday, so admitting it would be speculative reach). The reasoning is recorded in the docblock so
a later reader sees a decision rather than an oversight.
Changeset
Derived, not inherited: the diff is 1 file,
scripts/docs-audit/affected-docs.mjs, with zeropackages/**paths and no published-package change, so this PR releases nothing. Route taken isthe
skip-changesetlabel, whichpr-automation.ymlreads at lines 294 and 589 — not anempty-frontmatter changeset, which
check-empty-changeset.mjsrejects for new files.Gates — all run at
d3de352724, exit codes captured before any pipeDerived from the actual diff with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack(no STALE TREE; provenance line confirms commit
d3de352724). 15 path-matched plus 2convention-triggered for a gate-script edit; all 17 exit 0, plus this script's own
--self-test.Verdict lines, quoted from the gates themselves rather than from
$?:✓ affected-docs self-test: 474 cases pass.✓ check-self-test-wired: every one of the 143 script(s) CI runs that ship a --self-test has that self-test run by CI.✓ check-drift-comment: 56 cases pass across 5 fixture diff(s).check-nul-bytes: OK (scanned 7188 text file(s) ... no raw ASCII control bytes).check-affected-docsprints the79-tail ceilingline, confirming CI sees the same reading.This script has no separate vitest suite —
--self-testis its suite, andcheck-self-test-wired.mjsis the gate that enforces that wiring. Both are green above.Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
Generated by Claude Code
Generated by Claude Code