Found while implementing #7809 (PR #8106). Out of scope there, filed unassigned.
What
scripts/check-type-check-coverage.mjs carries a PINS_CHECKED invariant with a good rationale:
A @ts-expect-error outside every invoked tsc program is a retirement guard that cannot fail — deleting the directive changes nothing, which is the definition of a phantom check.
It detects that condition by looking for @ts-expect-error directives (pinFiles). That is one idiom. A compile-time pin written any other way is not seen by it.
The measured example
While writing a vocabulary pin for #7809 I first expressed it as a mutual-assignability pair — the ordinary way to assert "these two types are the same" without a directive:
const_forward: Declared=undefinedasunknownasEngineOp;// red if EngineOp widensconst_reverse: EngineOp=undefinedasunknownasDeclared;// red if EngineOp narrows
in packages/objectql/src/*.test.ts. packages/objectql/tsconfig.json excludes the test layer and its typecheck script is tsc --noEmit against that config, so the mandatory ablation (widen the union by one member, predict red) passed bothpnpm typecheck and vitest. The pin asserted nothing in either runner a developer runs locally, and PINS_CHECKED had nothing to say about it because it carries no @ts-expect-error.
What this is NOT
Worth stating precisely, because the first version of this analysis was wrong and the correction matters:
These files are not unchecked.check:type-check-coverage's TEST_DEBT ledger lifts the exclusion, compiles the test layer, and ratchets the raw error count (@objectstack/objectql: 355). Verified from both sides on PR #8106 — three type errors in a new test file made it report 358 and go red; fixing them returned it to exactly 355. So a widened union would have been caught in CI.
The gap is therefore about signal, not coverage:
- what fires is
TEST_DEBT +1 on a package with a 355-error pile, not "the dispatch vocabulary changed"; - it fires only in CI, not in the
pnpm typecheck the author just ran locally and watched pass; PINS_CHECKED, the mechanism whose whole job is to say "this pin is a phantom", stays silent — so the author gets no warning that the idiom they chose is not load-bearing where they put it.
Why it is worth recording
The repo already decided this class matters (#5286, #5449, #4642, the PINS_CHECKED invariant, and scripts/check-test-typecheck.mts, which packages/spec and packages/client have onboarded to). This is a narrow hole in the detector, not a new class: an author who avoids @ts-expect-error — as anyone asserting type equality rather than expected-failure naturally does — falls outside it.
Non-obvious, and the reason this is filed rather than fixed in passing: the naive repair (flag any type-only const in a hidden test file) would be noisy and wrong. Something narrower is probably right — e.g. treat a package's hidden test layer as a place where compile-time pins are reported as unenforced-by-the-local-runner, or extend PINS_CHECKED's notion of a pin, or simply document at the ledger entry that pins in these packages should be expressed at runtime. Which of those is correct is a judgement call, hence a finding rather than a patch.
Ancillary datum
24 of 71 package tsconfigs exclude the test layer; a keyword scan finds type-level assertion idioms (@ts-expect-error, satisfies, assignability consts) in test files across ~12 of those packages. Most are presumably covered as counts by TEST_DEBT exactly as objectql is — that number is offered as scope context for whoever picks this up, not as a claim that any of them is broken.
Generated by Claude Code
Found while implementing #7809 (PR #8106). Out of scope there, filed unassigned.
What
scripts/check-type-check-coverage.mjscarries aPINS_CHECKEDinvariant with a good rationale:It detects that condition by looking for
@ts-expect-errordirectives (pinFiles). That is one idiom. A compile-time pin written any other way is not seen by it.The measured example
While writing a vocabulary pin for #7809 I first expressed it as a mutual-assignability pair — the ordinary way to assert "these two types are the same" without a directive:
in
packages/objectql/src/*.test.ts.packages/objectql/tsconfig.jsonexcludes the test layer and itstypecheckscript istsc --noEmitagainst that config, so the mandatory ablation (widen the union by one member, predict red) passed bothpnpm typecheckandvitest. The pin asserted nothing in either runner a developer runs locally, andPINS_CHECKEDhad nothing to say about it because it carries no@ts-expect-error.What this is NOT
Worth stating precisely, because the first version of this analysis was wrong and the correction matters:
These files are not unchecked.
check:type-check-coverage'sTEST_DEBTledger lifts the exclusion, compiles the test layer, and ratchets the raw error count (@objectstack/objectql: 355). Verified from both sides on PR #8106 — three type errors in a new test file made it report 358 and go red; fixing them returned it to exactly 355. So a widened union would have been caught in CI.The gap is therefore about signal, not coverage:
TEST_DEBT +1on a package with a 355-error pile, not "the dispatch vocabulary changed";pnpm typecheckthe author just ran locally and watched pass;PINS_CHECKED, the mechanism whose whole job is to say "this pin is a phantom", stays silent — so the author gets no warning that the idiom they chose is not load-bearing where they put it.Why it is worth recording
The repo already decided this class matters (#5286, #5449, #4642, the
PINS_CHECKEDinvariant, andscripts/check-test-typecheck.mts, whichpackages/specandpackages/clienthave onboarded to). This is a narrow hole in the detector, not a new class: an author who avoids@ts-expect-error— as anyone asserting type equality rather than expected-failure naturally does — falls outside it.Non-obvious, and the reason this is filed rather than fixed in passing: the naive repair (flag any type-only const in a hidden test file) would be noisy and wrong. Something narrower is probably right — e.g. treat a package's hidden test layer as a place where compile-time pins are reported as unenforced-by-the-local-runner, or extend
PINS_CHECKED's notion of a pin, or simply document at the ledger entry that pins in these packages should be expressed at runtime. Which of those is correct is a judgement call, hence a finding rather than a patch.Ancillary datum
24 of 71 package tsconfigs exclude the test layer; a keyword scan finds type-level assertion idioms (
@ts-expect-error,satisfies, assignability consts) in test files across ~12 of those packages. Most are presumably covered as counts byTEST_DEBTexactly as objectql is — that number is offered as scope context for whoever picks this up, not as a claim that any of them is broken.Generated by Claude Code