Measured while fixing #13053 (PR #13102), whose closing argument was that "an invalid fixture is how a later assertion gets written against a shape that cannot occur." That argument is sound, so the obvious follow-up is a guard. This card records what the guard would actually cost, because the obvious version of it is not viable and a narrower one is free.
Nothing here was built — this is a measurement, filed for triage.
The broad guard is not cheap
Proposal as it naturally reads: run the lint package's object fixtures through ObjectSchema.safeParse in an existing self-test.
Measured with the TypeScript compiler API over test files, taking every object literal that carries both name and fields, statically evaluating the fully-literal ones, and running ObjectSchema.safeParse on each:
| scope | fixtures found | not statically evaluable | evaluated | parse OK | WOULD GO RED |
|---|
packages/lint | 352 | 59 | 293 | 190 | 103 (across 31 files) |
| whole repo | 2789 | 511 | 2278 | 1440 | 838 (across 303 files) |
The top refusal reasons say why, and they are not defects:
156 unrecognized_keys @ fields.id
149 invalid_type @ fields
126 invalid_value @ fields.name.type
14 invalid_value @ fields.name.type (packages/lint alone)
13 unrecognized_keys @ (root)
These are lint-rule and rejection tests feeding deliberately malformed metadata, which is exactly what those rules exist to judge. A fixture that parses cleanly cannot exercise a rule that fires on malformed input. The repo has already adjudicated this class in another gate — check:query-options-erasure prints its own carve-out as "a rejection test must be able to build off-contract input" under the triage recorded there.
So the broad guard would red 838 fixtures that are invalid on purpose. It should not be built.
The 511 not-statically-evaluable fixtures are a second, independent cost: they are assembled from variables, helpers or spreads, so no source scan can see them at all. A guard covering those needs a runtime hook, not a static pass.
The narrow guard is free
The defect in #13053 was not "an invalid fixture". It was a carrier shape the schema refuses at a position no rule reads — invisible to the suite in both directions. That class has a one-line predicate, and it is much narrower than whole-fixture parsing:
a reference at a field-def key position must be a string.
Measured cost on today's main, over 5,593 files (excluding generated declaration artifacts where reference is a map KEY rather than a carrier: packages/spec/json-schema/**, packages/spec/liveness/**, and generated i18n translation maps):
| carrier kind at field-def position | count |
|---|
| string | 451 |
| object | 0 (1 before PR #13102) |
| array | 0 |
| other (non-literal) | 17, all legitimate |
The 17 non-literal ones decompose to zero defects: 6 Zod schema declarations in .zod.ts, 4 runtime pass-throughs in product code, 3 identifiers that are string constants, and 4 reference: null in solution-blueprint.test.ts, which is spec-legal there because SolutionBlueprintStrict declares z.string().nullable().
So this guard would go green on the tree as it stands and could land as a zero-baseline ratchet — no grandfathering file, no baseline to shrink. That is the cheapest possible moment to add one, and the whole class it closes is the class #13053 was an instance of.
Worth noting it would also need to tolerate the non-literal spellings above rather than flag them, so the predicate is "a reference whose value is a literal must be a string literal" — the 17 non-literal sites stay unjudged, exactly as the census treated them.
Suggested disposition
Triage the narrow guard as a possible small gate; drop the broad one. Recorded rather than built, per the scope of #13053.
Generated by Claude Code
Measured while fixing #13053 (PR #13102), whose closing argument was that "an invalid fixture is how a later assertion gets written against a shape that cannot occur." That argument is sound, so the obvious follow-up is a guard. This card records what the guard would actually cost, because the obvious version of it is not viable and a narrower one is free.
Nothing here was built — this is a measurement, filed for triage.
The broad guard is not cheap
Proposal as it naturally reads: run the lint package's object fixtures through
ObjectSchema.safeParsein an existing self-test.Measured with the TypeScript compiler API over test files, taking every object literal that carries both
nameandfields, statically evaluating the fully-literal ones, and runningObjectSchema.safeParseon each:packages/lintThe top refusal reasons say why, and they are not defects:
These are lint-rule and rejection tests feeding deliberately malformed metadata, which is exactly what those rules exist to judge. A fixture that parses cleanly cannot exercise a rule that fires on malformed input. The repo has already adjudicated this class in another gate —
check:query-options-erasureprints its own carve-out as "a rejection test must be able to build off-contract input" under the triage recorded there.So the broad guard would red 838 fixtures that are invalid on purpose. It should not be built.
The 511 not-statically-evaluable fixtures are a second, independent cost: they are assembled from variables, helpers or spreads, so no source scan can see them at all. A guard covering those needs a runtime hook, not a static pass.
The narrow guard is free
The defect in #13053 was not "an invalid fixture". It was a carrier shape the schema refuses at a position no rule reads — invisible to the suite in both directions. That class has a one-line predicate, and it is much narrower than whole-fixture parsing:
Measured cost on today's
main, over 5,593 files (excluding generated declaration artifacts wherereferenceis a map KEY rather than a carrier:packages/spec/json-schema/**,packages/spec/liveness/**, and generated i18n translation maps):The 17 non-literal ones decompose to zero defects: 6 Zod schema declarations in
.zod.ts, 4 runtime pass-throughs in product code, 3 identifiers that are string constants, and 4reference: nullinsolution-blueprint.test.ts, which is spec-legal there becauseSolutionBlueprintStrictdeclaresz.string().nullable().So this guard would go green on the tree as it stands and could land as a zero-baseline ratchet — no grandfathering file, no baseline to shrink. That is the cheapest possible moment to add one, and the whole class it closes is the class #13053 was an instance of.
Worth noting it would also need to tolerate the non-literal spellings above rather than flag them, so the predicate is "a
referencewhose value is a literal must be a string literal" — the 17 non-literal sites stay unjudged, exactly as the census treated them.Suggested disposition
Triage the narrow guard as a possible small gate; drop the broad one. Recorded rather than built, per the scope of #13053.
Generated by Claude Code