Skip to content

[finding] nothing reads a DEBT/TEST_DEBT note, and --lower rewrites only the digits — so the advertised way to lower an entry desynchronises its prose by construction #10722

Description

@claude

Filed from the #10174 dispatch (the note-vs-field consistency check triage explicitly held back as
"a new finding, not a rider"). Not fixed here — proposal plus the root cause and a live second instance.

The class

scripts/check-type-check-coverage.mjs freezes two ledgers, DEBT and TEST_DEBT. Each entry is a
measured errors: number plus a free-prose note:. The ratchet compares errors against tsc and
never reads the prose, so a note may narrate any count at all — including one its own field
contradicts — and every gate stays green.

Known instances: #7038 and #8982 (both closed), and #10174 (service-automation recorded errors: 3
while its note opened 'code-tier 5.', closed 'Re-measured 5 at 5ab08428.', and itemised 2 + 3).

Root cause — --lower manufactures this drift by construction

lowerLedgerEntries() at :2229 rewrites the digits and nothing else:

constpattern=newRegExp(`('${...}'\s*:\s*\{[\s\S]*?errors:\s*)(\d+)`);
out=out.slice(0,block.start)+text.replace(pattern,`$1${l.to}`)+out.slice(block.end);

The capture group ends at errors:\s*, so the replacement can only touch the number. The note is
outside it and is carried through untouched.

That is not an incidental gap. The gate recommends--lower in its own surplus output —
"Close it with pnpm check:type-check-debt --lower, which writes the measured number for you"
and the surrounding docblock at :160 sells it as the thing that makes closing an entry FREE. So the
sanctioned, advertised, one-command path to lowering an entry is also the path that silently
desynchronises that entry's prose. Every future --lower mints another instance.

A live second instance, measured tonight

DEBT['@objectstack/metadata'] at :411-417:

errors: 89,note: 'code-tier 34 (TS2345 x30, TS2322 x4); config-tier 24 (TS2835); noise 34 (TS7006 x33, TS6133). '+'Re-measured 92 at 5ab08428, up from 87. ...'

The itemisation sums to 34 + 24 + 34 = 92, and the note says "Re-measured 92". The field says
89. The note is internally consistent at 92 and disagrees with its field by 3 — the same shape as
#10174, a different entry, still live.

It is genuinely stale rather than merely narrating history: tonight's full --re-measure (33 entries,
built closure) reported exactly one surplus entry, @objectstack/plugin-auth, so @objectstack/metadata
measures its recorded 89 exactly. Compare TEST_DEBT['@objectstack/objectql'], which also contains an
older number ("Re-measured 333") but frames it explicitly as history and re-tallies to its recorded 355 —
that one is correct and any check must not flag it.

What would actually close the class

A general "does this prose agree with this number" check is not tractable, and should not be attempted.
A bounded one is, because these notes already follow house conventions:

  1. Tier itemisationscode-tier N, config-tier N, noise N. Where an entry states a complete
    tier breakdown, require the tiers to sum to errors. This alone catches both finding: DEBT['@objectstack/service-automation'] says errors: 3 while its own note narrates 5, naming two engine.test.ts errors tsc no longer reports #10174 and metadata.
  2. Per-code talliesTSxxxx xN runs, which should reconcile with their stated tier.
  3. Leave Re-measured N narratives alone, or require only that the most recent one match. objectql
    shows why a naive scan over every such number produces false positives.

The cheaper half is worth pricing separately: make --lowerrefuse to lower an entry whose note
carries a tier itemisation that would no longer sum, or have it append a marker to the note so the
staleness is visible in the diff instead of invisible. That turns the advertised path from a drift
generator into a prompt.

Not proposed

Extending the prose rule at :167-173. It already states the lesson and cites #10174's entry as its
worked example; it did not prevent this because nothing mechanical reads it. Adding more prose about
prose drift is the one fix this class has already shown does not work.


Generated by Claude Code


Generated by Claude Code

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions