Skip to content

Seven gates report "clean" when they mean "I saw nothing I understood" — make unrecognised a verdict distinct from pass #9747

Description

@claude

Filed by the PM seat as the Q4 ruling on #9165, which measured this shape rather than assuming it. This is a design card, not a build: the remedy below is a change to how the whole patrol family reports, and that is a maintainer act.

The meta-shape

A gate's recognizer is narrower than the code shapes in the repo, and the shortfall is reported as a VERDICT rather than as "unrecognised."

Seven measured instances, all from this repo, all found within about a day:

Fails toward FALSE RED — self-announcing, cost is a wasted branch

#gatethe shape it cannot see
#8897check-durability-degradation-log-levelreceiver name
#9657sameparenthesized callee — (logger.error ?? logger.warn)(…) reads as silent-swallow

These are survivable: someone hits one, files a card, and the truth surfaces. ⚠️ The exception is #9657, whose cheapest satisfaction is actively harmful — the spelling the matcher does accept (logger.error?.(…)) prints nothing against a sink with no error, so "fixing" the false red converts a loud site into a genuinely silent one.

Fails toward FALSE GREEN — never self-announcing, found only by luck

#gatethe shape it cannot see
#8845check-durability-log-levelexit shape (the criterion half)
#9165 AsamepublishPackageDrafts — in the census, reported "no invented answer", while its catch pushed a fabricated existedBefore: false
#9165 BsamegetAllObjects — never in the population at all; DRIVER_READ_CALLEES is {find, findOne, count}
#9680check-engine-double-contractabsence — deleting a pinned double's delete() member took 319 → 318 with exit 0; discovery requires the member to exist
#9708sameconsumer seams — deleting one took 6 in 3 source file(s) → 5 with exit 0; SEAMS_DISCOVERED fires only at zero

This half is the problem.#9165's entire complaint is that both of its instances were discovered "because somebody removed the swallow for an unrelated reason" — a discovery mechanism that fires by luck, and the luck was spent twice in one shift.

Why the obvious remedy is wrong

"Widen each matcher" has been separately priced and separately declined every time:

Widening is expensive, per-gate, and leaves the next unrecognised shape in the same place.

The proposal

Make "I could not recognise this shape" a reportable verdict, distinct from "clean."

The germ already exists in this repo, in check-engine-double-contract's DISCOVERED invariant, whose own comment says it best:

Zero is not a clean repo, it is a broken scan.

The generalisation is from zero to delta — and #9680's dev has now demonstrated it works in practice for one population, with the churn numbers to price it: 269 commits on main in a month, pinned-set membership changed in 7 commits (2.6%), 8 files entered per verb, 0 left. The feared nuisance case (a legitimate decrease reddening CI and training everyone to bump a number) fired 0 times in a month.

What that suggests, for the maintainer to accept or reject:

  1. Every gate with a discovered population declares it, and reports a shrink as its own verdict — not silence, not necessarily red.
  2. A third exit state beside pass/fail: "N constructs in the scan roots matched no rule in this gate's vocabulary." Not a failure. But printed, counted, and visible in a round report, so a recognizer falling behind the codebase is observable instead of inferable.
  3. The distinction that matters: a gate saying clean when it means I saw nothing I understood is the single mechanism behind every false-green row above.

What this card is NOT asking for

Related, each an instance rather than a duplicate

#8897 · #9657 · #8845 · #9165 (the disposition comment carries the full measurement) · #9680 / PR #9712 · #9708 · #8901 (on hold; its restart-when is measured not met — the original cohort is shrinking 25 → 19, not gaining a second)


Generated by Claude Code

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions