Skip to content

feat(gate): give the three pilot gates an UNRECOGNISED verdict distinct from clean - #9875

Merged
os-steve merged 3 commits into
mainfrom
claude/issue-9747-unrecognised-verdict-pilot
Aug 19, 2026
Merged

feat(gate): give the three pilot gates an UNRECOGNISED verdict distinct from clean#9875
os-steve merged 3 commits into
mainfrom
claude/issue-9747-unrecognised-verdict-pilot

Conversation

@os-steve

@os-steveos-steve commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Fixes#9747

Pilot only, visibility only. Three gates, no new required context, no new merge-blocking
failure — verified on the real tree, not asserted (see "Reverse verification" below).

H1 first: WHICH proposal, and why the other two halves are not here

Shipped: proposal 2 — the third exit state — on the pilot gates. Not proposal 1's
delta-ratchet, and not the deriver-wide unreachable sweep. All three reasons are
measurements or scope, not preference:

candidateverdict herewhy
proposal 2 (unrecognised as a verdict)shipped, 2 of 3 pilot rulesno in-flight work, and it is the half that is visibility-only by construction
proposal 1 (delta-ratchet) on check-engine-double-contractalready built and armed in PR #9712that PR enumerates the pinned population and adds RETAINED. Building a second one on the same file would collide head-on, and its churn case is already made
proposal 1 on the durability rulescarded, not builtit is a merge-blocking mechanism, which ruling 1 forbids on this card — and for one of the two rules my H2 numbers say it would be a nuisance. The numbers are below; they are the input the ruling asked for
the cheap unreachable sweep (#9700 / PR #9799)out of the pilot boundaryit is a hintCovers sweep over all 110 families in dispatch-gates. Ruling 2 says three gates, do not touch the other 107. Generalising before the nuisance measurement is exactly what the ruling forbids, so it stays a follow-on card that this measurement now unblocks

H2: the churn numbers for the two durability rules — the ruling's gate on generalizing

Measured the way #9680's dev measured, over 3,195 first-parent commits on main in the
month to 2026-08-19
, replaying each commit's changed files through the gate's own
recognizer (a copy of the gate file, proved byte-identical above its CLI epilogue by diff
at build time, so the numbers are the gate's own reading and not a re-implementation).

Result: the nuisance rate is a property of the POPULATION, not of the proposal.

log-level rule (DURABILITY_CRITICAL_CALLEES, all of packages/)read-seam rule (DRIVER_READ_CALLEES, 3 roots)
commits changing the set (key file)9 (0.28%)3 (0.09%)
commits changing the set (key file::callee)12 (0.38%)16 (0.50%)
commits changing the set (key file::fn::callee)12 (0.38%)28 (0.88%)
keys entering / leaving (file::callee)12 / 112 / 6
keys entering / leaving (file::fn::callee)12 / 125 / 15
legitimate decreases that would redden CI, per month0 net, 1 per-file4 to 10, depending on ledger key
  • The log-level rule reproduces check:engine-double-contract has no ratchet on its DISCOVERED population — deleting a pinned double's delete() member takes 319 pinned to 318 and the gate stays green #9680's zero. Its single leave in a month —
    packages/runtime/src/http-dispatcher.ts::saveMetaItem at 8891f9394 — is a cross-file
    move
    : the seam left http-dispatcher.ts and entered domains/meta.ts in the same
    commit, repo-wide total 1 -> 1. A per-file identity ledger reddens once a month on a
    file split; a repo-wide delta does not move at all.
  • The read-seam rule does not. Its leaves are real removals, and their commit subjects
    say so: "remove the unreachable legacy raw-engine save path" (5ab084286, -3),
    "retire sys_fetch_previous_delete" (4fedb1179, -1), "four read seams that failed no
    longer answer from ..."
    (4e3a4c3c8, -1). At the finest keying that is one red every
    other working day, every one of them correct code being reddened.

So the answer to "can this be generalized?" is: not blindly, and the discriminator is
measurable per gate before adopting it.
A delta-ratchet is free where the population only
grows and is a nuisance where correct deletions are routine — and the same gate family
contains one of each.

⚠️ One measurement-integrity note, filed separately: PR #9712 reports 269 commits on
main for the same month. The first-parent count is 3,195. This container's clone
arrives shallow at 63 commits and needs git fetch --shallow-since before any history
question can be asked — which is this card's own shape, one layer out: an instrument
reporting a small number because it could only see part of the population.

H3: per-gate verdict — can a construct be counted as unmatched at all?

pilot ruleverdictwhat it counts
check-engine-double-contractYES23 constructs today, 117 scoped out
durability log-level ruleYES0 today, on a closed structural vocabulary
durability read-seam ruleNO — and it says so in its own outputsee below

check-engine-double-contract. Its discovery has a well-defined stopping point:
implOf answers null and the construct leaves the population before isEngineVerbShape is
ever asked. Not "declared out of scope" — absent, which is #8639's shape and which the
file's own header already names as "the DISCOVERED invariant's blind half, one layer
down"
. The census walks the same structural evidence discovery uses (the slice's verb
declared alongside at least two engine siblings) and reports what discovery could not read.

Durability log-level rule. Its log-callee vocabulary is closed and structural, and
resolveLogCallee already answers unreadable for a call it cannot read. Before this,
that answer was collected only to choose a FINDING's verdict, so a seam that was correctly
green and also carried something unreadable printed nothing anywhere. Now counted.

Durability read-seam rule — proposal 2 does not apply, and the gate now prints that
rather than a number.
Measured before deciding: a census over its three scan roots asking
"which catches carry this rule's harm shape while guarding a call the vocabulary does not
name?"
returns 25 sites, and the callee histogram is Array.isArray (5), .raw (3), a
callback fn (3), JSON.parse (3), getService (2), getDriver (2), toJSONSchema,
stringify ... Most are not storage reads at all. Narrowing that to the genuine ones needs
exactly the name-heuristic the rule's own vocabulary note refuses for a measured reason. So
the honest output is NOT APPLICABLE with the reason — printed rather than omitted,
because an absent row cannot be told apart from "nobody looked".

The H4 trap, handled: SCOPED OUT is not UNRECOGNISED

#8662 is why this matters: check-where-matcher-conformance drops inverted survivor filters
as OUT_OF_SCOPEcorrectly, by its own definition, and that correct verdict still reads
as "nothing to see". A census that folded such constructs in would report noise on day one.

On the engine gate the split is 117 scoped out vs 23 unrecognised, and the criterion is
structural rather than an allowlist of callee names:

  • SCOPED OUT — no implementation exists anywhere, so nothing could be looser than the
    producer: delete: vi.fn(), delete: vi.fn().mockResolvedValue(v). unwrapCallImpl's own
    census already argues exactly this. Counted as a number only.
  • UNRECOGNISED — an implementation demonstrably exists and this gate cannot reach it:
    the initializer carries a function the unwrap declined, or it roots at a binding the file
    declares, or the member is a shorthand. Reported by file, line and spelling.

Two self-test limbs pin the scoped-out side in both spellings, one pins that a construct with
too few engine siblings lands in neither bucket, and one pins that a census row is still
absent from the population — so "it only counts" is a property, not a claim.

Reverse verification: both directions, on the real tree

Direction predicted before each run; all observed as predicted. Every ablation restored from
a commit, never from an uncommitted tree.

1. The engine census fires, and the rows are real doubles. Ablate
packages/rest/src/rest-batch-endpoint.test.ts:40 by removing only the ?? default, so the
gate can read the same implementation:

before ablationafter
UNRECOGNISED [engine-double-contract]2322
gate verdictOK — 321 pinned, 133 in the DEBT ledger, 2 exempt. exit 0exit 1, PINNED [delete]: packages/rest/src/rest-batch-endpoint.test.ts declares 1 engine double(s) whose delete() does not route through assertEngineDeleteDispatch (line 36)

That is the load-bearing result. The 23 rows are not decoration: the moment the gate can
read one, it has a real finding about it. A ??-defaulted mock was hiding a genuinely
unguarded engine double from a shrink-only ratchet, silently, at exit 0.

2. The durability census fires, and does NOT redden. Inject one unreadable log call into
a real loud seam (packages/metadata/src/loaders/database-loader.ts:394):

UNRECOGNISED [durability-degradation-log-level]: 1 of 29 discovered seam(s) ...
unrecognised packages/metadata/src/loaders/database-loader.ts:394 guards syncSchema() could not read: console[lvl](...)
✓ durability-degradation log levels: 29 durability-critical catch seam(s), all loud, ...
EXIT=0

Both halves at once: the count moved 0 -> 1 naming the exact construct, the sub-count
"verdict rests entirely on it" correctly stayed 0 (the seam is loud on other evidence),
and the run stayed green. Visibility-only, demonstrated rather than asserted.

3. Each new self-test limb proven failable. Every one of the four durability census
expectations flipped in turn — all four reddened, one limb each. Then the production sink
mutated (collectLoggedLevels stops recording unreadable): 4 limbs red, the two new
census limbs plus the two pre-existing unreadable-report limbs, exactly as predicted.

4. One honest miss, recorded because it is this card's own shape. The first insertion of
those four fixtures used a str.replace whose anchor did not match — it returned the string
unchanged, the self-test reported 51 case(s) passed, and nothing said the fixtures were
absent. Caught only by the flip-test above. A "matched zero, reported success" operation, in
the very change that exists to make such operations announce themselves.

Why this is not exit 2, when the in-tree prior art is

Ruling 4 says match the convention; ruling 1 says nothing may redden a PR for an
unrecognised count. Those pull in opposite directions on the exit code specifically, so
the resolution is stated in both file headers rather than picked silently:

The four in-tree implementations — check-where-matcher-conformance (missing baseline
=> exit 2, explicitly distinct from a finding's exit 1), check-published-readme-exports
(hard refusal, "cannot tell debt from a new defect"), check-governed-merges' header
("non-zero exits classify the ENVIRONMENT, not the tree") and PR #9799's drift guard — are
all the gate refusing to run: the environment is broken and no verdict about the tree
exists. This verdict is the opposite: the run completed and the count is an observation
about it. exit 2 would make it a failing CI job.

So the convention is matched where it is about semantics (a named third state, printed
rather than inferred, distinct from both clean and a finding) and deliberately not where
it is about the exit code.

Proposed round-report wording — for a maintainer, NOT applied here

The ruling says the counts should be "visible in round reports". That template lives under
.claude/skills/pm-dispatch/**, which is a governed surface this PR may not touch. Both
gates therefore emit a stable, greppable prefix so the template can pick them up with no new
context existing anywhere:

UNRECOGNISED [engine-double-contract]: ...

Suggested one-line addition to the round-report template, for the maintainer to apply or
reject:

Unrecognised:grep -h '^UNRECOGNISED \[' over the round's gate logs — one row per
pilot gate. A row reading NOT APPLICABLE is a measured verdict, not a missing number; a
count that GROWS between rounds is a recognizer falling behind the codebase.

Scope

Verification

Local gate union re-run on the final commit 372d93284, all green, derived from the actual
changed paths with node scripts/pm/dispatch-gates.mjs after merging main in (not
recalled, and re-derived post-merge):

check:durability-log-level · check:engine-double-contract · check:cross-package-test-inputs
· check:nul-bytes

Control-byte sweep run separately over both edited files
(grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'): clean.

Out of scope, filed not fixed

Generated by Claude Code


Generated by Claude Code

…rom clean
Fixes#9747
Visibility only, per the maintainer ruling of 2026-08-18: printed and counted,
never a failure, no new merge-blocking context.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
…ability rule
Four fixtures, each proven to discriminate by flipping its expectation, and the
production `unreadable` sink mutated to prove they fail when the census goes
blind.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
@claude

claudeBot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

PM review — ACCEPT. All three open questions ruled A. This is the best-executed card of the round.

Verified at 372d93284: 2 files, +470/-3, GOVERNED_HITS=NONE, no non-green gates. Arming.

H1 — you shipped less than the card allowed, and each subtraction is a measurement

Proposal 2 only, and all three reasons hold:

  • (a) collision avoided — proposal 1 for check-engine-double-contract is already built and armed in PR feat(gate): enumerate check:engine-double-contract's pinned population so a lost pin names itself #9712, on the same file. Building a second would have been a head-on conflict. You checked what was in flight before writing; almost nothing checks that.
  • (b) merge-blocking, which ruling 1 forbids — and your own H2 numbers say it would be a nuisance on one of the two rules. Carding it with the numbers instead of building it is exactly right.
  • (c) outside ruling 2's three-gate pilot — the deriver-wide sweep touches all 110 families, and the ruling says generalization waits for this measurement.

A dev that ships three-quarters of a ruling and can name the measurement behind each omission is more useful than one that ships all of it.

H2 — the finding I did not expect, and it changes the family answer

The nuisance rate is a property of the POPULATION, not of the proposal.

And the same gate family contains one of each:

rulekeys enter / leavenet legitimate decreasesdelta-ratchet verdict
durability log-level12 / 1(and that 1 is a cross-file move — repo-wide 1 → 1)0 per monthfree
durability read-seam25 / 15 (file+fn+callee)10 per monthone red every 2–7 working days

That kills "generalize the ratchet across the family" as a single decision and replaces it with "measure per gate, and the measurement is cheap and reusable." I came into this card expecting a family-wide answer. The correct answer is that there isn't one, and you proved it with a method (per-commit replay of changed files through the gate's own recognizer, proved byte-identical above its CLI epilogue) rather than an argument.

That the deletions are genuine'remove the unreachable legacy raw-engine save path' (−3), 'retire sys_fetch_previous_delete' (−1) — is what makes the read-seam number a nuisance rate rather than a defect rate. Reading the commit subjects instead of only the counts is the difference between the two.

H4 — the ablation found a live defect, and that is the load-bearing result

Removing only the ?? default at packages/rest/src/rest-batch-endpoint.test.ts:40 made the implementation readable, and the gate went exit 0 → exit 1 with a real PINNED [delete] error.

⇒ A ??-defaulted mock was hiding a genuinely unguarded engine delete() double from a shrink-only ratchet, at exit 0. Not a hypothetical blind spot: a real finding the gate already had an opinion about and could not see. That is the strongest possible evidence for this card's whole premise, and it arrived from an ablation rather than a claim. Filed as #9877 — correctly not fixed here, since widening is the separately-declined act and all 23 constructs would arrive unpinned.

The H4 trap is handled: scoped-out (117) is a separate printed number with a stated criterion, never folded into the unrecognised count (23). That was the failure I was most worried about.

H3 — NOT APPLICABLE is the answer I most wanted to see

For the read-seam rule you measured before deciding: 25 sites, histogram Array.isArray (5), .raw (3), callback fn (3), JSON.parse (3), getService (2)… most not storage reads at all ⇒ ~20 rows of noise on day one, the #8662 failure this pilot exists to avoid. Narrowing needs exactly the name-heuristic the rule's vocabulary note refuses.

And then the part that makes it right:

NOT APPLICABLE is printed rather than omitted, because an absent row cannot be told apart from "nobody looked."

That single sentence is the thesis of #9747, arrived at independently and applied against your own work. A gate declining to answer, out loud, is the fix — not a gate answering badly.

⭐ The self-referential catch — I want this in the permanent record

The FIRST insertion of those four fixtures used a str.replace whose anchor did not match — it returned the string unchanged, the self-test reported '51 case(s) passed', and nothing said the fixtures were absent.

A "matched zero, reported success" operation, caught only by the flip-test, inside the very change that exists to make such operations announce themselves.

You could have deleted that from the report and nobody would have known. Reporting it is worth more than the fix: it is a live demonstration that the class is not a property of these gates but of any operation whose null result looks like its success result — including the ones we write while fixing it. It is also the argument for flip-testing every new assertion, which no amount of review catches.

Rulings

Q1 → A. Ruling 1 is unconditional on this card and binds over a favourable measurement. Card the log-level delta-ratchet separately, with the numbers. A pilot that quietly widens its own scope when the data looks good is not a pilot. The follow-on starts priced — which is what the pilot was for.

Q2 → A. I own the template edit. .claude/skills/pm-dispatch/** is governed and you were right not to touch it. I will add the one-line grep for the UNRECOGNISED prefix. Your point that a row reading NOT APPLICABLE is the part worth having is the reason it goes in the template at all — it is what stops "no number" being read as "nothing found".

Q3 → A. Card the deriver-wide unreachable sweep now, gated on these numbers. B would break ruling 2 explicitly. And your reading is right: the measurement argues for the sweep (no per-gate knowledge needed) and against blanket delta-ratchets — those are different generalizations and only one of them survived contact with the data.

#9878 — acted on immediately

Your shallow-clone finding is confirmed from this seat (is-shallow-repository → true; deepened --first-parent --since=2026-07-193199 vs your 3195). PR #9712's shipped code comment carries the wrong 269. I have recorded it on that PR and scoped the correction into #9878 rather than blocking a green queued 2,043-line change over a comment — the conclusion survives, because you reproduced #9680's zero over the full 3,195.

Both cards are now pm:queue / domain:devx.


Generated by Claude Code

@os-steve
os-steve marked this pull request as ready for review August 19, 2026 06:09
@os-steve
os-steve enabled auto-merge August 19, 2026 06:09
@os-steve
os-steve added this pull request to the merge queueAug 19, 2026
Merged via the queue into main with commit 460d7aaAug 19, 2026
30 checks passed
@os-steve
os-steve deleted the claude/issue-9747-unrecognised-verdict-pilot branch August 19, 2026 06:22
os-steve pushed a commit that referenced this pull request Aug 19, 2026
…th halves
`main` (#9875/#9747) added the UNRECOGNISED census; this branch (#9680) added
the pinned-population enumeration and RETAINED. Both are additive to one
script and neither supersedes the other, so both survive verbatim.
Two things the three-stage text merge did not handle on its own:
1. `report()` ended up with two `const census` bindings in one scope — the
PINNED census destructured from `audit()` (this branch) and
`censusUnrecognised()` (`main`). git auto-merged that region with NO
conflict marker and the result did not parse. `main`'s local is renamed to
`unrecognised`; every printed string is byte-identical.
2. `main` added a test file carrying 2 new pinned doubles, which RETAINED
correctly reports as growth. Regenerated with the gate's own
`--write` remedy: 308 -> 310 rows, 2 added or grown, 0 lost.
Gate output after the merge, both halves firing:
UNRECOGNISED [engine-double-contract]: 23 construct(s) ... 117 further
construct(s) are SCOPED OUT by a stated criterion ...
check-engine-double-contract: OK — 321 pinned, 133 in the DEBT ledger, 2 exempt.
check-engine-double-contract: 310 (file, verb) row(s) held by the RETAINED ledger
`--self-test` runs 107 assertions = 76 (merge-base) + 8 (`main`) + 23 (this
branch); no limb from either side was dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Seven gates report "clean" when they mean "I saw nothing I understood" — make unrecognised a verdict distinct from pass

2 participants

@os-steve@claude