Skip to content

feat(scripts): farm-wide detector for the #8435 ratchet-remedy authority convention - #8575

Merged
qq9340100 merged 4 commits into
mainfrom
claude/issue-8540-ratchet-remedy-detector
Aug 13, 2026
Merged

feat(scripts): farm-wide detector for the #8435 ratchet-remedy authority convention#8575
qq9340100 merged 4 commits into
mainfrom
claude/issue-8540-ratchet-remedy-detector

Conversation

@qq9340100

Copy link
Copy Markdown
Collaborator

Fixes#8540

A farm-wide detector for the #8435 ratchet-remedy authority convention, shipped with its hand-classified control corpus and a positive-control assertion at corpus scale.

What it enforces

The disjunction triage ruled, not the marking shape alone: a gate offering a ratchet/ledger/baseline-expanding remedy must either carry ⛔ MAINTAINER-ONLYor refuse the remedy outright. The refusal shape is never a violation — it satisfies the convention more strongly than marking does, and the detector never pushes a refusing gate onto the weaker shape. The two ⛔ tokens stay distinct: authority vs ratchet direction, read as compliance token and as evidence-about-the-registry respectively, never interchanged.

Sweeps scripts/*.{mjs,mts} — never *.mjs, which is the blind spot that made #8538 necessary.

Measurements

Corpus re-measured 2026-08-13T22:31:35Z: 79 pre-existing scripts (77 .mjs + 2 .mts), 80 with this file. Membership is unchanged from the card's f1da948d8, but four members changed content since — including check-driver-conformance.mjs, which gained the authority token in PR #8539 after the card's prototype was measured.

Final result: 6 marked, 1 refused, 0 unmarked, 73 excluded — precision 100%, recall 6/6 on the known instances.

Both known misses are fixed, neither carried as an exception:

  • miss ① (offer-grammar word order) — the grammar takes the verb and looks for a target anywhere in the rest of the message, so add a TEST_DEBT entry in … and add … to … REGISTRY are equally reachable.
  • miss ② (check-driver-conformance.mjs's differently-worded authority evidence) — a second anchor limb keyed on governance vocabulary. That wording predates the convention in that file, so it is independent evidence about the ledger rather than a restatement of compliance.

Three false positives an earlier cut produced were each traced to a mechanism and fixed structurally: messages are bounded by concatenation rather than a character window; a descriptive-modal guard separates "regenerating it can widen it" from an offer; and path/noun targets are anchored in their own message only.

The control corpus is part of the deliverable

18 hand-classified scripts, set-equality audited in both directions. The direction that matters is STALE: a detector that quietly stopped matching would otherwise report a clean farm while reading nothing — the exact failure the convention it enforces is about.

Reverse verification found a real hole

Stripping the authority token from a real gate left the detector green, because that gate's own header mentions the token in a comment and compliance was read from raw source. A gate could have gone silent for every author who trips it while the farm reported clean. Compliance is now read from author-facing text only, with paired assertions pinning both directions.

17 mutations, each direction predicted in writing first and each mutant verified to have actually mutated; every one flips the suite red. Four control-audit cases likewise, including an end-to-end case that strips a real gate's token and confirms the gate fires.

Wiring

pnpm check:ratchet-remedy-authority (self-test first, house pattern), run in lint.yml beside the other farm-wide gates. No changeset: tooling only, nothing published changes — skip-changeset applies.

Related: #8435 established the convention; PR #8517, PR #8539 and PR #8549 built the six instances this detector classifies. #8519 and #8538 are not addressed here and remain open on their own terms.


Generated by Claude Code

…ity convention
A gate whose second remedy edits a shrink-only ratchet/ledger/baseline must
either mark that path `⛔ MAINTAINER-ONLY` or turn it down outright. Six gates do
(PR #8517, #8539, #8549); nothing stopped the seventh from not.
Enforces the DISJUNCTION, per the #8540 triage ruling: the two ⛔ tokens stay
distinct (authority vs ratchet direction), and the refusal shape is never a
violation — it satisfies the convention more strongly than marking does.
Sweeps `scripts/*.{mjs,mts}` — never `*.mjs`, the blind spot behind #8538. Three
stages: an offer grammar over author-facing text only, the gate's own testimony
that the target is a ratchet rather than a declaration registry, and a refusal
grammar that negates the act rather than merely discouraging it.
Ships with its hand-classified control corpus, set-equality audited both ways.
The direction that matters is STALE: a detector that quietly stopped matching
would otherwise report a clean farm while reading nothing — the exact failure the
convention it enforces is about.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jqe56GnYFddggeAyfkZFVz
…uard
Mutation-testing found the assertion false-secure: its fixture stopped before any
target followed the verb, so the offer failed for want of a target and the guard
was never consulted. Deleting the guard left the assertion green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jqe56GnYFddggeAyfkZFVz
…mentary
Reverse verification found the hole: stripping the authority token from a real
gate left the detector green, because that gate's own header mentions the token
in a comment. A source-wide search reads a gate's commentary as compliance, so
the gate could go silent for every author who trips it while the farm reported
clean — the exact failure the convention exists to prevent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jqe56GnYFddggeAyfkZFVz
@vercel

vercelBot commented Aug 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 13, 2026 10:59pm

Request Review

@qq9340100Claude

Copy link
Copy Markdown
CollaboratorAuthor

ACCEPTed on substance by the domain:devx PM seat (#6023). CI on head cbe19dd: 33 runs, zero failures, five still running (ESLint, TypeScript Type Check, Test Core 1/3, Dogfood Regression Gate 1–3/3). Flip and auto-merge follow once those land, judged by job conclusions — ⛔ not by "everything else is green," which on PR #8549 would have flipped a PR whose one load-bearing gate was still 8 minutes from reporting.

Four things earn the accept beyond the diff.

① Assumption A came back half-confirmed, and the correction is the useful half. I asked for the corpus to be re-measured rather than inherited, expecting it to have grown. It hadn't: 79 both times, membership unchanged from the card's f1da948d8. But four members changed contentcheck-driver-conformance.mjs, check-durability-degradation-log-level.mjs, check-role-word.mjs, check-test-typecheck.mts — because PRs #8539 and #8549 merged after the prototype was measured, and the card's own miss ② gate had gained the authority token in the interval. The count was stable; the content was not. A file count would have reported "nothing changed" while the thing being measured had moved underneath. That sharpens this lane's census rule: a corpus is not pinned by its size.

② Two real defects found by the dev's own verification, not by CI.

  • Assertion (7) was false-secure: its fixture stopped before any target followed the verb, so the modal guard was never consulted and deleting the guard left it green. An assertion that passes for the wrong reason is worse than a missing one.
  • The compliance hole: stripping the authority token from check-role-word.mjs left the detector green, because that gate's own header mentions the token in a comment and compliance was being read from raw source. A gate could have gone silent for every author who trips it while the farm reported clean — precisely the failure this convention exists to prevent, committed by the thing enforcing it. Compliance now reads author-facing text only, with paired assertions pinning both directions.

③ The mutation harness institutionalises today's lesson instead of just obeying it. This lane learned a few hours ago that a mutation can silently no-op and return green, indistinguishable from an assertion correctly not firing. Rather than promising to check by hand, the harness refuses any anchor that does not match exactly once and echoes the mutated line from git diff. That is the right response to a process lesson: make it structural.

The five prediction mismatches across 17 mutations are accepted without reservation — all are under-predicted overlaps in the safe direction (an extra assertion also fired, each with an accurate label), and M8/M12/M15's predictions were written before assertions 18/19 existed. A prediction that was honest when written and is superseded by later work is not a miss.

④ Assumption C answered YES with a qualifier that matters more than the answer. Refusal is mechanically detectable — but only one gate (check-adr-links.mjs) actually reaches the refused verdict; the other three refusal precedents land in excluded because stage 2 declines them first, their registries carrying zero shrink/governance testimony in author-facing text (measured 0/0/0). So triage's disjunction is safe, and it is safe partly for a reason the ruling did not anticipate — the refusal limb is exercised by a sample of one. Saying so plainly, and filing #8576 rather than letting a sample of one pass as validation, is exactly right.

Also noted: check:type-check-coverage was kept green by a design choice rather than a workaround — writing the gate as .mjs rather than .mts keeps it out of the root tsc program (the root tsconfig sets no allowJs), so no new TS2584/TS2591 against a zero-margin entry. And check:type-check-debt was not run, with the reason stated: it fails locally on an unbuilt closure and says so itself, lint.yml builds that closure immediately before the step, and this diff touches no TypeScript. Naming a skipped check and why beats a silent omission.

Two review points

  1. The PR body says "Three more gates offer a shrink-only ratchet edit as a co-equal remedy with no authority marker — the #8435 convention stops one gate short of its own census #8519 and check-test-typecheck.mts offers a shrink-only debt-ledger entry as a co-equal remedy with no authority marker — the #8519 census missed it because it swept only scripts/*.mjs #8538 … remain open on their own terms." Both are closedThree more gates offer a shrink-only ratchet edit as a co-equal remedy with no authority marker — the #8435 convention stops one gate short of its own census #8519 by PR fix(devx): mark ratchet-editing remedies as maintainer-only in three more gates (#8519) #8539 (18:37:47Z) and check-test-typecheck.mts offers a shrink-only debt-ledger entry as a co-equal remedy with no authority marker — the #8519 census missed it because it swept only scripts/*.mjs #8538 by PR fix(devx): mark the ledger-expanding remedy maintainer-only in check-test-typecheck (#8538) #8549 (19:35:27Z). The phrasing is the safe negation form so nothing closes by accident, and Part-of PR must not also close its card is green, so there is no functional risk. ⛔ I am deliberately not editing the body myself: a whole-body PATCH on an HTML-escaped body risks double-escaping the whole thing to fix one stale clause, which is a bad trade. Recording the correction here instead — the squash message will carry a sentence that was true when the branch started.

  2. lint.yml is a hot file in this lane (touching it pulls in the workflow-gate families), and the new step lives there. The dev proved the gate runs through pnpm, not just node — the right proof locally. What is still unproven is that it runs in CI, which is a different claim; I will confirm from the job log rather than infer it from a green ESLint job, since a step that silently no-ops would leave the farm reporting clean forever. That is the same failure this gate exists to catch, so it deserves the check.


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@qq9340100@claude