Skip to content

test(qa): declare each authz probe's instrument kind and fail a probe that lost its population - #13436

Merged
os-trump merged 2 commits into
mainfrom
claude/issue-13411-authz-ratchet-route-population
Aug 30, 2026
Merged

test(qa): declare each authz probe's instrument kind and fail a probe that lost its population#13436
os-trump merged 2 commits into
mainfrom
claude/issue-13411-authz-ratchet-route-population

Conversation

@os-trump

Copy link
Copy Markdown
Collaborator

Part of #13411 — and deliberately not a closing keyword. See "The conflict I am reporting" at the bottom: the card's central deliverable is a population-source decision that this PR does not take, and a merge that silently retired the card would remove it from the open-card filter while that decision is still owed.

Part of #13260 / PR #13369, whose census is the baseline every number here moves from.

STEP ONE — where does the population come from? Measured, and it is a STOP

The card's first instruction was to measure whether the route ledgers can supply this ratchet's population before writing any regex. They were measured. They cover the routes; they cannot supply this ratchet's guarantee.

What the ledgers DO cover — richly, and far beyond this table

ledgerrowsmachine contractreaches
packages/rest/src/rest-route-ledger.ts94 over 19 familiesper-route, RestServer.getRoutes() on a booted server, rest-route-ledger.conformance.test.tsall 17 registrars — this ratchet reaches 1
packages/runtime/src/route-ledger.ts80 over 21 domainsdomain-level, live registry introspection (domainRegistry.list()); the per-route rows are documentationall 15async handle*( dispatcher methods; all 16DomainRoute prefixes

Nine further ledgers exist repo-wide (290 rows in total: auth 65, cloud-connection 16, datasource 11, storage 10, settings 4, console 4, i18n 3, metadata 2, trigger-api 1).

On FILE SELECTION the ledgers are simply the right answer — the #13376 layer. All 16 DomainRoute prefixes declared by the 15 domain files in packages/runtime/src/domains/ appear as ledger domains, enumerated from the live registry, so a new domain file cannot be silently absent. That is the property the probe table lacks and cannot acquire by widening: a file no probe names emits no signal at all — no key, no STALE, no UNCLASSIFIED — so its absence is structurally unobservable.

⛔ Why they still cannot be this ratchet's population — three measured blockers

1. No notion of "gated", and this ratchet's promise is specifically about UNGATED routes. Ledger dispositions grade SDK expressibility, not authorization: REST reads sdk 84 / server-only 7 / public 3. Cross-checked directly rather than assumed — of the 8 REST route mounts measured to carry no enforceAuth, the ledger grades 3 server-only, 3 public, and 2 sdk; and one of those two is GET /api/v1/ui/view/:object/:type, the single route in this entire population ever measured unguarded. Its ledger row is shape-identical to the 83 sdk rows that are gated. public states intent for 3 browser-facing form routes; it is not a gate measurement and was never built as one.

2. Deriving "gated" from source syntax is measurably unsafe. Measured, not assumed: scanning each of the 80 this.routeManager.register( call sites in rest-server.ts for enforceAuth reads 50 gated / 30 ungated — and 22 of those 30 are FALSE, in two structural shapes:

  • registerMetadataEndpoints installs a wrapping guardedRouteManager and restores it in a finally, so its 19 inner routes are gated with zero enforceAuth at the call site;
  • registerSecurityExplainEndpoints shares one handler const declared outside its 3register( calls.

A 73% false-ungated rate, concentrated on the largest registrar. Hand-annotating the exceptions is the same rot this card exists to end, and a wrong "gated" reading converts a visible gap into a written-down false assurance — which #13260's triage explicitly ruled worse than an honest UNCLASSIFIED.

3. ⭐ A ledger is a derived data file, one guarded step behind the source — so no ledger-sourced population can pass this card's own acceptance test. Leg A's mutation adds one this.routeManager.register( site to registerUiEndpoints in rest-server.ts. That edit does not touch rest-route-ledger.ts. A ledger-sourced population therefore mints no new key and authz-conformance.test.ts stays GREEN on exactly the mutation it should catch; the red lands in rest-route-ledger.conformance.test.ts instead — a different gate, in a different package, promising something else. A composed guarantee (no route without a ledger row; no ledger row without a classification) is defensible, but it is a different promise from the one the matrix header states, and adopting it is a contract decision.

Granularity, for completeness: the ledgers enumerate wire routes (94 + 80); the matrix classifies authz primitives (51 rows, 9 covers keys). Route-granular classification is ~174 keys; family/domain-granular is ~40 (~31 new). #13260's triage forbade bulk-classifying even 16.

The stop condition is met on the card's own named prong ("no notion of gated") plus two more. ⛔ No regex was widened, no file was hand-added, and the route population is byte-for-byte untouched — no probe added, removed, re-pointed or broadened. The full reading is recorded in the census header so it cannot rot, and the decision is handed back.

What this PR DOES repair — the population-independent half

1. The dead-probe CLASS, not just its one instance

The ratchet had a second blind-spot mechanism neither of its signals can reach. UNCLASSIFIED fires on a key that APPEARS; STALE fires on a covers key that DISAPPEARS. Both are keyed on a key. A probe that stops minting a key no row ever covered produces neither signal and reads exactly like a healthy one.

Each PROBES entry now declares its instrument kind, and checkProbeInstrumentIntegrity fails a ROUTE_ENUMERATION or GATE_PIN probe that mints zero keys as a DEAD PROBE. TRIPWIRE is exempt by declaration — matching nothing is its whole job — which is exactly why the kind is declared per probe rather than inferred from the pattern: the exemption has to be a written, reviewable claim.

The taxonomy the card requires is therefore carried as data on the probe, not as prose in a third file: 3 ROUTE_ENUMERATION · 6 GATE_PIN · 7 TRIPWIRE. The census derivation reads the declarations back out of the test's own source and the census test asserts them, so a kind that changes cannot leave the record describing the old table.

2. The data:hono-plugin.ts instance — re-declared, ⛔ not deleted

Commit e5a4d26901 (2026-07-31) deleted its population (3 matching mounts → 0) and the probe stayed, silent ever since. It is re-declared TRIPWIRE with the dated evidence, not deleted — deleting it would make this gate see less. Its zero becomes a checked reading (the census carries a positive control from that same file), and a /data mount returning to that file mints a key and lands UNCLASSIFIED. ⛔ Its key deliberately carries no TRANSPORT-WIRED marker, so the #9083 admission rule does not apply: that rule demands a per-recipient delivery authorization site, which is the wrong checklist for an ordinary data surface.

The live spelling this pattern watches exists one file away in current-user-endpoints.ts (3 mounts, none of them /data), which the table does not name. Naming it is a population decision and is not taken here.

3. The matrix header count — repaired, and the pin INVERTED

"15 probes" → 16. The census pin flips from not.toBe (drift recorded) to toBe (drift closed): the header sentence is now held equal to the table by a test, so the next probe added without moving the prose is RED instead of a discrepancy noted in a third file. MATRIX_HEADER_PROBE_CLAIM is 16 and is read out of the header's own text.

4. #13376 — honestly, this PR does NOT repair it

It is the file-selection layer and it is downstream of the population decision. ⭐ The measurement above is a direct input to it: a ledger-domain-sourced file selection would settle it by construction. #13376 remains open.

Reverse verification — three legs, and the most important one is a NEGATIVE result

Both instruments are source-text scanners (readFileSync over the probe files): no dist/ resolution, no rebuild leg on either side. Every leg proved mutation and restore on disk by blob hash, never by exit code, under a trap ... EXIT INT TERM with absolute paths.

Leg A — the card's load-bearing ablation. Predicted GREEN, and it is GREEN. One ungated route injected into registerUiEndpoints (this.routeManager.register( sites 80 → 81, marker present, blob b9809d0e52…565f267d45…).

  • authz-conformance.test.ts33 tests, 0 failed. GREEN, with an ungated route on disk.
  • authz-probe-blind-spot.test.tsRED: AssertionError: expected 81 to be 80.

This is the measurement that matters, and it is why the answer is a stop rather than a repair. The direction was predicted before running it: the repair in this PR does not touch the population, so it cannot close the route blind spot, and claiming otherwise would be the "numbers improved" outcome the card warns against. Leg A reproduces #13369's reading unchanged. The blind spot is exactly as large as it was, and closing it is the decision being handed back.

Leg B — the instrument is still alive where it can see. Predicted RED, and it is RED.registerMetadataEndpoints renamed (7 → 0 occurrences, replacement present).

  • RED, 7 failures, including AssertionError: STALE covers — surface no longer in source: meta:rest-server.ts:registerMetadataEndpoints.
  • ⭐ And the new check fires independently on the same regression: DEAD PROBE — packages/rest/src/rest-server.ts: a ROUTE_ENUMERATION probe minting ZERO keys. Two signals where there was one.

Leg C — the new check bites on the class it was built for. Predicted RED, and it is RED. The hono probe's kind reverted to ROUTE_ENUMERATION (its state before this PR; both directions counted at the anchor: TRIPWIRE 1 → 0, ROUTE_ENUMERATION 0 → 1, blob 15fdb0a6a7…6bd99b61d2…).

  • RED: DEAD PROBE — packages/plugins/plugin-hono-server/src/hono-plugin.ts: a ROUTE_ENUMERATION probe minting ZERO keys.

⇒ the condition that was silent on origin/main is now loud.

Restore proven on every leg: blob identical to the HEAD blob, git diff HEAD0 lines, git status --porcelainempty.

Census before → after

readingbeforeafter
MATRIX_HEADER_PROBE_CLAIM15 (drift pinned)16 (drift closed, pin inverted)
PROBE_TABLE entries / files / keys16 / 11 / 916 / 11 / 9 (unchanged — population untouched)
hono-plugin.ts kindsROUTE_ENUMERATIONTRIPWIRE
TRIPWIRE probes6 over 5 files7 over 6 files
BLIND_SPOT_TOTAL_STATIC / _RUNTIME75 / 8075 / 80 (unchanged, and Leg A proves it)
suite size5462

Every per-file population / reachable / blindSpot number and every positive control is unchanged and still non-zero — re-derived, not hand-edited.

Verification

Union run on 4f69a33484, the branch head, after merging current origin/main (6 commits) and re-deriving.

Green:pnpm lint (repo-wide, --no-inline-config, exit 0, 89s — no narrowing, so nothing to declare) · pnpm --filter @objectstack/dogfood typecheck (exit 0) · pnpm --filter @objectstack/dogfood exec vitest run test/authz-conformance.test.ts test/authz-probe-blind-spot.test.ts62/62.

The green is over source that was actually read:tsc --noEmit --listFiles (1277 files) contains all four edited files, 1 hit each, with a negative control at 0.

Gate families derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstackafter the merge (stderr banner confirms commit 4f69a33484, this checkout, --repo assertion holds), all run, all exit 0: check:doc-authoring · check:cross-package-test-inputs · check:nul-bytes · check:test-source-alias · check:logger-receiver-detach · check:objectql-double-limit · check:page-declaration-shape · check:published-files · check:slot-lookup · check:type-source-resolution · check:where-matcher · check:query-options-erasure · check:engine-double-contract · check:type-check-coverage · spec check:empty-state / check:liveness / check:strictness-ledger / check:variant-docs · check-ci-filter-parity · check-comment-mask-adoption · check-keyed-text-bounds · check-plugin-teardown-shape · check-shard-attestation · check-undeclared-dep-imports · check-affected-docs · check-drift-comment · check-cross-package-test-inputs.

⚠️check:doc-authoring was in the union as predicted, and it is not a vacuous green: its sibling-package prose-id leg read 79,572 strings across 1,050 parsed sources, 831 pinned sites, no growth. No internal tracker id was added to any runtime string; the ids in this change live in // comments. ⛔ No baseline entry was added.

NOT MEASURED (each self-declared by the gate as a prerequisite failure — ⛔ none of them a red, and none in the green list above):

  • check:test-completeness — grades a saved turbo run test log; there is none locally. Its own text: "the local reading for this gate is NOT MEASURED … it is not a red, and there is nothing here to fix." (exit 3)
  • check:dual-build-cjs-loads — reads built output; 11 packages have no dist/ here. Its own words: "PREREQUISITE NOT MET … ⛔ This is NOT a pass: nothing was measured." (exit 3)
  • check:type-check-debt — refuses --re-measure while 4 workspace dependencies have no built type entry point, because measuring would "silently measure a DIFFERENT WORLD". Its structural half check:type-check-coverage is green, and @objectstack/dogfood carries no DEBT or TEST_DEBT entry at all (no test-typecheck-debt.json, no mention in the ledger script), so the ratchet has nothing to re-measure for this package.

No changeset: the diff is four test files in @objectstack/dogfood, a private: true package that publishes nothing — skip-changeset.

⛔ What is NOT claimed, and what is NOT touched

⚠️ The conflict I am reporting rather than picking silently

The dispatch asked for a closing keyword on this card. My finding is a stop condition, so the card's central deliverable — where the population comes from — is still owed. A closing keyword would retire the card on merge while that decision is open, and an open-card filter would stop showing it. This PR therefore uses Part of, and the population-source decision returns to the PM/maintainer with the reading attached.

⚠️ Note on spelling, for whoever edits this body: GitHub's closing-keyword parser ignores negation and does not spare backticks, so the keyword is kept away from every card number on this page — including in the sentence you are reading.

Generated by Claude Code


Generated by Claude Code

… that lost its population
The authz-conformance ratchet had a second blind-spot mechanism that neither of
its two signals can reach. UNCLASSIFIED fires on a key that APPEARS; STALE fires
on a `covers` key that DISAPPEARS. Both are keyed on a key, so a probe that
stops minting a key no row ever covered produces neither signal and reads
exactly like a healthy one.
Each `PROBES` entry now declares its instrument kind (ROUTE_ENUMERATION /
GATE_PIN / TRIPWIRE), and a non-tripwire probe minting zero keys fails as a
DEAD PROBE. The raw-hono `/data` probe — whose population commit e5a4d26
deleted on 2026-07-31, 3 matching mounts to 0 — is re-declared TRIPWIRE with the
dated evidence rather than deleted, so the gate keeps watching that file.
The matrix header's probe count is corrected 15 to 16 and its census pin is
INVERTED from "these must differ" (drift recorded) to "these must be equal"
(drift closed). The three-kind taxonomy moves out of census prose into the probe
table as data, is read back by the census derivation, and is asserted.
The route population is deliberately UNTOUCHED: no probe added, removed or
widened. The measured reading on where it should come from is recorded in the
census header instead — the route ledgers cover the routes but carry no notion
of "gated", a syntactic gate reading is 73% false-ungated on rest-server.ts, and
a ledger lags source by one guarded step. That is a contract decision, not a
repair.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TvqBFLRzXdSPcbusDoED9k
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 3bfd4e39daf97c510c1ecccf9da773babc28c241packageMentionDocs.

@os-trumpos-trump added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 30, 2026 — with Claude
@os-trump
os-trump marked this pull request as ready for review August 30, 2026 11:57
@os-trump
os-trump added this pull request to the merge queueAug 30, 2026
Merged via the queue into main with commit 936893fAug 30, 2026
38 checks passed
@os-trump
os-trump deleted the claude/issue-13411-authz-ratchet-route-population branch August 30, 2026 12:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gatetests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@os-trump@claude