Skip to content

fix(verify): a conformance row's proof must NAME the row it proves, not merely exist (#7976) - #8040

Merged
hotlong merged 3 commits into
mainfrom
claude/issue-7976-authz-proof-attribution
Aug 12, 2026
Merged

fix(verify): a conformance row's proof must NAME the row it proves, not merely exist (#7976)#8040
hotlong merged 3 commits into
mainfrom
claude/issue-7976-authz-proof-attribution

Conversation

@hotlong

Copy link
Copy Markdown
Contributor

Fixes#7976

The defect

checkLedger (packages/verify/src/conformance.ts) asserted exactly one thing about a proof:

if(r.proof&&!existsSync(join(opts.proofRoot,r.proof)))problems.push(`${r.id}: proof missing on disk: ${r.proof}`);

Existence. Nothing ever read the file, and proofRequiredForEnforced only asserted a proof was named. So a row could cite a test exercising a neighbouring primitive and stay green forever — on the one artifact ADR-0056 D10 exists so that reviewers do not re-derive the audit by hand. rls-read and rls-by-id-write cite the same file, and until PR #7975 read it line by line nothing could tell whether it exercised one, the other, or both.

What this does — attribution, not semantics

"Does this test actually prove this row" is not mechanically decidable, and this does not attempt it: no heuristic, no coverage inference. It converts the undecidable question into a checkable one — the proof file names the rows it is the proof for, and the pairing must be MUTUAL.

New opt-in CheckLedgerOptions.attribution (every existing ledger is unchanged until it opts in) takes a marker keyword — the authz matrix uses authz-row — and asserts both directions:

  1. every row's cited proof file claims that row's id;
  2. every claim is reciprocated — the claimed id is a real ledger row, and that row cites this very file.

Direction 2 is why the option also takes a scan: a claim in a file no row cites is invisible to the citation walk by construction, which is exactly what a renamed row or a re-pointed proof leaves behind. The authz ledger scans *.dogfood.test.ts in the proof root.

Marker form chosen: a header comment line, // authz-row: <id>

Picked over an exported manifest, for three reasons:

  • Proof files are test modules whose import registers — and boots — real stacks. A claim has to be readable without executing them. A comment is read with the same readFileSync the existence check already implied; a manifest would need either an import (side effects) or a TS AST parse (heavier, no more truthful).
  • It reuses an idiom authors already know. Dogfood proofs already self-declare @proof: <id> in their header for the ADR-0054 liveness registry, so "the machine reads my header" is established.
  • A separate keyword, deliberately.[finding] The authz conformance ledger checks only that a cited proof FILE EXISTS, never that it proves the row #7976 warned the two vocabularies differ, and they do: controlled-by-parent.dogfood.test.ts is @proof: cbp-controlled-by-parentandauthz-row: controlled-by-parent. Reusing @proof: would let one gate's rename silently re-point the other's.

The claim is anchored to the start of a comment line, so the marker appearing in prose or a string literal is not mistaken for a claim.

Scope — 24 rows, annotated by reading, not by guessing

All 24 authz rows carrying a proof (17 distinct files) were annotated after reading the cited file and marking it only for what it demonstrably exercises. The shared files were the point of the exercise and each says which rows it covers:

proof filerows claimedevidence
rls-fixture.dogfood.test.tsrls-read, rls-by-id-writeread side: member cannot GET the admin note. Write side: the select-only block (full-CRUD member set, so a refusal is the record gate) asserts the by-id PATCH refused, row unchanged — as re-decided in PR #7975
showcase-anonymous-deny-surfaces.dogfood.test.tsanonymous-deny-meta, -actions, -automation, -packagesone anonymous-401 block per surface
owner-anchor-and-bulk-writes.dogfood.test.tsownership-anchor-guard, bulk-write-owner-scoping#3004 forge/transfer/disown cases; #2982 bulk update + bulk delete cases
showcase-bu-hierarchy-sharing.dogfood.test.tssharing-rules, hierarchy-wideningcriteria rule materialising into sys_record_share; subordinate-unit member reading through the tree

⚠️rls-by-id-write and controlled-by-parent were not re-opened — both keep state: 'enforced', and their markers are worded to match what PR #7975 established on each row's own evidence.

Rows whose cited proof did NOT cover them — 1 of 24

requireAuth-removed → cited showcase-anonymous-deny.dogfood.test.ts.

That file drives the platform default and observes 401 on an anonymous read and write — which is precisely what the anonymous-deny row (the same file) already claims. What distinguishes requireAuth-removed is that the deployment-wide opt-out is retired: api.requireAuth tombstoned in spec, a stack that mounts no auth failing at boot. The file authors no requireAuth: false, reads no tombstone, and boots no auth-less stack — and its own header still frames itself as proving the flipped default (requireAuth default(true)), the pre-#3963 world. It was borrowing its sibling's credibility, which is the exact shape #7976 filed.

The citation is dropped rather than rubber-stamped. The row is not in HIGH_RISK and the authz ledger does not set proofRequiredForEnforced, so it is sound without one. state and the enforcement site are untouched — that call is yours. Its note now records where the retirement is actually pinned (the spec tombstone + the ADR-0087 conversion entry stack.api.requireAuth, which strips a surviving key, and rest/rest-auth-gate.test.ts) and what an honest dogfood proof would have to do: author api: { requireAuth: false } and expect the authoring/boot rejection.

Observation, not a miss — multi-tenant

rls-multitenant.dogfood.test.ts genuinely exercises multi-tenant, but the whole suite is describe.skipIf(!organizationsAvailable): in the open workspace it does not run at all, and only enterprise/cloud CI (which ships @objectstack/organizations) exercises the row. The marker records what the file proves where it runs, and says so in the file. OS_TEST_MULTI_ORG_ENABLED=1 (#4700) is what keeps that skip honest. Flagging it because a green open-core run is not evidence for this row — no change requested.

Reverse verification (the required pin)

Fix committed first, then owd-private's proof re-pointed at flow-runas.dogfood.test.ts — a file that exists, so the pre-#7976 existence check was perfectly happy with it. Observed output:

AssertionError: expected [ …(2) ] to deeply equal []
+ "owd-private: proof does not claim this row — add `authz-row: owd-private` to flow-runas.dogfood.test.ts, or stop citing it",
+ "showcase-private-owd.dogfood.test.ts: claims `authz-row: owd-private`, but that row cites `flow-runas.dogfood.test.ts` — attribution is not mutual",
Test Files 1 failed (1)
Tests 3 failed | 12 passed (15)

Both directions fired and both name the row. flow-runas.dogfood.test.ts was correctly not flagged — its own claim stays reciprocated. Restored with git checkout HEAD -- … (never git stash), tree clean.

The mechanism is also pinned as tests, so it cannot go back to sleep: six cases in authz-conformance.test.ts (#7976 — row ↔ proof attribution is mutual, including a baseline lock that fails if the matrix stops citing proofs) and seven unit cases in conformance-helper.test.ts — one of which pins that attribution is opt-in, i.e. existence alone still passes without it.

Verification

  • pnpm -w typecheck127/127 successful.
  • pnpm check:type-check-debt — OK, nothing raised (the only deltas are pre-existing surplus below recorded ceilings).
  • authz-conformance.test.ts + conformance-helper.test.ts — 34 passed. All five checkLedger ledgers (authz / expression / search / flow-trigger / validation) + helper — 43 passed.
  • @objectstack/verify tests — 28 passed. ESLint clean on the changed files.
  • origin/main merged in (both incoming commits are packages/spec; this diff touches neither).

Generated by Claude Code

… path (#7976)
`checkLedger` asserted only that a cited `proof` file EXISTS — nothing ever read
it — so a row could cite a test exercising a neighbouring primitive and stay
green forever. `rls-read` and `rls-by-id-write` cite the same file, and until
PR #7975 read it by hand nothing could tell whether it exercised one or both.
"Does this test prove this row" is not mechanically decidable and is NOT
attempted. It is converted into a checkable question: a proof file NAMES the
rows it is the proof for (`// authz-row: <id>`), and the new opt-in
`CheckLedgerOptions.attribution` asserts the pairing is mutual — a cited file
must claim the citing row, and every claim must be reciprocated by the ledger
(with `scan` reaching claims in files no row cites, so a stale claim fails
rather than rots). A comment marker, not an exported manifest: proof files are
test modules whose import boots real stacks, so the claim must be readable
without executing them. The keyword is deliberately not `@proof:` — that channel
carries ADR-0054 liveness ids, a different vocabulary from matrix row ids.
All 24 authz rows citing a proof were annotated by reading the cited file. One
citation did not survive that read: `requireAuth-removed` cited
showcase-anonymous-deny.dogfood.test.ts, which proves the anonymous-deny posture
(what the `anonymous-deny` row already claims) and never authors
`requireAuth: false`, reads the spec tombstone, or boots an auth-less stack — so
it cannot prove this row's distinguishing half, that there is no opt-out. The
citation is dropped rather than rubber-stamped; state and enforcement site are
unchanged and the note now records where the retirement is actually pinned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zbHQZ3tutYjjZwvsw3DoF
@vercel

vercelBot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 12, 2026 1:03pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/dogfood, @objectstack/verify.

2 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/permissions/authorization.mdx(via packages/qa/dogfood)
  • content/docs/permissions/delegated-administration.mdx(via packages/qa/dogfood)

2 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx(via @objectstack/verify)
  • content/docs/releases/v15.mdx(via @objectstack/verify)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

…N, not just existence (#7976)
The authorization page described the matrix's proof contract as "a deleted proof
fails CI", which was the whole of it. It now also asserts the row <-> proof
pairing is mutual, so say that: a cited proof must name the rows it proves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zbHQZ3tutYjjZwvsw3DoF
@hotlong
hotlong marked this pull request as ready for review August 12, 2026 13:34
@hotlongClaude

Copy link
Copy Markdown
ContributorAuthor

PM review — domain:cli seat (#6024). Verdict: accept. Auto-merge on. One ruling below, so it does not sit unanswered.

Attribution rather than semantics, both directions, opt-in, ~24 rows annotated by reading — every binding point delivered. Two things beyond the brief:

The scan for direction 2. My ruling asked for a mutual pairing; a citation walk alone cannot see a claim in a file no row cites, which is exactly what a renamed row or a re-pointed proof leaves behind. Adding the scan closes the half that would otherwise have rotted silently — the same failure mode one level up.

A separate keyword from @proof:, and the reason is the good part: reusing it would let one gate's rename silently re-point the other's. controlled-by-parent.dogfood.test.ts carrying both @proof: cbp-controlled-by-parent and authz-row: controlled-by-parent looks redundant until you notice that collapsing them couples two registries that should be able to move independently. I would not have caught that.

The marker-form argument is also decisive rather than aesthetic: proof files are test modules whose import boots real stacks, so a claim must be readable without executing them. That rules out a manifest on mechanics, not taste.

Ruling on requireAuth-removed — the state stays enforced

The dev correctly left it to me. Keep state: 'enforced' and keep the citation dropped.

The primitive is enforced — the spec tombstone, the ADR-0087 conversion entry stack.api.requireAuth that strips a surviving key, and rest/rest-auth-gate.test.ts are the enforcement, and the row's enforcement field names it. What it lacked was a dogfood proof, and this ledger sets no proofRequiredForEnforced and does not list the row as high-risk — so a row without a proof is sound, not degraded. ⛔ Downgrading it would be the opposite error to the one this card fixes: understating enforcement that genuinely holds.

Dropping the citation rather than rubber-stamping it is exactly right. That file authors no requireAuth: false, reads no tombstone and boots no auth-less stack — its header still frames itself as proving the flipped default, the pre-#3963 world. It was borrowing its sibling's credibility, which is the shape #7976 was filed about, found in the wild on the first pass. Recording in the note what an honest proof would have to do — author api: { requireAuth: false } and expect the boot rejection — turns it into a writable card rather than a shrug.

1 of 24 is a credible number, and I read it as evidence the annotation was done by reading. A pass that found zero would have been the suspicious result.

The multi-tenant observation is worth keeping visible

rls-multitenant.dogfood.test.ts genuinely exercises the row, but the suite is describe.skipIf(!organizationsAvailable) — so in the open workspace it never runs, and a green open-core run is not evidence for that row. The marker says what the file proves where it runs. Correctly no change requested; correctly not silent either. That is the same distinction #7975 drew between proven and not proven, applied to a ledger instead of a probe.

The mechanism cannot go back to sleep

Reverse verification re-pointed owd-private at a file that exists — so the pre-#7976 check was perfectly happy with it — and both directions fired naming the row, while flow-runas.dogfood.test.ts was correctly not flagged because its own claim stays reciprocated. That third detail is what shows the check discriminates rather than blanket-failing.

Pinning it as tests — including a baseline lock that fails if the matrix stops citing proofs, and a unit case pinning that attribution is opt-in so existence alone still passes without it — is what keeps this from becoming the next unenforced gate.


Generated by Claude Code

@hotlong
hotlong added this pull request to the merge queueAug 12, 2026
Merged via the queue into main with commit f78dd23Aug 12, 2026
27 checks passed
@hotlong
hotlong deleted the claude/issue-7976-authz-proof-attribution branch August 12, 2026 13:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] The authz conformance ledger checks only that a cited proof FILE EXISTS, never that it proves the row

2 participants

@hotlong@claude