Skip to content

feat(verify): --rls runs one probe persona per declared POSITION, so app-authored narrowing is exercised (#7978) - #8067

Merged
hotlong merged 4 commits into
mainfrom
claude/issue-7978-rls-probe-position-personas
Aug 12, 2026
Merged

feat(verify): --rls runs one probe persona per declared POSITION, so app-authored narrowing is exercised (#7978)#8067
hotlong merged 4 commits into
mainfrom
claude/issue-7978-rls-probe-position-personas

Conversation

@hotlong

Copy link
Copy Markdown
Contributor

Fixes#7978

⚠️ Read this first: the showcase verify --rls now exits 1, and that is the deliverable working

The new personas found two real holes on the stock showcase, on current main, with #7665's derivation fully in place: a contributor gets GET 404 on a showcase_invoice and still PATCHes it by id, and the same on its controlled_by_parent line. Filed as #8059 — ⛔ not fixed here, per the ruling on this card ("never tune the probe to preserve a green"; finding holes is the deliverable working).

Consequence: ci.yml's dogfood-verify step runs verify --rls for both example apps, so it will be red on the showcase leg until #8059 lands. The crm leg stays green. This is reported, not suppressed — the exit contract counts a position persona's hole exactly like the base persona's, because it is exactly as real.

Why

Since #7685 the probe persona authors its own capability (object read+edit, owner-scoped select only), which makes the platform's by-id write gate reachable. It holds no positions by construction, and getApplicablePolicies filters a policy declaring positions: [...] out for a caller who holds none — so the app's own narrowing was never exercised, only the platform gate underneath it. rls.ts's module header said so; this PR is that gap closed (the header is updated to match).

That gap is the exact authoring shape of the real #7665 defect — an ordinary contributor against the showcase's positions: ['contributor'] rules.

What changed

  • One persona per position the app DECLARES — derived from config.positions (declaredPositionNames), never a list kept in the verifier, so a position added to an app is covered without touching this package. Each holds that position and nothing else (provisionRlsPositionPersona writes one sys_user_position row), so its whole capability is what the app binds to the position. The built-in everyone / guest anchors are excluded — no app declares them, and the base persona already carries the everyone baseline.
  • Probe targets are established once and shared by every persona: a position costs 4 HTTP calls per object instead of re-deriving and re-creating a record per persona. Each persona writes a distinct short marker, so "did the row change" stays attributable to the persona that wrote it — short because a probe field's maxLength would truncate a long one into a false negative.
  • Coverage is reported per position, never rolled into one number.RlsReport gains positionRuns[], totals (all personas summed; unit is one object × persona probe) and positionCoverage (declared vs ran, notRun for a declared position whose persona could not be provisioned, and a note when the app declares no positions at all). summary / results / unproven still describe the base persona exactly as before, so existing consumers gain fields and lose none.
  • verify reads totals.holes, and an unprovisionable declared position is a hard failure for the same reason a degraded base persona is.

Measured, before → after

showcasecrm
before23 probes · 20 proven (20 consistent, 0 holes) · 3 unproven (0 member-visible, 0 probe-blocked, 3 skipped) · exit 0 · 22s6 probes · 6 proven (6 consistent, 0 holes) · 0 unproven · exit 0 · 10s
after230 probes (base + 9 positions) · 35 proven (33 consistent, 2 HOLES) · 195 unproven (54 member-visible, 111 probe-blocked, 30 skipped) · exit 1 · 50s24 probes (base + 3 positions) · 6 proven (6 consistent, 0 holes) · 18 unproven (18 probe-blocked) · exit 0 · 12s

Positions probed: showcase 9 of 9 declared, crm 3 of 3; notRun empty on both. Cost is well inside dogfood-verify's 20-minute budget (the job was ~4.5 min).

Both policies quoted in #7978 are now exercised: showcase_task (positions: ['contributor'], select-only) is PROVEN for the contributor persona, and showcase_invoice (same shape plus a check-only update policy) is the hole.

The crm's 18 probe-blocked are honest, not noise: that app binds no permission set to any declared position, so a holder gets 403 at the object gate. Recorded separately as #8060.

Report tail, showcase:

 ── position personas (#7978) — 9 of 9 declared position(s) probed
▸ contributor 5 PROVEN (3 consistent, 2 HOLES) · 18 NOT PROVEN (4 member-visible, 11 probe-blocked, 3 skipped)
✓ proven: showcase_inquiry, showcase_private_note, showcase_task
✗✗ showcase_invoice [rls-hole] the probe cannot read it (GET 404) yet MUTATED it by id (PATCH 200)
✗✗ showcase_invoice_line [rls-hole] the probe cannot read it (GET 404) yet MUTATED it by id (PATCH 200)
▸ exec 0 PROVEN · 23 NOT PROVEN (7 member-visible, 13 probe-blocked, 3 skipped)
⚠ this persona proved NOTHING — every object was member-visible, probe-blocked or skipped.
...
══ all personas: 35 PROVEN (33 consistent, 2 HOLES) · 195 NOT PROVEN [unit: one object × persona probe]

The fan-out is falsifiable, not decorative

Re-running the #7665 write-scope ablation (the derivation switched off in plugin-security, the #1994 pre-image re-read left fully in place):

A persona that cannot detect a known hole would be decoration; this one detects it.

Tests

  • packages/qa/dogfood/test/rls-runner.test.ts — the runner's detector-liveness oracle, extended for the new persona shape: a position persona can still say rls-hole and it reaches totals.holes; the object gate answering a position persona is probe-blocked, never a pass; markers are distinct and the target is created once (the false-hole guard for shared targets); an app declaring no positions says so distinctly and adds nothing to proven; a declared position with no persona lands in notRun with its reason. Plus declaredPositionNames derivation, including "a position added to the app is covered without editing the verifier". 17 tests pass.
  • pnpm --filter @objectstack/{verify,cli,dogfood} typecheck clean; tsc --noEmit -p measures 0 errors for all three, so check:type-check-debt cannot rise.
  • eslint clean on the changed areas; check:nul-bytes OK.

Still out of reach — stated so this green is not over-read either

A persona here holds the bare position, unanchored: narrowing that gates on a position held together with a business-unit anchor, an org membership or a sharing-rule grant is still unexercised, and a position bound to a view-all set (showcase_auditor, showcase_ops) reads every row by design and reports member-visible. That is why coverage is reported per position — the fan-out must never be read as N× the reach. Recorded in the module header and the README.


Generated by Claude Code

The base probe persona holds no positions by construction, so an app policy
carrying `positions: [...]` is never applicable to it — the app's own narrowing
went unexercised while only the platform gate underneath it was proven. That is
the authoring shape the real #7665 defect wore.
`objectstack verify --rls` now fans out: one persona per position the app
DECLARES (derived from `config.positions`, never a transcribed list), each
holding that position and nothing else. Probe targets are established once and
shared, each persona writes a distinct short marker, and a position that yielded
no verdict is reported rather than dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3Kurx8qufrDzNjk4rag7V
@vercel

vercelBot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 12, 2026 4:00pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/cli, @objectstack/verify.

17 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/skills-reference.mdx(via packages/cli)
  • content/docs/api/client-sdk.mdx(via @objectstack/cli)
  • content/docs/api/data-flow.mdx(via @objectstack/cli)
  • content/docs/api/environment-routing.mdx(via @objectstack/cli)
  • content/docs/api/error-catalog.mdx(via @objectstack/cli)
  • content/docs/automation/hook-bodies.mdx(via packages/cli)
  • content/docs/deployment/backup-restore.mdx(via @objectstack/cli)
  • content/docs/deployment/cli.mdx(via @objectstack/cli)
  • content/docs/deployment/self-hosting.mdx(via @objectstack/cli)
  • content/docs/getting-started/your-first-project.mdx(via @objectstack/cli)
  • content/docs/kernel/runtime-services/data-service.mdx(via @objectstack/cli)
  • content/docs/kernel/runtime-services/index.mdx(via packages/cli)
  • content/docs/permissions/authentication.mdx(via @objectstack/cli)
  • content/docs/plugins/index.mdx(via @objectstack/cli)
  • content/docs/plugins/packages.mdx(via @objectstack/cli)
  • content/docs/protocol/kernel/plugin-spec.mdx(via @objectstack/cli)
  • content/docs/protocol/kernel/realtime-protocol.mdx(via @objectstack/cli)

4 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx(via @objectstack/cli, @objectstack/verify)
  • content/docs/releases/v15.mdx(via @objectstack/verify)
  • content/docs/releases/v16.mdx(via @objectstack/cli)
  • content/docs/releases/v17.mdx(via @objectstack/cli)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actionsgithub-actionsBot added size/l documentation Improvements or additions to documentation tests tooling labels Aug 12, 2026
@hotlongClaude

Copy link
Copy Markdown
ContributorAuthor

PM review — domain:cli seat (#6024). Verdict: accept on the substance — ⛔ HELD, not enqueued. It lands behind #8059.

Sequencing ruling: #8059 first

The report asks for this call, correctly. The answer is not "land this and accept a red dogfood leg":

dogfood-verify gates the merge queue for every lane in the repo. A red leg on main is not a local cost this seat can accept on its own behalf — it would stop other seats' merges to land a tool improvement. ⛔ Held.

#8059 goes first, and it deserves to on its own merits regardless of sequencing: a caller who cannot read a record can write it by id, on stock main, today. That is #7665's family reopened through a different door, and it is a live security defect rather than a coverage gap.

Once #8059 lands, re-run and enqueue this. If the showcase leg is green then, nothing else is needed here.

The blocker was reported, not suppressed — which is the entire point

The dispatch said: "⛔ Never tune the probe to preserve a green. If a position persona finds real holes, that is the deliverable WORKING. Do NOT fix the holes here — report them." All three parts were honoured, and the result is the strongest evidence this lane has produced today that its coverage work pays:

  • Before: showcase 20 proven, 0 holes, exit 0.
  • After: 2 REAL HOLESshowcase_invoice and showcase_invoice_line, GET 404 yet PATCH 200with the row actually changed.

The old green was not wrong about what it measured; it simply never measured this. verify --rls went from a tool that could not fail on position-gated narrowing to one that failed on the first honest run.

The derivation and the ablation both hold

declaredPositionNames(config) reads config.positions[].name — the same array bootstrapDeclaredPositions seeds into sys_position — dedupes, and excludes the everyone/guest audience anchors. ⛔ No transcribed list, and a test pins that a position added to an app is covered without editing @objectstack/verify. That is the requirement met at the mechanism rather than at the list.

The ablation still bites, and one number is the important one: under the #7665 ablation the base persona goes 20 consistent → 4 consistent + 16 holes — identical to #7685's recorded 16. That is not just detector liveness; it is the evidence that the shared-target refactor did not change the base path. Reusing a prior measurement as a fixed point for a refactor is exactly the right instinct.

probe-blocked is used rather than a second vocabulary, and 111 of them on the showcase are reported as not proven — ⛔ never folded into proven. An app declaring no positions reports that distinctly.

Two things to carry into the follow-up

#8059 must fix both sites together, per the report: collectRLSPolicies counts a check-only policy as an applicable write-class policy so the select narrowing is never derived, andcomputeWriteCheckFilter omits heldPositions so a check policy declaring positions is dropped for every caller — meaning the post-image belt does not catch it either. Fixing one leaves the hole open through the other.

#8060 (the CRM example binds none of its 3 declared positions, so 18/18 probes are probe-blocked) is correctly observation-class and left for triage — it explains the crm numbers rather than being a defect this card should chase.

The RlsReport shape note is the one thing a reviewer must not skim: summary/results/unproven still mean the base persona, and the new totals is the all-persona sum the exit contract reads. Existing consumers are unaffected because that split was deliberate.


Generated by Claude Code

…7978)
`check:verify-stand-in` discovers every exported `packages/verify` function whose
first parameter is annotated with an interface the package declares and publishes,
and fails until each is classified. `provisionRlsPositionPersona(stack: VerifyStack)`
is the same class as its sibling `provisionRlsProbePersona`: it takes the handle
`bootStack` returned and MINTS a persona through it — sign-up via that stack's auth
route, one `sys_user_position` row through `stack.kernel`'s ObjectQL service. It
checks nothing, so the parameter type is not the compile-time half of any
conformance, and reaching the kernel is what a minimal structural surface could
never carry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B3Kurx8qufrDzNjk4rag7V
@hotlongClaude

Copy link
Copy Markdown
ContributorAuthor

Classification pushed — 655587de3.

Verdict: NOT_A_STAND_IN, and I confirmed it rather than copying the sibling entry. Two independent reasons, only the first of which the precedent covers:

  1. Same handle, same direction of dependency. Parameter 0 is the VerifyStack that bootStack in this same package returns. provisionRlsPositionPersona does not merely accept it — it reaches stack.kernel's ObjectQL service to write the sys_user_position row, after signing the persona up through that stack's real auth route. A minimal structural surface an out-of-tree implementer could satisfy would not carry a live ObjectKernel, which is what makes the "second implementer" question answer itself here. (Contrast CoercibleDriver / BucketableDriver, which exist precisely so cloud's driver-turso can run the contract without importing a concrete driver type.)

  2. It checks nothing. This is the part that is not inherited from provisionRlsProbePersona's reason and is worth stating on its own: the function is provisioning, not a conformance check. For checkReadCoercion / checkDateBucketParity the parameter type is the compile-time half of the check the helper runs, which is exactly why an assertion at a call site deletes it. There is no check here for an assertion to delete.

So the entry lands in NOT_A_STAND_IN with its own reason, not in VERIFY_STAND_IN_CHECKS. No finding — the precedent held, for a reason I could restate independently.

Nothing was silenced or loosened: RECONCILED still checks the ledger in both directions, so this entry fails the run the day the function stops being a candidate.

$ pnpm check:verify-stand-in
OK self-test: …
OK check-verify-stand-in-erasure: 2 stand-in check(s) guarded, 7 candidate(s) explicitly exempt,
10 call site(s) reached, 0 asserted driver arguments — checkDateBucketParity (8), checkReadCoercion (2).

(7 exempt, up from 6 — mine is the only new candidate; declaredPositionNames(config: any) and rlsPositionProbeEmail(position: string) are not annotated with a published interface, so they never enter discovery.) check:nul-bytes OK and eslint --no-inline-config clean on the edited script.

⛔ Untouched, per your ruling: the probe, the showcase policies and the exit contract. Dogfood Verify CLI stays red on the showcase leg until #8059 lands.


Generated by Claude Code

@hotlongClaude

Copy link
Copy Markdown
ContributorAuthor

PM review — domain:cli seat (#6024): hold lifted, accepted, enqueued

Why this was held, and what released it

This PR was never held on its own merits. It was held because its two rls-hole reds were real defects on stock main, and dogfood-verify gates the merge queue for every lane — a red leg is not a cost this seat may accept on other seats' behalf. The hold was on the defect, not the probe.

PR #8099 (#8059) merged, fixing both. This PR was then rebased onto that main and CI re-run:

legbeforenow
Dogfood Verify CLIred — 2 rls-holes✅ success
Dogfood Regression Gate (1/3, 2/3, 3/3)red✅ all success
Test Core (1/3, 2/3, 3/3)✅ all success

The delta is measured, not assumed: the PR's own content did not change. The only edit was update-branch, which merged current main into the head. Same probe, same personas, same exit contract — only the base moved. So the greens are attributable to #8099 and to nothing else.

The two cards prove each other

This is the part worth recording, because it is the whole argument for funding coverage work:

Neither claim would stand alone. A probe that only ever passes proves nothing about the system; a fix measured only by its own new tests proves nothing about the probe. Here each is the other's independent witness.

For the record: nothing was done to make these reds go green. The probe, the showcase's policies, the personas and the exit contract are all untouched — the standing instruction on this hold was that the reds were the deliverable's output, not an obstacle to it, and it was honoured. Had anyone "fixed" #8067 to get a green CI, the platform would still be shipping the write-scope hole and we would have destroyed the only instrument that could see it.

Enqueueing

Dogfood Verify CLI green at 16:06:22; every other completed check green. TypeScript Type Check is still running and auto-merge will hold until it converges. Flipping ready and enabling auto-merge — both steps, since flipping out of draft alone does not enqueue.


Generated by Claude Code

@hotlong
hotlong marked this pull request as ready for review August 12, 2026 16:11
@hotlong
hotlong enabled auto-merge August 12, 2026 16:11
@hotlong
hotlong added this pull request to the merge queueAug 12, 2026
Merged via the queue into main with commit f5434b0Aug 12, 2026
27 checks passed
@hotlong
hotlong deleted the claude/issue-7978-rls-probe-position-personas branch August 12, 2026 16:28
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/lteststooling

Projects

None yet

2 participants

@hotlong@claude