fix(verify): make the RLS by-id-write class reachable; re-decide the two authz-matrix rows on their own evidence

.changeset/verify-rls-probe-reachability.md

@@ -0,0 +1,66 @@
+---
+"@objectstack/verify": minor
+"@objectstack/cli": minor
+---
+
+fix(verify): `--rls` reported 0 HOLES over a probe that could not reach the class it claims to prove (#7685)
+
+`objectstack verify --rls` exercises one invariant — **you cannot mutate what you
+cannot see** (#1994). It reported `0 HOLES` on both example apps. Neither number
+was evidence.
+
+**The probe was answered by the OBJECT gate, not by record scope.** The persona
+was a bare `signUp()` member holding no object grants, so
+`checkObjectPermission` refused with 403 before the row-level gate was ever
+consulted — and the runner banked that 403 as `rls-consistent`. Measured on the
+stock showcase: **11 of 13 "consistent" verdicts were the object-gate 403**, and
+only 2 were a record-scope 404. On those 11 objects no platform regression could
+have produced `rls-hole`, so their green was unfalsifiable by construction.
+
+**A skip read as a pass, and it cascaded.** One `showcase_account` auto-record
+400 skipped that object and every object with a required relation to it — 8 of
+23 objects skipped — while the summary line still said `0 HOLES`.
+
+## What changed
+
+- **The probe persona is now the one the class needs**: object read+edit on every
+ declared object, narrowed by an owner policy authored `operation: 'select'`
+ only, registered at boot (`rlsProbeSecurity`) so its policies are actually on
+ the resolution path. That is deliberately the authoring shape that WAS the hole
+ (#7665), so every object of every verified app is now a live regression guard
+ for the by-id write-scope derivation.
+- **Reachability is measured, not assumed.** Before probing an object the runner
+ asks whether the persona can list it at all; a 403 is reported as
+ `probe-blocked` — a distinct status that is never a pass — instead of being
+ recorded as a consistent verdict.
+- **An unsatisfiable create no longer cascades.** When the app's own validation
+ rejects the derived record, the runner adopts an existing row as the probe
+ target, so the object is probed and its dependents keep their master.
+- **The report distinguishes PROVEN from NOT-PROVEN.** `RlsReport.summary` gains
+ `proven` / `unproven` / `probeBlocked`, `RlsReport.unproven` lists every object
+ the run did not exercise with its reason, `RlsReport.probe` names the persona,
+ and the formatted output prints an explicit "this run is not a clean bill of
+ health" line whenever anything went unproven.
+- **A degraded probe fails the run.** If the persona cannot be provisioned the
+ report says so and `verify` exits non-zero, rather than quietly probing with an
+ ungranted member and reporting success.
+
+## Measured, before → after
+
+| | showcase | crm |
+|:--|:--|:--|
+| before | 13 consistent (11 object-masked), 0 holes, 2 member-visible, **8 skipped** | 4 consistent (all object-masked), 0 holes, **2 skipped** |
+| after | **20 of 23 PROVEN**, 0 holes, 3 unproven | **6 of 6 PROVEN**, 0 holes, 0 unproven |
+
+The remaining 3 are honestly unprovable by this runner: one object has no
+plain-text field to mutate, two are read-only federated objects.
+
+The new green is falsifiable: ablating the #7665 write-scope derivation in
+`plugin-security` — while leaving the #1994 pre-image re-read fully in place —
+turns 16 of the 20 proven showcase objects into `rls-hole` and exits 1.
+
+## No behaviour change outside the verifier
+
+This is tooling: no runtime, spec or enforcement path is touched.
+`runRlsProofs`' existing four-argument call shape still works, and consumers that
+only read the report gain fields rather than losing any.

packages/cli/src/commands/verify.ts

@@ -10,8 +10,11 @@ import {
  formatReport,
  runRlsProofs,
  formatRlsReport,
+ provisionRlsProbePersona,
+ rlsProbeSecurity,
  type VerifyReport,
  type RlsReport,
+ type RlsProbeDescriptor,
 } from '@objectstack/verify';
 import { loadConfig } from '../utils/config.js';
 
@@ -106,22 +109,50 @@ export default class Verify extends Command {
  // proving its authorization.
  let rls: RlsReport | undefined;
  if (flags.rls) {
- const rlsStack = await bootStack(config, { multiTenant });
+ // [#7685] The by-id-write class is only REACHABLE for a persona holding
+ // the object-level grants AND standing outside the record scope. A bare
+ // `signUp()` member holds no grants, so `checkObjectPermission` answers
+ // 403 before record scope is consulted and every probe was masked — 11 of
+ // 13 "consistent" showcase verdicts were that 403, an unfalsifiable green.
+ // `rlsProbeSecurity` registers the capability #7665's acceptance
+ // criterion 2 names (object read+edit, owner-scoped SELECT only) and
+ // carries the app's declared default profile through unchanged.
+ const rlsStack = await bootStack(config, { multiTenant, security: rlsProbeSecurity(config) });
  try {
  const adminToken = await rlsStack.signIn();
- const memberToken = await rlsStack.signUp('verify-member@objectstack.test');
- rls = await runRlsProofs(rlsStack, adminToken, memberToken, config);
+ let probeToken: string;
+ let probe: RlsProbeDescriptor;
+ try {
+ const persona = await provisionRlsProbePersona(rlsStack, config);
+ probeToken = persona.token;
+ probe = { label: persona.email, grantedObjects: persona.grantedObjects };
+ } catch (e) {
+ // Prefer failing to falling back (Route & surface ownership §3): a
+ // weaker persona still produces a report, so the degradation is
+ // recorded on the report itself AND counted as a hard failure below.
+ // Silently probing with an ungranted member is the exact false green
+ // this issue exists to remove.
+ probeToken = await rlsStack.signUp('verify-member@objectstack.test');
+ probe = {
+ label: 'verify-member@objectstack.test',
+ degraded: `probe persona provisioning failed: ${(e as Error).message}`,
+ };
+ }
+ rls = await runRlsProofs(rlsStack, adminToken, probeToken, config, { probe });
  } finally {
  await rlsStack.stop();
  }
  }
 
  // Failure contract: a "real" runtime break the app's author must see.
+ // A degraded RLS probe counts: the run reported verdicts it could not have
+ // established, which is worse than no verifier at all.
  const hardFailures =
  crud.summary.createFailed +
  crud.summary.readFailed +
  crud.summary.fidelityGaps +
- (rls?.summary.holes ?? 0);
+ (rls?.summary.holes ?? 0) +
+ (rls?.probe.degraded ? 1 : 0);
 
  if (flags.json) {
  this.log(JSON.stringify({ app: crud.app, config: absolutePath, multiTenant, crud, rls, hardFailures }, null, 2));

packages/qa/dogfood/test/authz-conformance.matrix.ts

@@ -37,7 +37,9 @@ export const AUTHZ_CONFORMANCE: AuthzPrimitive[] = [
  { id: 'rls-read', summary: 'RLS `using` read filter', state: 'enforced',
  enforcement: 'plugin-security/security-plugin.ts computeRlsFilter (AND-injected)', proof: 'rls-fixture.dogfood.test.ts' },
  { id: 'rls-by-id-write', summary: 'by-id write enforcement (#1994)', state: 'enforced',
- enforcement: 'plugin-security/security-plugin.ts pre-image re-read', proof: 'rls-fixture.dogfood.test.ts' },
+ enforcement: 'plugin-security/security-plugin.ts step 2.7 pre-image re-read, composed with computeRlsFilter\'s write-scope DERIVATION (#7665): when no update/delete-class policy applies, the write class is compiled from the caller\'s SELECT narrowing, so the pre-image gate has a predicate to enforce',
+ proof: 'rls-fixture.dogfood.test.ts',
+ note: '[#7685] Re-verified as `enforced`, both halves of the enforcement named because ONLY BOTH hold it. The pre-image re-read alone is a no-op under select-only authoring — that was #7665, and it is why the site had to be re-cited here: ablating the derivation while leaving the pre-image re-read fully in place turns 16 of 20 probed showcase objects into `rls-hole`. The proof is NOT vacuous for this row despite sharing `rls-fixture.dogfood.test.ts` with `rls-read`: since #7665/PR #7792 that file carries a dedicated select-only block whose member set grants FULL CRUD on `rls_note` (so a refusal is the record gate, never the object gate) asserting the by-id PATCH is refused with the row unchanged, plus that an in-scope write still lands. Second, independent measurement: `objectstack verify --rls`, whose probe persona reaches this class since #7685 — 20/23 showcase and 6/6 crm objects PROVEN, 0 holes, and 16 holes under the same ablation.' },
  { id: 'rls-write-check', summary: 'RLS `check` write post-image validation (ADR-0058 D4)', state: 'enforced',
  enforcement: 'plugin-security/security-plugin.ts step 3.6 — compileCelToFilter + matchesFilterCondition against the post-image (fail-closed)',
  note: 'Unit-proven in plugin-security/security-plugin.test.ts (RLS check enforcement); see ADR-0058 D7 ledger.' },
@@ -46,7 +48,8 @@ export const AUTHZ_CONFORMANCE: AuthzPrimitive[] = [
  { id: 'owd-public-read', summary: 'OWD public_read (everyone reads, owner writes)', state: 'enforced',
  enforcement: 'plugin-sharing/sharing-service.ts (read model + canEdit)', proof: 'showcase-public-read-owd.dogfood.test.ts' },
  { id: 'controlled-by-parent', summary: 'master-detail controlled_by_parent', state: 'enforced',
- enforcement: 'plugin-security/security-plugin.ts computeControlledByParentFilter + assertControlledByParentWrite', proof: 'controlled-by-parent.dogfood.test.ts' },
+ enforcement: 'plugin-security/security-plugin.ts computeControlledByParentFilter + assertControlledByParentWrite', proof: 'controlled-by-parent.dogfood.test.ts',
+ note: '[#7685] Re-verified as `enforced` on its OWN evidence, not by association with `rls-by-id-write`. The cited proof is DEDICATED and non-vacuous: `fixtures/cbp-fixture.ts` grants the member full CRUD on BOTH `cbp_account` and `cbp_note`, so every refusal it asserts is the derived record gate rather than the object gate, and the detail carries no authored RLS at all — access is derived from the owner-scoped master. It asserts the derived READ denial, the derived by-id WRITE denial with the row unchanged as ground truth, and that a note under a master the member owns stays readable AND writable (so the guard is not over-blocking). Second measurement: `verify --rls` probes `showcase_invoice_line` — a real `controlled_by_parent` detail — as `rls-consistent`, and it flips to `rls-hole` when the #7665 write-scope derivation its master depends on is ablated.' },
  { id: 'multi-tenant', summary: 'organization isolation', state: 'enforced',
  enforcement: '@objectstack/organizations (enterprise) + Layer 0 tenant wall (plugin-security/tenant-layer.ts, AND-composed ahead of business RLS — ADR-0095 D1)', proof: 'rls-multitenant.dogfood.test.ts' },
  { id: 'multi-tenant-write-postimage', summary: 'Layer 0 tenant post-image check on INSERT + UPDATE (#2937 / Finding 1 — a forged OR re-pointed organization_id cannot cross the tenant wall)', state: 'enforced',

packages/qa/dogfood/test/rls-runner.test.ts

@@ -1,10 +1,25 @@
 // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
 //
 // Unit proof that the RLS runner's #1994 classification is correct — driven by a
-// scripted fake stack so we can exercise the three outcomes deterministically
-// (a live owner-isolated fixture to exercise them end-to-end is the next step).
+// scripted fake stack so we can exercise the outcomes deterministically.
 //
 // The invariant: a user who CANNOT READ a record must not be able to WRITE it.
+//
+// This is also the runner's DETECTOR-LIVENESS oracle. The live fixture
+// (`rls-fixture.dogfood.test.ts`) can no longer plant the hole — #7665 closed
+// the class platform-side, so its red block is now a green regression guard —
+// and `objectstack verify --rls` reports 0 holes against the real apps. Only a
+// scripted stack can still answer "the runner CAN say `rls-hole`", which is what
+// keeps every green above from being a runner that lost the ability to fail.
+//
+// [#7685] The three classifications below are joined by the two the same issue
+// added, and both exist because a NOT-PROVEN object used to be indistinguishable
+// from a proven one:
+// • `probe-blocked` — the OBJECT gate refused the persona, so record scope was
+// never consulted. On the stock showcase that was 11 of 13 "consistent"
+// verdicts, i.e. green that no platform change could have turned red.
+// • target adoption — an admin create the app's own validation rejects no
+// longer cascades its dependents out of the run.
 
 import { describe, it, expect } from 'vitest';
 import { runRlsProofs } from '@objectstack/verify';
@@ -15,24 +30,47 @@ const CONFIG = {
  objects: [{ name: 'note', fields: { name: { type: 'text', required: true } } }],
 };
 
-/** A fake stack: admin always sees/owns; member behaviour is scripted per scenario. */
-function fakeStack(opts: {
+interface FakeOpts {
  memberCanRead: boolean;
  memberWriteMutates: boolean; // does member's PATCH actually change the row?
-}): VerifyStack {
+ /** The object-level gate refuses the member outright (403 before record scope). */
+ objectGateDenies?: boolean;
+ /** Status the admin POST answers with (e.g. 400 — the app's own validation). */
+ adminCreateStatus?: number;
+ /** Rows that already exist on the object, as seed data would. */
+ seeded?: Array<Record<string, unknown>>;
+}
+
+/** A fake stack: admin always sees/owns; member behaviour is scripted per scenario. */
+function fakeStack(opts: FakeOpts): VerifyStack {
  const store: Record<string, any> = {};
+ for (const row of opts.seeded ?? []) store[String(row.id)] = { ...row };
  const json = (body: unknown, status = 200) =>
  new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } });
 
  const apiAs: VerifyStack['apiAs'] = async (token, method, path, body) => {
  const isAdmin = token === 'admin';
- const [, , object, id] = path.split('/'); // /data/<object>/<id>
+ // `/data/<object>[?query]` (list) or `/data/<object>/<id>` (by id)
+ const [, , objectSegment, id] = path.split('/');
+ const object = String(objectSegment).split('?')[0];
+
+ // The object-level gate answers first, for every verb — exactly what made
+ // the grant-less persona's verdicts meaningless (#7685).
+ if (!isAdmin && opts.objectGateDenies) return json({ code: 'PERMISSION_DENIED' }, 403);
+
  if (method === 'POST') {
+ if (opts.adminCreateStatus) return json({ error: 'VALIDATION_FAILED' }, opts.adminCreateStatus);
  const newId = 'rec1';
  store[newId] = { id: newId, ...(body as object) };
  return json({ object, id: newId, record: store[newId] }, 201);
  }
  if (method === 'GET') {
+ if (id === undefined) {
+ // LIST — the runner's reachability probe, and the admin-side source the
+ // cascade-stopper adopts a target from.
+ if (!isAdmin && !opts.memberCanRead) return json({ records: [] });
+ return json({ records: Object.values(store) });
+ }
  if (!isAdmin && !opts.memberCanRead) return json({ error: 'not found' }, 404);
  return json({ object, id, record: store[id] ?? null });
  }
@@ -78,3 +116,63 @@ describe('runRlsProofs #1994 classification', () => {
  expect(report.results[0].status).toBe('member-visible');
  });
 });
+
+describe('[#7685] a NOT-PROVEN object never reads as a pass', () => {
+ it('reports probe-blocked — NOT rls-consistent — when the OBJECT gate refuses the persona', async () => {
+ // The exact shape of the old defect: the persona holds no object grants, so
+ // the 403 that comes back says nothing about record scope. The runner used
+ // to bank it as `rls-consistent`; a platform regression in the by-id write
+ // could not have flipped it, which is what made the green worthless.
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: true, objectGateDenies: true });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
+ expect(report.results[0].status).toBe('probe-blocked');
+ expect(report.summary.consistent).toBe(0);
+ expect(report.summary.proven).toBe(0);
+ expect(report.summary.unproven).toBe(1);
+ expect(report.unproven.map((u) => u.object)).toEqual(['note']);
+ });
+
+ it('counts member-visible and skipped as UNPROVEN, separately from holes', async () => {
+ const stack = fakeStack({ memberCanRead: true, memberWriteMutates: false });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
+ expect(report.summary).toMatchObject({
+ objects: 1, consistent: 0, holes: 0, memberVisible: 1, probeBlocked: 0, skipped: 0,
+ proven: 0, unproven: 1,
+ });
+ });
+
+ it('carries the probe descriptor through to the report so a run\'s REACH is legible', async () => {
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG, {
+ probe: { label: 'probe@test', grantedObjects: 1 },
+ });
+ expect(report.probe).toEqual({ label: 'probe@test', grantedObjects: 1 });
+ });
+});
+
+describe('[#7685] an unsatisfiable admin create does not cascade objects out of the run', () => {
+ it('adopts an existing row when the derived create is rejected, and still probes the object', async () => {
+ const stack = fakeStack({
+ memberCanRead: false,
+ memberWriteMutates: true,
+ adminCreateStatus: 400,
+ seeded: [{ id: 'seed1', name: 'seeded' }],
+ });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
+ expect(report.results[0].target).toBe('adopted');
+ // The point of adopting: the object is PROVEN rather than skipped — and it
+ // is the hole here, which a skip would have hidden.
+ expect(report.results[0].status).toBe('rls-hole');
+ expect(report.summary.skipped).toBe(0);
+ expect(report.summary.holes).toBe(1);
+ });
+
+ it('still skips — with the create failure in the reason — when there is nothing to adopt', async () => {
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false, adminCreateStatus: 400 });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
+ expect(report.results[0].status).toBe('skipped');
+ expect(report.results[0].detail).toContain('admin create failed (400)');
+ expect(report.results[0].detail).toContain('no existing note row to adopt');
+ expect(report.summary.proven).toBe(0);
+ });
+});

packages/verify/src/index.ts

@@ -16,8 +16,22 @@ export type { CrudCase, DerivedAssert, AssertKind, RelationalRef } from './deriv
 export { runCrudVerification, formatReport } from './verify.js';
 export type { VerifyReport, ObjectVerifyResult } from './verify.js';
 
-export { runRlsProofs, formatRlsReport } from './rls.js';
-export type { RlsReport, RlsResult } from './rls.js';
+export {
+ runRlsProofs,
+ formatRlsReport,
+ provisionRlsProbePersona,
+ rlsProbePermissionSet,
+ rlsProbeSecurity,
+ RLS_PROBE_EMAIL,
+} from './rls.js';
+export type {
+ RlsReport,
+ RlsResult,
+ RlsStatus,
+ RlsProbeDescriptor,
+ RlsProbePersona,
+ RlsProofOptions,
+} from './rls.js';
 
 // ADR-0060 — reusable conformance-ledger helper (static complement to the
 // runtime harness): classify every declarable property, fail closed on drift.