feat(verify): --rls runs one probe persona per declared POSITION, so app-authored narrowing is exercised (#7978)

.changeset/verify-rls-position-personas.md

@@ -0,0 +1,71 @@
+---
+"@objectstack/verify": minor
+"@objectstack/cli": minor
+---
+
+feat(verify): `--rls` runs one probe persona per DECLARED POSITION, so app-authored narrowing is exercised (#7978)
+
+`objectstack verify --rls` proves one invariant — **you cannot mutate what you
+cannot see** (#1994). Since #7685 its probe persona authors its own capability
+(object read+edit, owner-scoped `select` only), which makes the *platform's*
+by-id write gate reachable. That persona holds **no positions** by construction,
+and a policy carrying `positions: [...]` is never applicable to a caller who does
+not hold one — so an app's own position-gated narrowing was never exercised, only
+the platform gate underneath it. That is exactly the authoring shape the real
+#7665 defect wore: an ordinary `contributor` against
+`positions: ['contributor']` rules.
+
+## What changed
+
+- **One persona per position the app DECLARES.** The set is derived from
+ `config.positions` (`declaredPositionNames`), never a list kept in the
+ verifier — a position added to an app is covered without touching this
+ package. Each persona holds that position and nothing else, so its whole
+ capability is what the app itself binds to it (`provisionRlsPositionPersona`
+ writes one `sys_user_position` row; the built-in `everyone` / `guest` anchors
+ are excluded, since no app declares them).
+- **Probe targets are established once and shared** by every persona, so a
+ position costs 4 HTTP calls per object rather than re-deriving and re-creating
+ a record per persona. Each persona mutates with a **distinct short marker**, so
+ "did the row change" stays attributable — and short, because a probe field's
+ `maxLength` would truncate a long marker into a false negative.
+- **Coverage is reported per position, never rolled into one number.**
+ `RlsReport` gains `positionRuns[]` (one summary per position), `totals` (every
+ persona's verdicts summed; the unit is one *object × persona* probe) and
+ `positionCoverage` (`declared` vs `ran`, plus `notRun` for a declared position
+ whose persona could not be provisioned, and a `note` when the app declares no
+ positions at all — "nothing to run" must not read like "nothing to find").
+ `summary` / `results` / `unproven` still describe the base persona exactly as
+ before.
+- **`verify` counts a position persona's holes.** The exit contract reads
+ `totals.holes`, and an unprovisionable declared position is a hard failure for
+ the same reason a degraded base persona is: the run covered less than its
+ numbers read.
+
+## Measured, before → after
+
+| | showcase | crm |
+|:--|:--|:--|
+| before | 23 probes: 20 proven (20 consistent, **0 holes**), 3 unproven (0 probe-blocked, 3 skipped) — exit 0 | 6 probes: 6 proven, 0 holes, 0 unproven — exit 0 |
+| after | 230 probes (base + 9 positions): 35 proven (33 consistent, **2 HOLES**), 195 unproven (54 member-visible, 111 probe-blocked, 30 skipped) — exit 1 | 24 probes (base + 3 positions): 6 proven, 0 holes, 18 unproven (18 probe-blocked) — exit 0 |
+
+Cost: showcase 22s → 50s, crm 10s → 12s (`dogfood-verify` budget is 20 min).
+
+**The two showcase holes are real and are NOT fixed here** — filed as #8059. A
+`contributor` reads `GET 404` on a `showcase_invoice` and still PATCHes it by id;
+the app's check-only `update` policy suppresses #7665's write-scope derivation,
+and the post-image `check` that should have caught it is dropped for
+position-scoped callers. Tuning the probe to keep the run green is the one thing
+this verifier must never do, so `verify --rls` on the showcase now exits 1 until
+#8059 lands.
+
+The new personas are falsifiable, not decorative: ablating the #7665 write-scope
+derivation flips the `contributor` persona's `showcase_task` from
+`rls-consistent` to `rls-hole` (and the base persona's 16, unchanged from #7685's
+measurement).
+
+## No behaviour change outside the verifier
+
+Tooling only: no runtime, spec or enforcement path is touched. `runRlsProofs`'
+existing call shape still works, and consumers that only read the report gain
+fields rather than losing any.

packages/cli/src/commands/verify.ts

@@ -11,10 +11,13 @@ import {
  runRlsProofs,
  formatRlsReport,
  provisionRlsProbePersona,
+ provisionRlsPositionPersona,
+ declaredPositionNames,
  rlsProbeSecurity,
  type VerifyReport,
  type RlsReport,
  type RlsProbeDescriptor,
+ type RlsPositionPersonaInput,
 } from '@objectstack/verify';
 import { loadConfig } from '../utils/config.js';
 
@@ -138,7 +141,31 @@ export default class Verify extends Command {
  degraded: `probe persona provisioning failed: ${(e as Error).message}`,
  };
  }
- rls = await runRlsProofs(rlsStack, adminToken, probeToken, config, { probe });
+ // [#7978] The base persona holds no positions by construction, so an
+ // app policy carrying `positions: [...]` is never applicable to it and
+ // the app's OWN narrowing goes unexercised. Mint one persona per
+ // DECLARED position — derived from the config, never a list written
+ // here — so the position-gated half is probed too. Provisioning lives
+ // on this side because it needs the live stack; the runner re-derives
+ // the intended reach from the config, so a position missing from this
+ // loop reports as `positionCoverage.notRun` instead of quietly
+ // shrinking the run.
+ const positionPersonas: RlsPositionPersonaInput[] = [];
+ const positionFailures: Array<{ position: string; error: string }> = [];
+ for (const position of declaredPositionNames(config)) {
+ try {
+ const persona = await provisionRlsPositionPersona(rlsStack, position);
+ positionPersonas.push({ position, token: persona.token, label: persona.email });
+ } catch (e) {
+ positionFailures.push({ position, error: (e as Error).message });
+ }
+ }
+
+ rls = await runRlsProofs(rlsStack, adminToken, probeToken, config, {
+ probe,
+ positionPersonas,
+ positionFailures,
+ });
  } finally {
  await rlsStack.stop();
  }
@@ -147,12 +174,19 @@ export default class Verify extends Command {
  // Failure contract: a "real" runtime break the app's author must see.
  // A degraded RLS probe counts: the run reported verdicts it could not have
  // established, which is worse than no verifier at all.
+ //
+ // [#7978] `totals`, not `summary`: a hole a POSITION persona found is
+ // exactly as real as one the base persona found — reading `summary` here
+ // would run the fan-out and then throw its findings away. A declared
+ // position that could not be provisioned counts for the same reason a
+ // degraded base persona does: the run covered less than its numbers read.
  const hardFailures =
  crud.summary.createFailed +
  crud.summary.readFailed +
  crud.summary.fidelityGaps +
- (rls?.summary.holes ?? 0) +
- (rls?.probe.degraded ? 1 : 0);
+ (rls?.totals.holes ?? 0) +
+ (rls?.probe.degraded ? 1 : 0) +
+ (rls?.positionCoverage.notRun.length ?? 0);
 
  if (flags.json) {
  this.log(JSON.stringify({ app: crud.app, config: absolutePath, multiTenant, crud, rls, hardFailures }, null, 2));

packages/qa/dogfood/test/rls-runner.test.ts

@@ -20,16 +20,35 @@
 // verdicts, i.e. green that no platform change could have turned red.
 // • target adoption — an admin create the app's own validation rejects no
 // longer cascades its dependents out of the run.
+//
+// [#7978] And by the POSITION fan-out, which this file is the liveness oracle
+// for in exactly the same sense: the live apps report 0 holes for every position
+// persona too, so only a scripted stack can answer "can a position persona still
+// SAY `rls-hole`". A fan-out that cannot fail is decoration, and this file is
+// what stops it becoming that.
 
 import { describe, it, expect } from 'vitest';
-import { runRlsProofs } from '@objectstack/verify';
+import { runRlsProofs, declaredPositionNames } from '@objectstack/verify';
 import type { VerifyStack } from '@objectstack/verify';
 
 const CONFIG = {
  manifest: { id: 'fixture' },
  objects: [{ name: 'note', fields: { name: { type: 'text', required: true } } }],
 };
 
+/** [#7978] The same app, declaring one position — the fan-out's input. */
+const CONFIG_WITH_POSITION = {
+ ...CONFIG,
+ positions: [{ name: 'contributor', label: 'Contributor' }],
+};
+
+/** How one persona behaves. The member token uses the top-level scenario fields. */
+interface PersonaScript {
+ canRead: boolean;
+ writeMutates: boolean;
+ objectGateDenies?: boolean;
+}
+
 interface FakeOpts {
  memberCanRead: boolean;
  memberWriteMutates: boolean; // does member's PATCH actually change the row?
@@ -39,6 +58,10 @@ interface FakeOpts {
  adminCreateStatus?: number;
  /** Rows that already exist on the object, as seed data would. */
  seeded?: Array<Record<string, unknown>>;
+ /** [#7978] Per-token scripts — one per position persona. */
+ personas?: Record<string, PersonaScript>;
+ /** [#7978] Call log, so cost and marker-distinctness are assertable. */
+ calls?: { posts: number; patches: Array<{ token: string; value: unknown }> };
 }
 
 /** A fake stack: admin always sees/owns; member behaviour is scripted per scenario. */
@@ -47,18 +70,26 @@ function fakeStack(opts: FakeOpts): VerifyStack {
  for (const row of opts.seeded ?? []) store[String(row.id)] = { ...row };
  const json = (body: unknown, status = 200) =>
  new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } });
+ const scriptFor = (token: string): PersonaScript =>
+ opts.personas?.[token] ?? {
+ canRead: opts.memberCanRead,
+ writeMutates: opts.memberWriteMutates,
+ objectGateDenies: opts.objectGateDenies,
+ };
 
  const apiAs: VerifyStack['apiAs'] = async (token, method, path, body) => {
  const isAdmin = token === 'admin';
+ const script = scriptFor(token);
  // `/data/<object>[?query]` (list) or `/data/<object>/<id>` (by id)
  const [, , objectSegment, id] = path.split('/');
  const object = String(objectSegment).split('?')[0];
 
  // The object-level gate answers first, for every verb — exactly what made
  // the grant-less persona's verdicts meaningless (#7685).
- if (!isAdmin && opts.objectGateDenies) return json({ code: 'PERMISSION_DENIED' }, 403);
+ if (!isAdmin && script.objectGateDenies) return json({ code: 'PERMISSION_DENIED' }, 403);
 
  if (method === 'POST') {
+ if (opts.calls) opts.calls.posts += 1;
  if (opts.adminCreateStatus) return json({ error: 'VALIDATION_FAILED' }, opts.adminCreateStatus);
  const newId = 'rec1';
  store[newId] = { id: newId, ...(body as object) };
@@ -68,17 +99,18 @@ function fakeStack(opts: FakeOpts): VerifyStack {
  if (id === undefined) {
  // LIST — the runner's reachability probe, and the admin-side source the
  // cascade-stopper adopts a target from.
- if (!isAdmin && !opts.memberCanRead) return json({ records: [] });
+ if (!isAdmin && !script.canRead) return json({ records: [] });
  return json({ records: Object.values(store) });
  }
- if (!isAdmin && !opts.memberCanRead) return json({ error: 'not found' }, 404);
+ if (!isAdmin && !script.canRead) return json({ error: 'not found' }, 404);
  return json({ object, id, record: store[id] ?? null });
  }
  if (method === 'PATCH') {
+ if (opts.calls && !isAdmin) opts.calls.patches.push({ token, value: (body as any)?.name });
  // Admin always writes. Member writes only "land" when the scenario says so
  // (i.e. RLS failed to scope the by-id write — the #1994 bug).
- if (isAdmin || opts.memberWriteMutates) Object.assign(store[id], body as object);
- return json({ object, id, record: store[id] }, isAdmin || opts.memberWriteMutates ? 200 : 403);
+ if (isAdmin || script.writeMutates) Object.assign(store[id], body as object);
+ return json({ object, id, record: store[id] }, isAdmin || script.writeMutates ? 200 : 403);
  }
  return json({}, 405);
  };
@@ -176,3 +208,139 @@ describe('[#7685] an unsatisfiable admin create does not cascade objects out of
  expect(report.summary.proven).toBe(0);
  });
 });
+
+describe('[#7978] declaredPositionNames — the fan-out reads the app, never a list', () => {
+ it('derives the position names from the config, in order, deduplicated', () => {
+ expect(
+ declaredPositionNames({
+ positions: [{ name: 'contributor' }, { name: 'manager' }, { name: 'contributor' }],
+ }),
+ ).toEqual(['contributor', 'manager']);
+ });
+
+ it('covers a position added to the app WITHOUT any change here — the point of deriving', () => {
+ const tomorrow = { positions: [{ name: 'contributor' }, { name: 'field_ops_delegate' }] };
+ expect(declaredPositionNames(tomorrow)).toContain('field_ops_delegate');
+ });
+
+ it('excludes the built-in audience anchors and tolerates an app with no positions', () => {
+ // No app declares `everyone`/`guest` (ADR-0090 D5/D9): every authenticated
+ // member already holds `everyone` — the base persona's own baseline — and
+ // `guest` is the anonymous audience no signed-up persona can hold.
+ expect(declaredPositionNames({ positions: [{ name: 'everyone' }, { name: 'guest' }, { name: 'ops' }] }))
+ .toEqual(['ops']);
+ expect(declaredPositionNames({})).toEqual([]);
+ expect(declaredPositionNames(undefined)).toEqual([]);
+ });
+});
+
+describe('[#7978] a POSITION persona can still say `rls-hole` — fan-out detector liveness', () => {
+ it('flags a HOLE found by a position persona, and rolls it into `totals` where the CLI reads it', async () => {
+ // The base persona is clean; only the persona holding the app's position
+ // can see the defect, because only its policies are position-gated. This is
+ // the whole class #7978 exists to reach — and the assertion that keeps the
+ // fan-out falsifiable rather than decorative.
+ const stack = fakeStack({
+ memberCanRead: false,
+ memberWriteMutates: false,
+ personas: { 'tok-contributor': { canRead: false, writeMutates: true } },
+ });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
+ positionPersonas: [{ position: 'contributor', token: 'tok-contributor', label: 'pos@test' }],
+ });
+
+ expect(report.summary.holes).toBe(0); // base persona: consistent
+ expect(report.positionRuns).toHaveLength(1);
+ expect(report.positionRuns[0].position).toBe('contributor');
+ expect(report.positionRuns[0].results[0].status).toBe('rls-hole');
+ expect(report.positionRuns[0].summary.holes).toBe(1);
+ // `totals` is what `objectstack verify` counts as a hard failure: a hole a
+ // position persona found is exactly as real as one the base persona found.
+ expect(report.totals.holes).toBe(1);
+ expect(report.totals.proven).toBe(2); // base consistent + position hole
+ expect(report.positionCoverage).toMatchObject({ declared: ['contributor'], ran: ['contributor'], notRun: [] });
+ });
+
+ it('reports probe-blocked — NOT a pass — when the object gate refuses a position persona', async () => {
+ // A declared position the app binds no object grants to. Honest: the
+ // by-id-write class was not exercised for it, and it must not read as reach.
+ const stack = fakeStack({
+ memberCanRead: false,
+ memberWriteMutates: false,
+ personas: { 'tok-finance': { canRead: false, writeMutates: true, objectGateDenies: true } },
+ });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
+ positionPersonas: [{ position: 'contributor', token: 'tok-finance', label: 'pos@test' }],
+ });
+ expect(report.positionRuns[0].results[0].status).toBe('probe-blocked');
+ expect(report.positionRuns[0].summary).toMatchObject({ consistent: 0, proven: 0, probeBlocked: 1, unproven: 1 });
+ expect(report.positionRuns[0].unproven.map((u) => u.object)).toEqual(['note']);
+ expect(report.totals.holes).toBe(0);
+ expect(report.totals.proven).toBe(1); // the base persona's verdict, and only that
+ });
+
+ it('gives each persona a DISTINCT marker and creates the probe target only once', async () => {
+ // Two properties in one run. Shared targets are what keeps the fan-out
+ // affordable (one admin create for N personas, not N). Distinct markers are
+ // what keeps it correct: with one shared marker, persona 2's refused write
+ // would re-read persona 1's successful mutation as its own — a fabricated
+ // hole on a platform that is behaving.
+ const calls = { posts: 0, patches: [] as Array<{ token: string; value: unknown }> };
+ const stack = fakeStack({
+ memberCanRead: false,
+ memberWriteMutates: false,
+ calls,
+ personas: {
+ 'tok-a': { canRead: false, writeMutates: true },
+ 'tok-b': { canRead: false, writeMutates: false },
+ },
+ });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
+ positionPersonas: [
+ { position: 'contributor', token: 'tok-a', label: 'a@test' },
+ { position: 'manager', token: 'tok-b', label: 'b@test' },
+ ],
+ });
+
+ expect(calls.posts).toBe(1);
+ const markers = calls.patches.map((p) => p.value);
+ expect(new Set(markers).size).toBe(markers.length);
+ // `tok-b` writes after `tok-a` mutated the row, and is still judged correctly.
+ expect(report.positionRuns[0].results[0].status).toBe('rls-hole');
+ expect(report.positionRuns[1].results[0].status).toBe('rls-consistent');
+ });
+});
+
+describe('[#7978] a position that produced no verdict never reads as a pass', () => {
+ it('says so distinctly when the app declares NO positions — and adds nothing to `proven`', async () => {
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
+ expect(report.positionRuns).toEqual([]);
+ expect(report.positionCoverage.declared).toEqual([]);
+ expect(report.positionCoverage.note).toMatch(/no position personas to run/);
+ // No silent contribution: the totals are exactly the base persona's.
+ expect(report.totals).toEqual(report.summary);
+ });
+
+ it('records a declared position whose persona could not be provisioned, with the reason', async () => {
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
+ positionFailures: [{ position: 'contributor', error: 'no sys_user row for pos@test' }],
+ });
+ expect(report.positionRuns).toEqual([]);
+ expect(report.positionCoverage.notRun).toEqual([
+ { position: 'contributor', reason: 'no sys_user row for pos@test' },
+ ]);
+ // No `note` here: the app DOES declare positions, so "none ran" is a gap,
+ // not the app-declares-nothing case.
+ expect(report.positionCoverage.note).toBeUndefined();
+ });
+
+ it('reports a declared position the caller simply never provisioned a persona for', async () => {
+ const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
+ const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION);
+ expect(report.positionCoverage.notRun).toHaveLength(1);
+ expect(report.positionCoverage.notRun[0]).toMatchObject({ position: 'contributor' });
+ expect(report.positionCoverage.notRun[0].reason).toMatch(/NOT exercised/);
+ });
+});