diff --git a/packages/dogfood/README.md b/packages/dogfood/README.md index bd5b930386..77c6964b94 100644 --- a/packages/dogfood/README.md +++ b/packages/dogfood/README.md @@ -63,4 +63,89 @@ the spec's zod surface; then run the AI **template corpus** through the harness. The binding policy — every authorable+live primitive must carry a runtime proof — is the subject of a dedicated ADR. +## Authorization gate (RLS / #1994) + +The capability matrix above proves *data* round-trips. The authorization +dimension proves a record the caller must not touch stays untouched. The +app-agnostic invariant (`src/rls.ts`, `runRlsProofs`): + +> **A user who cannot READ a record must not be able to WRITE it.** + +[#1994](https://github.com/objectstack-ai/framework/pull/1994) was exactly this +hole: a single-id `update`/`delete` goes straight to `driver.update(object, id)` +and builds no query AST, so the row-level `where` filter the middleware injects +on the *read* path was never applied to *by-id writes*. Any member could PATCH a +record they couldn't even see. The fix is a **pre-image check** in +`plugin-security` (re-read the target row under the write-op RLS filter before +mutating; deny if invisible). + +### The runner, and why it needs a fixture + +Per object: admin creates a record → a fresh member (`signUp`, no grants) tries +to read it, then mutate it by id → re-read as admin decides if the row actually +changed. Verdicts: `rls-consistent` (can't read **and** can't write — good), +`rls-hole` (can't read **yet** wrote — the #1994 bug), `member-visible` +(member *can* read it — inconclusive, not a cross-owner scenario). + +`auto-verify-rls.dogfood.test.ts` runs this over the example apps, but they boot +**single-tenant**, where every object comes back `member-visible` — so the +by-id-write path is never actually exercised. Two ways to create real isolation: + +### 1. Owner-scoped fixture — `test/rls-fixture.dogfood.test.ts` (hard gate) + +`fixtures/rls-owner-fixture.ts` is a one-object app (`rls_note`) whose member +permission set carries `RLS.ownerPolicy('rls_note', 'created_by')`. The predicate +is `created_by = current_user.id` — keyed on the column the engine stamps on +every record and referencing `current_user.id`, **not** +`current_user.organization_id`, so it survives single-tenant policy stripping. A +fresh member genuinely can't read the admin's note. `bootDogfoodStack` takes a +`security:` override so the fixture's permission set is the member's fallback: + +```ts +bootDogfoodStack(rlsFixtureStack, { security: rlsFixtureSecurity(ownerScopedMemberSet) }) +``` + +- **Green gate** (owner policy on `all` ops) → `rls-consistent`. Safe *only* + because the pre-image check enforces the by-id write. +- **Automated red proof** (owner policy on `select` only) → `rls-hole`. Read is + owner-scoped but no write policy applies, so the by-id write lands — the #1994 + hole class reproduced at the policy layer, on every CI run. A gate that can't + go red isn't a gate. + +**Manual revert proof** (confirms the *fix*, not just the hole class, is what +keeps the green gate green): + +```sh +# 1. In plugin-security/src/security-plugin.ts, disable the pre-image check: +# change `if (` to `if ( false &&` at the `(opCtx.operation === 'update' …` block. +pnpm --filter @objectstack/plugin-security build # package resolves to dist +cd packages/dogfood && npx vitest run test/rls-fixture.dogfood.test.ts -t "owner-scoped" +# → rls_note flips to [rls-hole]: "GET 404 yet MUTATED it by id (PATCH 200)". +git checkout -- ../plugins/plugin-security/src/security-plugin.ts && pnpm --filter @objectstack/plugin-security build +``` + +### 2. Org-scoped / multi-tenant — `test/rls-multitenant.dogfood.test.ts` + +Why the single-tenant example-app run is all `member-visible`: `member_default` +scopes rows with a wildcard `tenant_isolation` policy +(`organization_id = current_user.organization_id`), and +`SecurityPlugin.collectRLSPolicies` **strips** every `current_user.organization_id` +policy when the org-scoping plugin is absent — while `member_default` carries no +owner-scoped *read* policy. So the member reads everything. That is the harness +booting single-tenant, **not** a broad-read default of the app (hotcrm's 9 +sharing files / `requires: ['sharing']` rely on exactly this org boundary). + +Faithful fix — boot multi-tenant so `@objectstack/plugin-org-scoping` registers +before `SecurityPlugin` and the `organization_id` policies apply: + +```ts +bootDogfoodStack(crmStack, { multiTenant: true }) +``` + +The dev admin is bound to the seeded default org; a fresh `signUp` member is not, +so the admin's org-scoped records are invisible to them. Empirically CRM flips +from *every object `member-visible`* (single-tenant) to **`4 consistent, 0 +holes, 0 member-visible`** (multi-tenant) — the runner now exercises the #1994 +by-id-write invariant over org-scoped (not just owner-scoped) RLS. + Runs in CI as the `Dogfood Regression Gate` job (and under `turbo run test`). diff --git a/packages/dogfood/package.json b/packages/dogfood/package.json index d90d1be138..91144f6480 100644 --- a/packages/dogfood/package.json +++ b/packages/dogfood/package.json @@ -22,7 +22,8 @@ "@objectstack/service-analytics": "workspace:*", "@objectstack/example-crm": "workspace:*", "@objectstack/example-showcase": "workspace:*", - "@objectstack/plugin-sharing": "workspace:*" + "@objectstack/plugin-sharing": "workspace:*", + "@objectstack/plugin-org-scoping": "workspace:*" }, "devDependencies": { "@types/node": "^25.9.3", diff --git a/packages/dogfood/src/harness.ts b/packages/dogfood/src/harness.ts index 415a610b14..6487ef868f 100644 --- a/packages/dogfood/src/harness.ts +++ b/packages/dogfood/src/harness.ts @@ -57,6 +57,24 @@ export interface DogfoodStack { export interface BootOptions { /** Override the dev admin credentials the harness signs in with. */ admin?: { email: string; password: string }; + /** + * Override the SecurityPlugin instance. Pass a `new SecurityPlugin({...})` + * to carry a custom `fallbackPermissionSet` / extra permission sets — this + * is how the owner-isolated RLS fixture makes a fresh member fall back to a + * permission set that carries `RLS.ownerPolicy(...)` instead of the broad-read + * `member_default`. Defaults to a vanilla `new SecurityPlugin()`. + */ + security?: SecurityPlugin; + /** + * Boot multi-tenant: register `@objectstack/plugin-org-scoping` BEFORE the + * SecurityPlugin so the wildcard `organization_id` RLS policies that ship in + * the default permission sets actually apply (SecurityPlugin probes the + * `org-scoping` service once at start and otherwise STRIPS them — see + * `collectRLSPolicies`). This exercises the org-scoped isolation real apps + * (e.g. hotcrm) rely on, rather than the single-tenant default where every + * tenant policy is stripped and a member sees every row. Default `false`. + */ + multiTenant?: boolean; } /** @@ -90,7 +108,17 @@ export async function bootDogfoodStack( await kernel.use(new SettingsServicePlugin()); await kernel.use(new AnalyticsServicePlugin()); await kernel.use(new AuthPlugin({ secret: 'dogfood-regression-secret' })); - await kernel.use(new SecurityPlugin()); + + // Multi-tenant: org-scoping MUST register BEFORE SecurityPlugin — the latter + // probes the `org-scoping` service exactly once at start and caches it, then + // keeps (vs strips) the wildcard `organization_id` RLS policies accordingly. + // Mirrors `plugin-dev`'s ordering for `OS_MULTI_ORG_ENABLED`. + if (opts.multiTenant) { + const { OrgScopingPlugin } = await import('@objectstack/plugin-org-scoping'); + await kernel.use(new OrgScopingPlugin()); + } + + await kernel.use(opts.security ?? new SecurityPlugin()); // Sharing service — apps that declare `requires: ['sharing']` rely on it for // record-share grants; without it their RLS/sharing rules are inert and the // verifier would under-report authorization. diff --git a/packages/dogfood/test/auto-verify-rls.dogfood.test.ts b/packages/dogfood/test/auto-verify-rls.dogfood.test.ts index 16543f4cb7..b2a9c29a7f 100644 --- a/packages/dogfood/test/auto-verify-rls.dogfood.test.ts +++ b/packages/dogfood/test/auto-verify-rls.dogfood.test.ts @@ -4,14 +4,17 @@ // real second user. The runner's hole-detection logic is unit-proven in // `rls-runner.test.ts`; this exercises it end-to-end against real apps. // -// HONEST CURRENT STATE: the harness boots single-tenant, so org-scoped RLS is -// stripped and a fresh member falls back to `member_default` (broad read) — so -// every object reports `member-visible` and the #1994 by-id-write path isn't -// exercised here. A hard, revert-provable gate needs an owner-scoped fixture -// (a private-default object + a member permission set carrying RLS.ownerPolicy -// + SecurityPlugin.fallbackPermissionSet) — tracked as the next step. The -// invariant asserted now (zero holes) still guards against a regression that -// makes a member able to mutate a record it cannot read. +// SCOPE: this file is the single-tenant SMOKE over real apps. Single-tenant +// strips the org `tenant_isolation` policy and a fresh member falls back to +// `member_default` (broad read), so every object reports `member-visible` and +// the by-id-write path isn't exercised HERE. That gap is now closed by two +// sibling tests, so the hard gate lives there, not here: +// • rls-fixture.dogfood.test.ts — owner-scoped fixture; green gate + +// automated red proof + a documented manual #1994 revert proof (README). +// • rls-multitenant.dogfood.test.ts — `{ multiTenant: true }`; org-scoped +// (organization_id) isolation, the model real apps like hotcrm use. +// The invariant asserted here (zero holes) still guards against a regression +// that makes a member able to mutate a record it cannot read. import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import crmStack from '@objectstack/example-crm'; diff --git a/packages/dogfood/test/fixtures/rls-owner-fixture.ts b/packages/dogfood/test/fixtures/rls-owner-fixture.ts new file mode 100644 index 0000000000..4110ee9b76 --- /dev/null +++ b/packages/dogfood/test/fixtures/rls-owner-fixture.ts @@ -0,0 +1,111 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// Owner-isolated RLS fixture — the live, revert-provable #1994 gate. +// +// The example apps and hotcrm boot single-tenant, where the wildcard +// `organization_id` tenant policy is stripped and a fresh member falls back to +// `member_default` (broad read). Result: every object reports `member-visible`, +// so the #1994 cross-owner write invariant ("a user who cannot READ a record +// must not be able to WRITE it") is never actually exercised. +// +// This fixture creates the missing precondition with ZERO dependence on +// org-scoping: a single object `rls_note` whose member permission set carries an +// OWNER policy keyed on `created_by` (`RLS.ownerPolicy`). `created_by` is stamped +// on every record by the engine and the predicate references `current_user.id` +// (not `current_user.organization_id`), so it survives single-tenant stripping. +// A fresh member therefore genuinely CANNOT read a note the admin created — the +// exact cross-owner scenario the runner needs. +// +// Two member permission sets, identical except for the scope of the owner +// policy, drive the green gate and the automated red proof: +// +// ownerScopedMemberSet (operation: 'all') → reads AND writes owner-scoped. +// The #1994 pre-image check enforces the by-id write → `rls-consistent`. +// readOnlyScopedMemberSet (operation: 'select') → reads owner-scoped, but NO +// write policy applies, so the pre-image check has nothing to enforce and +// the by-id write lands → `rls-hole`. This reproduces the #1994 hole CLASS +// at the policy layer ("can't read, yet can write") without touching +// engine code, so the gate's red path is proven on every CI run. + +import { defineStack } from '@objectstack/spec'; +import { ObjectSchema, Field } from '@objectstack/spec/data'; +import { PermissionSetSchema, RLS, type PermissionSet } from '@objectstack/spec/security'; +import { SecurityPlugin, securityDefaultPermissionSets } from '@objectstack/plugin-security'; + +/** The one object under test: a private note, owner-scoped via `created_by`. */ +export const RlsNote = ObjectSchema.create({ + name: 'rls_note', + label: 'RLS Note', + pluralLabel: 'RLS Notes', + fields: { + name: Field.text({ label: 'Name', required: true }), + body: Field.text({ label: 'Body' }), + }, +}); + +/** A minimal, self-contained app config the dogfood harness can boot. */ +export const rlsFixtureStack = defineStack({ + manifest: { + id: 'com.dogfood.rls_fixture', + namespace: 'rls', + version: '0.0.0', + type: 'app', + name: 'RLS Owner Fixture', + description: 'Owner-isolated single-object app exercising the #1994 by-id-write invariant.', + }, + objects: [RlsNote], +}); + +/** + * The fallback permission set a fresh member resolves to. Both variants grant + * CRUD on `rls_note` (so the request reaches the RLS layer rather than being + * denied by RBAC) and carry an owner RLS policy keyed on `created_by`. They + * SHARE a name so each can be the `fallbackPermissionSet` of its own boot. + */ +const FIXTURE_MEMBER_SET = 'rls_fixture_member'; + +const noteCrud = { + rls_note: { allowRead: true, allowCreate: true, allowEdit: true, allowDelete: true }, +} as const; + +/** + * GREEN. Owner policy on ALL operations — reads and writes are both + * owner-scoped. A member cannot read another user's note, and the #1994 + * pre-image check (security-plugin.ts) re-reads the target row under the + * write-op owner filter before a by-id update/delete, so the write is denied. + * Expected runner verdict: `rls-consistent`. + */ +export const ownerScopedMemberSet: PermissionSet = PermissionSetSchema.parse({ + name: FIXTURE_MEMBER_SET, + label: 'RLS Fixture Member — owner-scoped (all ops)', + isProfile: true, + objects: noteCrud, + rowLevelSecurity: [RLS.ownerPolicy('rls_note', 'created_by')], +}); + +/** + * RED. Owner policy on SELECT only — reads stay owner-scoped (member still + * can't see others' notes) but no UPDATE/DELETE policy applies, so + * `computeRlsFilter` returns null for the write op and the pre-image check is + * skipped → the by-id write lands. The member mutated a row it could not read: + * the #1994 hole class. Expected runner verdict: `rls-hole`. + */ +export const readOnlyScopedMemberSet: PermissionSet = PermissionSetSchema.parse({ + name: FIXTURE_MEMBER_SET, + label: 'RLS Fixture Member — owner-scoped reads only (#1994 hole)', + isProfile: true, + objects: noteCrud, + rowLevelSecurity: [{ ...RLS.ownerPolicy('rls_note', 'created_by'), operation: 'select' }], +}); + +/** + * Build a SecurityPlugin whose fallback (for a fresh, grant-less member) is the + * given fixture permission set, layered on top of the real platform defaults so + * the seeded admin keeps `admin_full_access`. + */ +export function rlsFixtureSecurity(memberSet: PermissionSet): SecurityPlugin { + return new SecurityPlugin({ + defaultPermissionSets: [...securityDefaultPermissionSets, memberSet], + fallbackPermissionSet: memberSet.name, + }); +} diff --git a/packages/dogfood/test/rls-fixture.dogfood.test.ts b/packages/dogfood/test/rls-fixture.dogfood.test.ts new file mode 100644 index 0000000000..0726344fac --- /dev/null +++ b/packages/dogfood/test/rls-fixture.dogfood.test.ts @@ -0,0 +1,107 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// The HARD, revert-provable #1994 gate. +// +// `auto-verify-rls.dogfood.test.ts` runs the cross-owner runner over the real +// apps, but single-tenant boot strips the tenant policy so every object is +// `member-visible` — the by-id-write path is never exercised. This test boots a +// purpose-built owner-isolated fixture (see `fixtures/rls-owner-fixture.ts`) so +// a fresh member genuinely cannot read an admin-created record, then asserts the +// runner's verdict in BOTH directions: +// +// • owner policy on ALL ops → `rls-consistent` (green gate). Safe ONLY +// because the #1994 pre-image check enforces the by-id write — revert that +// fix and this flips to `rls-hole` (see README for the manual revert proof, +// and the RED block below for the automated analogue). +// • owner policy on SELECT only → `rls-hole` (automated red proof). The read +// is owner-scoped but no write policy applies, so the by-id write lands — +// the #1994 hole class, reproduced without touching engine code. This proves +// the gate can actually go red. + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { bootDogfoodStack, type DogfoodStack } from '../src/harness.js'; +import { runRlsProofs, formatRlsReport, type RlsReport } from '../src/rls.js'; +import { + rlsFixtureStack, + ownerScopedMemberSet, + readOnlyScopedMemberSet, + rlsFixtureSecurity, +} from './fixtures/rls-owner-fixture.js'; + +describe('objectstack verify RLS: owner-isolated fixture (#1994 hard gate)', () => { + // ── GREEN: the gate that must stay consistent ────────────────────────────── + describe('owner-scoped member set (all ops)', () => { + let stack: DogfoodStack; + let report: RlsReport; + let adminToken: string; + let memberToken: string; + + beforeAll(async () => { + stack = await bootDogfoodStack(rlsFixtureStack, { + security: rlsFixtureSecurity(ownerScopedMemberSet), + }); + adminToken = await stack.signIn(); + memberToken = await stack.signUp('owner-green@verify.test'); + report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack); + // eslint-disable-next-line no-console + console.error(formatRlsReport(report)); + }, 60_000); + + afterAll(async () => { + await stack?.stop(); + }); + + it('precondition: a fresh member CANNOT read an admin-created note (owner RLS reaches the member)', async () => { + const created = await stack.apiAs(adminToken, 'POST', '/data/rls_note', { + name: 'admin note', + body: 'admin-only secret', + }); + expect(created.status).toBeLessThan(300); + const cj = (await created.json()) as { id?: string; record?: { id?: string } }; + const id = cj.id ?? cj.record?.id; + expect(id, 'admin create should return an id').toBeTruthy(); + + const bRead = await stack.apiAs(memberToken, 'GET', `/data/rls_note/${id}`); + // Owner-scoped: the member is not the creator, so the row is invisible. + expect(bRead.status, 'member B must not be able to read the admin note').not.toBe(200); + }); + + it('rls_note is rls-consistent — member can neither read nor mutate it by id', () => { + const note = report.results.find((r) => r.object === 'rls_note'); + expect(note?.status, formatRlsReport(report)).toBe('rls-consistent'); + }); + + it('the report has ZERO holes and ZERO member-visible objects (real isolation)', () => { + expect(report.summary.holes, formatRlsReport(report)).toBe(0); + expect(report.summary.memberVisible, formatRlsReport(report)).toBe(0); + expect(report.summary.consistent).toBeGreaterThanOrEqual(1); + }); + }); + + // ── RED: proof the gate can go red on the #1994 hole class ────────────────── + describe('read-only-scoped member set (select only) — #1994 hole reproduced', () => { + let stack: DogfoodStack; + let report: RlsReport; + + beforeAll(async () => { + stack = await bootDogfoodStack(rlsFixtureStack, { + security: rlsFixtureSecurity(readOnlyScopedMemberSet), + }); + const adminToken = await stack.signIn(); + const memberToken = await stack.signUp('owner-red@verify.test'); + report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack); + // eslint-disable-next-line no-console + console.error(formatRlsReport(report)); + }, 60_000); + + afterAll(async () => { + await stack?.stop(); + }); + + it('rls_note is rls-hole — member cannot read it yet mutated it by id', () => { + const note = report.results.find((r) => r.object === 'rls_note'); + expect(note?.status, formatRlsReport(report)).toBe('rls-hole'); + expect(report.summary.holes).toBe(1); + }); + }); +}); diff --git a/packages/dogfood/test/rls-multitenant.dogfood.test.ts b/packages/dogfood/test/rls-multitenant.dogfood.test.ts new file mode 100644 index 0000000000..681a1a8b73 --- /dev/null +++ b/packages/dogfood/test/rls-multitenant.dogfood.test.ts @@ -0,0 +1,62 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. +// +// Org-scoped (#1994) cross-tenant gate — the faithful counterpart to the +// single-tenant smoke in `auto-verify-rls.dogfood.test.ts`. +// +// THE INVESTIGATION (why the single-tenant run showed all `member-visible`): +// `member_default` scopes rows with a wildcard `tenant_isolation` policy +// (`organization_id = current_user.organization_id`). When the org-scoping +// plugin is absent, SecurityPlugin.collectRLSPolicies STRIPS every policy whose +// predicate references `current_user.organization_id` (security-plugin.ts) — and +// `member_default` carries NO owner-scoped READ policy — so a fresh member can +// read every row. That is the `member-visible` verdict: not a broad-read default +// of the app, but the harness booting single-tenant. Apps like hotcrm (9 sharing +// files, `requires: ['sharing']`) rely on exactly this org boundary, so a +// single-tenant boot under-reports their authorization model. +// +// THE FIX: boot with `{ multiTenant: true }` so OrgScopingPlugin registers +// before SecurityPlugin and the wildcard `organization_id` policies APPLY. The +// dev admin is bound to the seeded default organization; a fresh `signUp` member +// is not in it, so the admin's org-scoped records are invisible to them — the +// real cross-tenant scenario. The runner then exercises the #1994 by-id-write +// invariant over org-scoped (not owner-scoped) RLS: member can't read AND the +// pre-image check denies the by-id write → `rls-consistent`. +// +// Empirically (CRM): single-tenant → every object `member-visible`; multi-tenant +// → `rls-consistent` with zero holes. This test asserts that faithful state. + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import crmStack from '@objectstack/example-crm'; +import { bootDogfoodStack, type DogfoodStack } from '../src/harness.js'; +import { runRlsProofs, formatRlsReport, type RlsReport } from '../src/rls.js'; + +describe('objectstack verify RLS: CRM multi-tenant (#1994 org-scoped)', () => { + let stack: DogfoodStack; + let report: RlsReport; + + beforeAll(async () => { + stack = await bootDogfoodStack(crmStack as never, { multiTenant: true }); + const adminToken = await stack.signIn(); + const memberToken = await stack.signUp('member-mt@verify.test'); + report = await runRlsProofs(stack, adminToken, memberToken, crmStack); + // eslint-disable-next-line no-console + console.error(formatRlsReport(report)); + }, 90_000); + + afterAll(async () => { + await stack?.stop(); + }); + + it('org-scoping engaged: at least one object is rls-consistent (NOT member-visible)', () => { + // The single-tenant run reports every object `member-visible`; under + // multi-tenant the tenant policy applies, so cross-tenant objects flip to + // `rls-consistent`. ≥1 consistent proves the org boundary is genuinely + // exercised — the difference single-tenant could never show. + expect(report.summary.consistent, formatRlsReport(report)).toBeGreaterThanOrEqual(1); + }); + + it('has ZERO by-id-write RLS holes across org boundaries (#1994 invariant)', () => { + const holes = report.results.filter((r) => r.status === 'rls-hole'); + expect(holes, formatRlsReport(report)).toHaveLength(0); + }); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9d5393c0dd..cd91c9c19e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -844,6 +844,9 @@ importers: '@objectstack/plugin-hono-server': specifier: workspace:* version: link:../plugins/plugin-hono-server + '@objectstack/plugin-org-scoping': + specifier: workspace:* + version: link:../plugins/plugin-org-scoping '@objectstack/plugin-security': specifier: workspace:* version: link:../plugins/plugin-security