Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions packages/dogfood/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -63,4 +63,89 @@ the spec's zod surface; then run the AI **template corpus** through the harness.
The binding policy — every authorable+live primitive must carry a runtime proof
— is the subject of a dedicated ADR.

## Authorization gate (RLS / #1994)

The capability matrix above proves *data* round-trips. The authorization
dimension proves a record the caller must not touch stays untouched. The
app-agnostic invariant (`src/rls.ts`, `runRlsProofs`):

> **A user who cannot READ a record must not be able to WRITE it.**

[#1994](https://github.com/objectstack-ai/framework/pull/1994) was exactly this
hole: a single-id `update`/`delete` goes straight to `driver.update(object, id)`
and builds no query AST, so the row-level `where` filter the middleware injects
on the *read* path was never applied to *by-id writes*. Any member could PATCH a
record they couldn't even see. The fix is a **pre-image check** in
`plugin-security` (re-read the target row under the write-op RLS filter before
mutating; deny if invisible).

### The runner, and why it needs a fixture

Per object: admin creates a record → a fresh member (`signUp`, no grants) tries
to read it, then mutate it by id → re-read as admin decides if the row actually
changed. Verdicts: `rls-consistent` (can't read **and** can't write — good),
`rls-hole` (can't read **yet** wrote — the #1994 bug), `member-visible`
(member *can* read it — inconclusive, not a cross-owner scenario).

`auto-verify-rls.dogfood.test.ts` runs this over the example apps, but they boot
**single-tenant**, where every object comes back `member-visible` — so the
by-id-write path is never actually exercised. Two ways to create real isolation:

### 1. Owner-scoped fixture — `test/rls-fixture.dogfood.test.ts` (hard gate)

`fixtures/rls-owner-fixture.ts` is a one-object app (`rls_note`) whose member
permission set carries `RLS.ownerPolicy('rls_note', 'created_by')`. The predicate
is `created_by = current_user.id` — keyed on the column the engine stamps on
every record and referencing `current_user.id`, **not**
`current_user.organization_id`, so it survives single-tenant policy stripping. A
fresh member genuinely can't read the admin's note. `bootDogfoodStack` takes a
`security:` override so the fixture's permission set is the member's fallback:

```ts
bootDogfoodStack(rlsFixtureStack, { security: rlsFixtureSecurity(ownerScopedMemberSet) })
```

- **Green gate** (owner policy on `all` ops) → `rls-consistent`. Safe *only*
because the pre-image check enforces the by-id write.
- **Automated red proof** (owner policy on `select` only) → `rls-hole`. Read is
owner-scoped but no write policy applies, so the by-id write lands — the #1994
hole class reproduced at the policy layer, on every CI run. A gate that can't
go red isn't a gate.

**Manual revert proof** (confirms the *fix*, not just the hole class, is what
keeps the green gate green):

```sh
# 1. In plugin-security/src/security-plugin.ts, disable the pre-image check:
# change `if (` to `if ( false &&` at the `(opCtx.operation === 'update' …` block.
pnpm --filter @objectstack/plugin-security build # package resolves to dist
cd packages/dogfood && npx vitest run test/rls-fixture.dogfood.test.ts -t "owner-scoped"
# → rls_note flips to [rls-hole]: "GET 404 yet MUTATED it by id (PATCH 200)".
git checkout -- ../plugins/plugin-security/src/security-plugin.ts && pnpm --filter @objectstack/plugin-security build
```

### 2. Org-scoped / multi-tenant — `test/rls-multitenant.dogfood.test.ts`

Why the single-tenant example-app run is all `member-visible`: `member_default`
scopes rows with a wildcard `tenant_isolation` policy
(`organization_id = current_user.organization_id`), and
`SecurityPlugin.collectRLSPolicies` **strips** every `current_user.organization_id`
policy when the org-scoping plugin is absent — while `member_default` carries no
owner-scoped *read* policy. So the member reads everything. That is the harness
booting single-tenant, **not** a broad-read default of the app (hotcrm's 9
sharing files / `requires: ['sharing']` rely on exactly this org boundary).

Faithful fix — boot multi-tenant so `@objectstack/plugin-org-scoping` registers
before `SecurityPlugin` and the `organization_id` policies apply:

```ts
bootDogfoodStack(crmStack, { multiTenant: true })
```

The dev admin is bound to the seeded default org; a fresh `signUp` member is not,
so the admin's org-scoped records are invisible to them. Empirically CRM flips
from *every object `member-visible`* (single-tenant) to **`4 consistent, 0
holes, 0 member-visible`** (multi-tenant) — the runner now exercises the #1994
by-id-write invariant over org-scoped (not just owner-scoped) RLS.

Runs in CI as the `Dogfood Regression Gate` job (and under `turbo run test`).
3 changes: 2 additions & 1 deletion packages/dogfood/package.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,7 +22,8 @@
"@objectstack/service-analytics": "workspace:*",
"@objectstack/example-crm": "workspace:*",
"@objectstack/example-showcase": "workspace:*",
"@objectstack/plugin-sharing": "workspace:*"
"@objectstack/plugin-sharing": "workspace:*",
"@objectstack/plugin-org-scoping": "workspace:*"
},
"devDependencies": {
"@types/node": "^25.9.3",
Expand Down
30 changes: 29 additions & 1 deletion packages/dogfood/src/harness.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -57,6 +57,24 @@ export interface DogfoodStack {
export interface BootOptions {
/** Override the dev admin credentials the harness signs in with. */
admin?: { email: string; password: string };
/**
* Override the SecurityPlugin instance. Pass a `new SecurityPlugin({...})`
* to carry a custom `fallbackPermissionSet` / extra permission sets — this
* is how the owner-isolated RLS fixture makes a fresh member fall back to a
* permission set that carries `RLS.ownerPolicy(...)` instead of the broad-read
* `member_default`. Defaults to a vanilla `new SecurityPlugin()`.
*/
security?: SecurityPlugin;
/**
* Boot multi-tenant: register `@objectstack/plugin-org-scoping` BEFORE the
* SecurityPlugin so the wildcard `organization_id` RLS policies that ship in
* the default permission sets actually apply (SecurityPlugin probes the
* `org-scoping` service once at start and otherwise STRIPS them — see
* `collectRLSPolicies`). This exercises the org-scoped isolation real apps
* (e.g. hotcrm) rely on, rather than the single-tenant default where every
* tenant policy is stripped and a member sees every row. Default `false`.
*/
multiTenant?: boolean;
}

/**
Expand DownExpand Up@@ -90,7 +108,17 @@ export async function bootDogfoodStack(
await kernel.use(new SettingsServicePlugin());
await kernel.use(new AnalyticsServicePlugin());
await kernel.use(new AuthPlugin({ secret: 'dogfood-regression-secret' }));
await kernel.use(new SecurityPlugin());

// Multi-tenant: org-scoping MUST register BEFORE SecurityPlugin — the latter
// probes the `org-scoping` service exactly once at start and caches it, then
// keeps (vs strips) the wildcard `organization_id` RLS policies accordingly.
// Mirrors `plugin-dev`'s ordering for `OS_MULTI_ORG_ENABLED`.
if (opts.multiTenant) {
const { OrgScopingPlugin } = await import('@objectstack/plugin-org-scoping');
await kernel.use(new OrgScopingPlugin());
}

await kernel.use(opts.security ?? new SecurityPlugin());
// Sharing service — apps that declare `requires: ['sharing']` rely on it for
// record-share grants; without it their RLS/sharing rules are inert and the
// verifier would under-report authorization.
Expand Down
19 changes: 11 additions & 8 deletions packages/dogfood/test/auto-verify-rls.dogfood.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,14 +4,17 @@
// real second user. The runner's hole-detection logic is unit-proven in
// `rls-runner.test.ts`; this exercises it end-to-end against real apps.
//
// HONEST CURRENT STATE: the harness boots single-tenant, so org-scoped RLS is
// stripped and a fresh member falls back to `member_default` (broad read) — so
// every object reports `member-visible` and the #1994 by-id-write path isn't
// exercised here. A hard, revert-provable gate needs an owner-scoped fixture
// (a private-default object + a member permission set carrying RLS.ownerPolicy
// + SecurityPlugin.fallbackPermissionSet) — tracked as the next step. The
// invariant asserted now (zero holes) still guards against a regression that
// makes a member able to mutate a record it cannot read.
// SCOPE: this file is the single-tenant SMOKE over real apps. Single-tenant
// strips the org `tenant_isolation` policy and a fresh member falls back to
// `member_default` (broad read), so every object reports `member-visible` and
// the by-id-write path isn't exercised HERE. That gap is now closed by two
// sibling tests, so the hard gate lives there, not here:
// • rls-fixture.dogfood.test.ts — owner-scoped fixture; green gate +
// automated red proof + a documented manual #1994 revert proof (README).
// • rls-multitenant.dogfood.test.ts — `{ multiTenant: true }`; org-scoped
// (organization_id) isolation, the model real apps like hotcrm use.
// The invariant asserted here (zero holes) still guards against a regression
// that makes a member able to mutate a record it cannot read.

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import crmStack from '@objectstack/example-crm';
Expand Down
111 changes: 111 additions & 0 deletions packages/dogfood/test/fixtures/rls-owner-fixture.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
//
// Owner-isolated RLS fixture — the live, revert-provable #1994 gate.
//
// The example apps and hotcrm boot single-tenant, where the wildcard
// `organization_id` tenant policy is stripped and a fresh member falls back to
// `member_default` (broad read). Result: every object reports `member-visible`,
// so the #1994 cross-owner write invariant ("a user who cannot READ a record
// must not be able to WRITE it") is never actually exercised.
//
// This fixture creates the missing precondition with ZERO dependence on
// org-scoping: a single object `rls_note` whose member permission set carries an
// OWNER policy keyed on `created_by` (`RLS.ownerPolicy`). `created_by` is stamped
// on every record by the engine and the predicate references `current_user.id`
// (not `current_user.organization_id`), so it survives single-tenant stripping.
// A fresh member therefore genuinely CANNOT read a note the admin created — the
// exact cross-owner scenario the runner needs.
//
// Two member permission sets, identical except for the scope of the owner
// policy, drive the green gate and the automated red proof:
//
// ownerScopedMemberSet (operation: 'all') → reads AND writes owner-scoped.
// The #1994 pre-image check enforces the by-id write → `rls-consistent`.
// readOnlyScopedMemberSet (operation: 'select') → reads owner-scoped, but NO
// write policy applies, so the pre-image check has nothing to enforce and
// the by-id write lands → `rls-hole`. This reproduces the #1994 hole CLASS
// at the policy layer ("can't read, yet can write") without touching
// engine code, so the gate's red path is proven on every CI run.

import { defineStack } from '@objectstack/spec';
import { ObjectSchema, Field } from '@objectstack/spec/data';
import { PermissionSetSchema, RLS, type PermissionSet } from '@objectstack/spec/security';
import { SecurityPlugin, securityDefaultPermissionSets } from '@objectstack/plugin-security';

/** The one object under test: a private note, owner-scoped via `created_by`. */
export const RlsNote = ObjectSchema.create({
name: 'rls_note',
label: 'RLS Note',
pluralLabel: 'RLS Notes',
fields: {
name: Field.text({ label: 'Name', required: true }),
body: Field.text({ label: 'Body' }),
},
});

/** A minimal, self-contained app config the dogfood harness can boot. */
export const rlsFixtureStack = defineStack({
manifest: {
id: 'com.dogfood.rls_fixture',
namespace: 'rls',
version: '0.0.0',
type: 'app',
name: 'RLS Owner Fixture',
description: 'Owner-isolated single-object app exercising the #1994 by-id-write invariant.',
},
objects: [RlsNote],
});

/**
* The fallback permission set a fresh member resolves to. Both variants grant
* CRUD on `rls_note` (so the request reaches the RLS layer rather than being
* denied by RBAC) and carry an owner RLS policy keyed on `created_by`. They
* SHARE a name so each can be the `fallbackPermissionSet` of its own boot.
*/
const FIXTURE_MEMBER_SET = 'rls_fixture_member';

const noteCrud = {
rls_note: { allowRead: true, allowCreate: true, allowEdit: true, allowDelete: true },
} as const;

/**
* GREEN. Owner policy on ALL operations — reads and writes are both
* owner-scoped. A member cannot read another user's note, and the #1994
* pre-image check (security-plugin.ts) re-reads the target row under the
* write-op owner filter before a by-id update/delete, so the write is denied.
* Expected runner verdict: `rls-consistent`.
*/
export const ownerScopedMemberSet: PermissionSet = PermissionSetSchema.parse({
name: FIXTURE_MEMBER_SET,
label: 'RLS Fixture Member — owner-scoped (all ops)',
isProfile: true,
objects: noteCrud,
rowLevelSecurity: [RLS.ownerPolicy('rls_note', 'created_by')],
});

/**
* RED. Owner policy on SELECT only — reads stay owner-scoped (member still
* can't see others' notes) but no UPDATE/DELETE policy applies, so
* `computeRlsFilter` returns null for the write op and the pre-image check is
* skipped → the by-id write lands. The member mutated a row it could not read:
* the #1994 hole class. Expected runner verdict: `rls-hole`.
*/
export const readOnlyScopedMemberSet: PermissionSet = PermissionSetSchema.parse({
name: FIXTURE_MEMBER_SET,
label: 'RLS Fixture Member — owner-scoped reads only (#1994 hole)',
isProfile: true,
objects: noteCrud,
rowLevelSecurity: [{ ...RLS.ownerPolicy('rls_note', 'created_by'), operation: 'select' }],
});

/**
* Build a SecurityPlugin whose fallback (for a fresh, grant-less member) is the
* given fixture permission set, layered on top of the real platform defaults so
* the seeded admin keeps `admin_full_access`.
*/
export function rlsFixtureSecurity(memberSet: PermissionSet): SecurityPlugin {
return new SecurityPlugin({
defaultPermissionSets: [...securityDefaultPermissionSets, memberSet],
fallbackPermissionSet: memberSet.name,
});
}
107 changes: 107 additions & 0 deletions packages/dogfood/test/rls-fixture.dogfood.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
//
// The HARD, revert-provable #1994 gate.
//
// `auto-verify-rls.dogfood.test.ts` runs the cross-owner runner over the real
// apps, but single-tenant boot strips the tenant policy so every object is
// `member-visible` — the by-id-write path is never exercised. This test boots a
// purpose-built owner-isolated fixture (see `fixtures/rls-owner-fixture.ts`) so
// a fresh member genuinely cannot read an admin-created record, then asserts the
// runner's verdict in BOTH directions:
//
// • owner policy on ALL ops → `rls-consistent` (green gate). Safe ONLY
// because the #1994 pre-image check enforces the by-id write — revert that
// fix and this flips to `rls-hole` (see README for the manual revert proof,
// and the RED block below for the automated analogue).
// • owner policy on SELECT only → `rls-hole` (automated red proof). The read
// is owner-scoped but no write policy applies, so the by-id write lands —
// the #1994 hole class, reproduced without touching engine code. This proves
// the gate can actually go red.

import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { bootDogfoodStack, type DogfoodStack } from '../src/harness.js';
import { runRlsProofs, formatRlsReport, type RlsReport } from '../src/rls.js';
import {
rlsFixtureStack,
ownerScopedMemberSet,
readOnlyScopedMemberSet,
rlsFixtureSecurity,
} from './fixtures/rls-owner-fixture.js';

describe('objectstack verify RLS: owner-isolated fixture (#1994 hard gate)', () => {
// ── GREEN: the gate that must stay consistent ──────────────────────────────
describe('owner-scoped member set (all ops)', () => {
let stack: DogfoodStack;
let report: RlsReport;
let adminToken: string;
let memberToken: string;

beforeAll(async () => {
stack = await bootDogfoodStack(rlsFixtureStack, {
security: rlsFixtureSecurity(ownerScopedMemberSet),
});
adminToken = await stack.signIn();
memberToken = await stack.signUp('owner-green@verify.test');
report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack);
// eslint-disable-next-line no-console
console.error(formatRlsReport(report));
}, 60_000);

afterAll(async () => {
await stack?.stop();
});

it('precondition: a fresh member CANNOT read an admin-created note (owner RLS reaches the member)', async () => {
const created = await stack.apiAs(adminToken, 'POST', '/data/rls_note', {
name: 'admin note',
body: 'admin-only secret',
});
expect(created.status).toBeLessThan(300);
const cj = (await created.json()) as { id?: string; record?: { id?: string } };
const id = cj.id ?? cj.record?.id;
expect(id, 'admin create should return an id').toBeTruthy();

const bRead = await stack.apiAs(memberToken, 'GET', `/data/rls_note/${id}`);
// Owner-scoped: the member is not the creator, so the row is invisible.
expect(bRead.status, 'member B must not be able to read the admin note').not.toBe(200);
});

it('rls_note is rls-consistent — member can neither read nor mutate it by id', () => {
const note = report.results.find((r) => r.object === 'rls_note');
expect(note?.status, formatRlsReport(report)).toBe('rls-consistent');
});

it('the report has ZERO holes and ZERO member-visible objects (real isolation)', () => {
expect(report.summary.holes, formatRlsReport(report)).toBe(0);
expect(report.summary.memberVisible, formatRlsReport(report)).toBe(0);
expect(report.summary.consistent).toBeGreaterThanOrEqual(1);
});
});

// ── RED: proof the gate can go red on the #1994 hole class ──────────────────
describe('read-only-scoped member set (select only) — #1994 hole reproduced', () => {
let stack: DogfoodStack;
let report: RlsReport;

beforeAll(async () => {
stack = await bootDogfoodStack(rlsFixtureStack, {
security: rlsFixtureSecurity(readOnlyScopedMemberSet),
});
const adminToken = await stack.signIn();
const memberToken = await stack.signUp('owner-red@verify.test');
report = await runRlsProofs(stack, adminToken, memberToken, rlsFixtureStack);
// eslint-disable-next-line no-console
console.error(formatRlsReport(report));
}, 60_000);

afterAll(async () => {
await stack?.stop();
});

it('rls_note is rls-hole — member cannot read it yet mutated it by id', () => {
const note = report.results.find((r) => r.object === 'rls_note');
expect(note?.status, formatRlsReport(report)).toBe('rls-hole');
expect(report.summary.holes).toBe(1);
});
});
});
Loading
Loading