docs(qa): R4 coverage sweep — retire both stale waivers, author 8 items for the post-2026-08-08 window

docs/qa/platform-checklist/FOLLOW-UPS.md

@@ -1,4 +1,8 @@
-# Follow-ups — open items from the capability-coverage sweep (2026-08-08)
+# Follow-ups — open items from the capability-coverage sweeps
+
+Standing decision register, one section per sweep (append, never rewrite — a resolved
+row stays with its resolution so the next sweep can see what was already decided).
+Sections §1–§4 are the **2026-08-08** sweep; §5 is the **2026-08-17** re-audit (R4).
 
 Decision register from the capability-coverage sweep. The gap items found by the
 five-angle sweep have all been authored into `areas/*.json` (checklist grew 84 → 170
@@ -67,3 +71,68 @@ these to the showcase would make them runnable:
 - The checklist itself (`areas/*.json`, `coverage.json`, `README.md`, `RUNNER.md`,
  `scripts/check-platform-checklist.mjs`) ships in this branch; this file carries the
  decisions that remain with the maintainer.
+
+## 5. Sweep 2026-08-17 (R4) — re-audit against the window since the 2026-08-08 ledger
+
+Ledger 182 → 190 items; `coverage.json` 28 mapped / 2 waived → **30 mapped / 0 waived**.
+
+### 5a. Docs drift (PD#10 class)
+
+| # | drift | evidence | captured in |
+|---|---|---|---|
+| E1 | **Stale defect note in `packages/spec/liveness/doc.json`.** Its `_note` records "DocSchema declares no `tags`, yet the book-side `include: { tag }` rule and the REST corpus both expect one — the tag rule can currently never match". That defect is **fixed**: `DocSchema` now declares `tags` (`packages/spec/src/system/doc.zod.ts:126`, with the history spelled out in the surrounding comment). The ledger note now describes a bug that no longer exists, which is the same failure class this sweep is correcting in `SWEEP.md`. Outside this card's file surface (`docs/qa/platform-checklist/**`), so it is reported, not edited. | `packages/spec/liveness/doc.json` `_note` vs `packages/spec/src/system/doc.zod.ts:111-127` | — (liveness ledger prose, not a checklist item) |
+
+### 5b. Checked and CLEAN (recorded so the next sweep does not re-derive it)
+
+- **The blank template's `dev` script omits `--ui`, and that is CORRECT — the console is
+ served anyway.** This sweep first read `"dev": "objectstack dev"` against the
+ quick-start's `npx os dev --ui` and inferred that a newcomer running `npm run dev`
+ would land on a server with no console. **Source-checking the chain refuted it**, and
+ the refutation is recorded here because the inference is an easy one to make twice:
+ - `packages/cli/src/commands/serve.ts:221` — `ui: Flags.boolean({ …, default: true,
+ allowNo: true })`. The console is **default-ON** at `serve`; `--no-ui` is the off
+ switch.
+ - `packages/cli/src/commands/dev.ts:370` — `...(flags.ui ? ['--ui'] : [])`. `dev` only
+ ever **adds** `--ui`; it never forwards `--no-ui`. With `dev.ts:69` declaring `ui`
+ with no `default`, an unflagged `dev` spawns `serve` with no ui flag at all, so
+ serve's own default takes over — on.
+ - `content/docs/deployment/cli.mdx:139` says it outright: "`--ui` | Force Console UI on
+ (**already on by default in dev**)", and `:196` documents `--ui / --no-ui … (default
+ on)`.
+
+ So `--ui` on `dev` is a **no-op forwarder**, there is no divergence between the
+ template script and quick-start, and the newcomer is not stranded. What made the wrong
+ reading tempting is `dev.ts:69`'s own flag description ("Enable the bundled Console
+ portal at /_console/"), which reads like the mechanism when it is only a forwarder —
+ not worth a change on its own, but worth knowing before inferring from it.
+ `cli.scaffold-console-first-paint` clause 3 now asserts the two invocations **agree**,
+ with a difference between them as the failure.
+- **No published doc prescribes a retired scaffolder template.** Grepped all of
+ `content/docs/` for `todo` / `compliance` / `content` / `contracts` / `procurement` as
+ `-t` / `--template` operands: zero hits. Every documented invocation is a bare
+ `npx create-objectstack my-app`, which resolves to `blank`, the only entry in
+ `TEMPLATES`. The delisting is cleanly reflected on the docs side.
+- **`npm run validate` and `npx os validate` are the same binary** (the blank template's
+ `validate` script is `objectstack validate`), so the spelling difference between
+ `cli.scaffold-first-run` and the quick-start is not a divergence.
+- **The first-run *commands* are covered** by `cli.scaffold-first-run` (P1, CI-automated)
+ and `studio-authoring.first-run-loop` (P0). Only the seam between them was open, and
+ that is now `cli.scaffold-console-first-paint`. No duplicates were authored.
+
+### 5c. For the maintainer
+
+- **Both surviving coverage waivers were stale** (`book`, `doc`) and are retired in this
+ PR — see `SWEEP.md` "Discipline". Running total: 6 of 6 waivers ever written turned out
+ stale. Worth considering whether a waiver should carry a mandatory re-audit date.
+- **Cost figures in the corrected `SWEEP.md` sentence are this run's own measurements**,
+ which came out higher than the ones supplied with the card (validator ~0.12–0.18 s vs
+ ~80 ms; `pnpm check:platform-checklist` ~2.9 s wall vs 0.165 s). The container was under
+ load from a parallel cold monorepo install, and the `pnpm` wrapper's startup dominates
+ the wall figure. The corrected text therefore records the *direct-node* cost (~0.25 s
+ for self-test + validator) and explicitly warns that timing through `pnpm` measures the
+ wrapper — a durable statement rather than a number that goes stale.
+- **No security-sensitive finding to withhold from this PR.** The two highest-severity
+ items authored (`access-security.no-active-org-session-semantics`,
+ `integration-system.datasource-credential-refusal-matrix`) assert guards that are
+ already shipped and already public in their ADRs/issues; nothing here discloses an
+ unfixed hole.

docs/qa/platform-checklist/SWEEP.md

@@ -6,11 +6,28 @@ executable by the AI session itself. Expected cadence: before each major release
 after any large platform surface lands.
 
 This is the AI-participation half of keeping the checklist current. The other half is
-automatic and needs no human at all: `scripts/check-platform-checklist.mjs` (CI, every
-PR) fails when a new metadata **kind** is unmapped (coverage ratchet) or a spec **enum**
-grows a value a matrix item was pinned against (`enumSource` freshness ratchet). Those
-catch drift on the PR that causes it. This sweep catches the harder class — a whole
-surface or behavior nobody wrote an item for — which no deterministic gate can find.
+**mechanical but still operator-triggered**: `scripts/check-platform-checklist.mjs` fails
+when a new metadata **kind** is unmapped (coverage ratchet) or a spec **enum** grows a
+value a matrix item was pinned against (`enumSource` freshness ratchet).
+
+⚠️ Those ratchets are **NOT wired into per-PR CI** — by maintainer decision they run on a
+periodic **manual** cadence (see this dir's [README.md](./README.md) "Operating cadence",
+and the recorded note at `.github/workflows/lint.yml:426`). So they do **not** catch drift
+on the PR that causes it; they catch it at the **next manual run**. Do not read a green
+ledger as "drift is already gated" — between runs, an unmapped kind or a grown enum sits
+undetected on `main`, which is precisely why a sweep must re-derive from source rather
+than trust the last green check. Cost is not the reason it stays out of CI: measured
+2026-08-17 on a 190-item ledger, the validator alone is ~0.12–0.18 s and the full
+`pnpm check:platform-checklist` (selector self-test + validator) is ~0.25 s of actual
+work, zero-dependency and token-free — the `pnpm` wrapper's own startup dominates any
+figure you measure through it, so time the scripts directly if the number matters. The
+reason is PR independence: the checklist is a QA ledger, not a code gate, so an unrelated
+PR is never blocked by checklist drift.
+
+This sweep catches the harder class — a whole surface or behavior nobody wrote an item
+for — which no deterministic gate can find. Note what that leaves uncovered even in
+principle: **new behavior inside an existing kind** is invisible to BOTH ratchets (no new
+kind, no new enum member), so it is found only here.
 
 ## What a sweep is
 
@@ -58,6 +75,14 @@ single reader would not have covered.
  corrected them against source rather than parroting.
 - **Stale waivers are the highest-value find.** Four of six coverage waivers turned out
  false (api/datasource/mapping/hook all ship stock fixtures). Re-audit every waiver
- each sweep — a waiver is a claim that ages.
+ each sweep — a waiver is a claim that ages. **Confirmed again 2026-08-17**: the two
+ waivers that survived that sweep (`book`, `doc`) were *also* false. Both claimed the
+ kinds are "display-only, no independent runtime behavior to gate beyond serving"; the
+ REST read layer in fact runs a three-member audience vocabulary with an ADR-0049
+ fail-closed branch, a two-layer gate (book audience, then per-entry doc-audience
+ union), a deliberate orphan-group non-leak rule, and locale resolution — an
+ access-control surface. `coverage.json` now carries **zero** waivers. Running total:
+ **6 of 6** waivers ever written turned out stale, which is the strongest argument in
+ this file for auditing them against the runtime rather than against their own prose.
 - **Defects found while grounding go to FOLLOW-UPS.md as expected-fail probes**, not
  silent passes; security-sensitive ones are not filed publicly without the maintainer.

docs/qa/platform-checklist/areas/access-security.json

@@ -1910,6 +1910,115 @@
  "ref": "claude/issue-7680-package-capability-doctrine"
  }
  ]
+ },
+ {
+ "id": "access-security.no-active-org-session-semantics",
+ "title": "A session with no active organization is a LEGAL state with fail-closed semantics: tenant-scoped reads resolve to nothing, tenant-scoped writes are refused loudly naming the missing org, and nothing is ever NULL-stamped",
+ "since": "v17",
+ "status": "active",
+ "revision": 1,
+ "priority": "P0",
+ "surface": "api",
+ "personas": [
+ "a freshly signed-up user on their FIRST session (structurally predates their sys_member row)",
+ "a user removed from their organization",
+ "seeded admin (control — has an active organization)"
+ ],
+ "fixtures": {
+ "app": "showcase",
+ "requires": [
+ "a tenancy-walled deployment (ADR-0095 Layer 0 active on at least one tenant-scoped object) — confirm the posture before scoring, since on an unwalled deployment every clause here is vacuous",
+ "the ability to complete a signup and use the resulting first session before the membership reconciler settles, OR to remove a member and reuse their live session"
+ ],
+ "knownGaps": [
+ "The state is timing-derived on the signup path (better-auth defers the membership write past the signup transaction, ADR-0093), so reproducing it by racing signup is flaky. The DURABLE reproduction is the removed-member path, or a multi-organization deployment whose reconciler binds nobody (ADR-0093 D1 `no-target-org`). Record WHICH producer the run used — a clause proven only on a racy producer is weaker evidence and must say so.",
+ "No stock showcase fixture is known to configure `invite-only` or SSO-JIT-pending; those two producers are blocked(fixture) unless the run configures them."
+ ]
+ },
+ "steps": [
+ "establish the state and PROVE it: obtain a session whose `activeOrganizationId` is null, and cite the evidence (session row / session endpoint payload) — do not infer the state from the symptom you are about to test",
+ "read a tenant-scoped object over the data API with that session",
+ "write (create) a row on a tenant-scoped object with that session",
+ "immediately read back, as the SAME caller, anything the write may have produced",
+ "query the underlying table as admin for rows whose tenant column is NULL, before and after the write attempt",
+ "repeat the read and the write against a NON-tenant-scoped object to bound the blast radius",
+ "sign in and out with the state in force, then inspect the audit ledger rows the session produced (plugin-audit auth-session-audit login/logout)",
+ "run the same read+write as the control admin persona to show the refusal is state-derived, not a blanket denial"
+ ],
+ "acceptance": [
+ {
+ "clause": "tenant-scoped READS resolve to nothing — an empty result set, NOT an error: Layer 0's deny sentinel (RLS_DENY_FILTER) yields zero rows and a 200",
+ "oracle": "api",
+ "verify": "GET returns 200 with an empty collection; a 4xx/5xx here is a FAIL (the read side is deliberately not an error path)",
+ "evidence": "status + body"
+ },
+ {
+ "clause": "tenant-scoped WRITES are refused LOUDLY with a 4xx whose message NAMES the missing active organization — not a generic denial, not a 500",
+ "oracle": "api",
+ "verify": "status is 4xx; the error `code` is drawn from the CLOSED ADR-0112 vocabulary (an invented code is a FAIL of D3) and the message text names the missing active organization",
+ "evidence": "status + full error envelope, quoted"
+ },
+ {
+ "clause": "⛔ NO silent NULL stamping, anywhere: the refused write leaves NO row behind, and no subsystem writes a row whose tenant column is NULL because the caller had no organization to supply",
+ "oracle": "api",
+ "verify": "the admin-side NULL-tenant row count is IDENTICAL before and after the attempt; a row that appears and is invisible to its own creator is the #8208 defect and is a P0 FAIL",
+ "evidence": "both row counts + the diff"
+ },
+ {
+ "clause": "the refusal does not fall OPEN in the other direction: an org-scoped permission holder with no active organization reads and writes NOTHING across tenants (the #8158 class — `scope: 'org'` grants nothing when there is no org)",
+ "oracle": "api",
+ "verify": "as the stateless caller holding an org-scoped permission (e.g. manage_sharing), attempt a cross-tenant read and write; both must be empty/refused, never another tenant's rows",
+ "evidence": "responses + a cross-check that another tenant's rows exist and were not returned"
+ },
+ {
+ "clause": "the audit carve-out is paid for by ORDERING, not by an exemption: audit rows from a first session carry a real tenant — no audit row is NULL-tenanted and therefore permanently invisible to RLS readers (#8245)",
+ "oracle": "api",
+ "verify": "read the login/logout audit rows the session produced as a tenant-scoped reader; they must be visible. A row present in the raw table but invisible to every RLS reader is a FAIL even though nothing errored",
+ "evidence": "raw rows + the RLS-scoped read of the same rows"
+ },
+ {
+ "clause": "the state is LEGAL, not a crash: the session authenticates, non-tenant-scoped surfaces still work, and nothing 500s",
+ "oracle": "api",
+ "verify": "the session's whoami/session endpoint answers 200; a read on a non-tenant-scoped object succeeds; no 5xx anywhere in the run",
+ "evidence": "the response set"
+ },
+ {
+ "clause": "three subsystems agree — sharing, the Layer 0 wall + write path, and the audit ledger give the SAME answer for the SAME state (this item exists because they once gave three contradictory ones on the same working day)",
+ "oracle": "api",
+ "verify": "the run record states each subsystem's observed behavior side by side; any disagreement is the finding, whichever way it leans",
+ "evidence": "the three-way comparison table"
+ }
+ ],
+ "negative": [
+ "fell OPEN: an org-scoped holder with no active organization reading or writing another tenant's rows (#8158) — P0",
+ "fell CLOSED SILENTLY: an HTTP-created record stamped organization_id NULL and immediately invisible to its own creator (#8208) — P0, and the reason the NULL-count clause is measured rather than inferred",
+ "wrote INVISIBLE rows: audit rows carrying a NULL tenant, permanently unreadable through RLS (#8245)",
+ "a 500 or an unhandled rejection on any of the three paths — the state is declared legal, so crashing is a defect",
+ "a refusal whose message does NOT name the missing active organization: it is indistinguishable from an ordinary permission denial and sends the debugger to the wrong layer",
+ "an error code invented for this refusal instead of drawn from the ADR-0112 closed vocabulary"
+ ],
+ "traps": ["auth-state-leak", "eventual-consistency"],
+ "source": [
+ "ADR-0123 (an authenticated session with no active organization — reads resolve to nothing, tenant-scoped writes refused loudly), accepted 2026-08-13",
+ "ADR-0095 D1/W1/W2 (Layer 0 independent always-first AND-composed tenant filter)",
+ "ADR-0105 D1/D2 (posture spectrum, group union wall, empty/absent scope → deny)",
+ "ADR-0093 D1/D2 (membership reconciler; session.create.before resolving activeOrganizationId from sys_member)",
+ "ADR-0112 D3 (closed error-code vocabulary this refusal draws from)",
+ "ADR-0049 (a declared wall the write path does not meet is the defect)",
+ "#8247 (the ruling), from #8158 (fell open) · #8208 (fell closed silently) · #8245 (wrote invisible rows)",
+ "PR #8471 (ADR-0123 + enforcement)",
+ "@objectstack/plugin-security security-plugin.ts step 3.7 + tenant-layer.ts (computeTenantLayer0Filter, RLS_DENY_FILTER)",
+ "@objectstack/plugin-auth auth-manager.ts composeDatabaseHooks + reconcile-membership.ts",
+ "@objectstack/plugin-audit auth-session-audit.ts (login/logout rows)"
+ ],
+ "history": [
+ {
+ "revision": 1,
+ "date": "2026-08-17",
+ "change": "new — the ADR-0123 tenant-semantics wall had ZERO checklist coverage. Neither ratchet could see it: no new metadata kind (it is new behavior across the already-mapped permission/object kinds) and no new enum member. Authored P0 because the ADR records three independently measured production-shaped defects in three subsystems for one state, two of which are silent (a row invisible to its own creator; audit rows invisible to every RLS reader) and therefore invisible to any test that only asserts 'no error'",
+ "ref": "#9299"
+ }
+ ]
  }
  ]
 }