diff --git a/docs/qa/platform-checklist/RUNNER.md b/docs/qa/platform-checklist/RUNNER.md index d3601f6439..c0e1e82b94 100644 --- a/docs/qa/platform-checklist/RUNNER.md +++ b/docs/qa/platform-checklist/RUNNER.md @@ -115,11 +115,31 @@ contradicts it, and correct it here when it does. section at all.** `runRlsProofs` runs only behind the flag (`packages/cli/src/commands/verify.ts`: `rls: Flags.boolean({ default: false })`, the proofs sit inside `if (flags.rls)`, and the report prints `if (rls)`). **Check:** the - last block of a bare run is the CRUD summary — `── 15 verified, 0 gaps, 0 FAILED, 1 + last block of a bare run is the CRUD summary — `── 16 verified, 0 gaps, 0 FAILED, 1 needs-fixture, 7 skipped` on stock showcase — with no `PROVEN`/`HOLES` line anywhere. - Adding `--rls` appends the RLS block: `20 PROVEN (20 consistent, 0 HOLES)` over 23 - objects, plus `9 of 9 declared position(s) probed`. ⛔ Do not cite plain `verify` - output as the oracle for an RLS clause — that run never consulted one. + Adding `--rls` appends the RLS block. ⛔ Do not cite plain `verify` output as the + oracle for an RLS clause — that run never consulted one. + + **The discriminator is the PRESENCE of a `PROVEN`/`HOLES` line, never its digits.** + The counts move with the seed set and are recorded here only as the shape to expect: + the CRUD tail read `15 verified` through 2026-08-18 and reads `16` on 17.1.0 because + the showcase gained one object (`client-brief.object.ts`, added between the two sweep + subjects); a run whose digits differ has a different seed, not a regression. Re-derive + the number, do not file it. + + ⚠️ **The `--rls` block prints TWO summary lines in DIFFERENT units — do not compare + them.** `formatRlsReport` (`packages/verify/src/rls.ts`) emits a per-persona + `── PROVEN ( consistent, HOLES) · NOT PROVEN (…)` for the probe + persona, then — once position personas have run — a total line + `══ all personas: … [unit: one object × persona probe]`. The first counts OBJECTS for + one persona; the second counts object × persona PAIRS across all of them, and the + source stamps that unit into the line precisely so the two are not read as one number + moving. Measured on 17.1.0: the total line reads + `all personas: 38 PROVEN (38 consistent, 0 HOLES) · 226 NOT PROVEN`, alongside + `9 of 9 declared position(s) probed`. An older note here recorded + `20 PROVEN … over 23 objects` — that was the per-persona line, so it is **not + comparable** to the total rather than merely stale. Name which of the two lines you + are quoting whenever you cite either. - **Playwright needs an explicit `executablePath` on these containers.** `@playwright/test` 1.62.1 resolves chromium build **1234**; only **1194** is installed. @@ -147,13 +167,45 @@ contradicts it, and correct it here when it does. and App declares no `id` of its own. `?id=com.example.showcase` (the package id, from `objectstack.config.ts`) returns `{"items":[]}` — which reads exactly like "the app metadata is gone", the highest-value false P0 shape there is. Real names: `showcase_app` - (showcase), `setup` / `studio` / `account` (platform built-ins). ⚠️ **An empty - `items` has two distinct causes** — a wrong spelling, or an app that is genuinely not - installed. `studio` is defined (`packages/platform-objects/src/apps/studio.app.ts`) but - the showcase does **not** install it, so `?id=studio` is legitimately empty there; a - stock admin list is `["showcase_app","setup","account"]`. **Check:** fetch - `/api/v1/meta/app` with no query first and read the names it actually returns, then - filter. + (showcase), `setup` / `account` (the platform built-ins a stock boot registers). ⚠️ + **An empty `items` has two distinct causes** — a wrong spelling, or an app that is + genuinely not registered. `studio` is the standing instance of the second cause: the + App document is defined (`packages/platform-objects/src/apps/studio.app.ts`) and + packaged (`packages/apps/studio`, `com.objectstack.studio`), but **no stock boot loads + it, by design** — both boot paths skip it with the identical note, "`@objectstack/studio` + is intentionally NOT default-loaded — the console ships a dedicated Studio surface at + `/_console/studio//`, so Studio no longer needs to exist as a navigable + app tile" (`packages/plugins/plugin-dev/src/dev-plugin.ts`, the ADR-0048 app-package + loop; `packages/cli/src/commands/serve.ts`, same loop — both register only + `@objectstack/setup` and `@objectstack/account`). So `?id=studio` is legitimately empty + on every stock boot, not just on the showcase, and a stock admin list is + `["showcase_app","setup","account"]`. ⛔ Do not file the absence as a missing built-in, + and do not install the package to "restore" it — that changes what the boot ships. + **Check:** fetch `/api/v1/meta/app` with no query first and read the names it actually + returns, then filter. + +- **An isolated boot on a non-3000 port breaks the showcase's self-pinging connectors + unless you export `OS_PORT` — `-p` alone is not enough, and the symptom impersonates an + egress block.** The showcase's REST/OpenAPI connector instances point at the running + server itself, resolved by `resolveShowcaseSelfUrl()` + (`examples/app-showcase/src/system/self-url.ts`) in this order: `SHOWCASE_SELF_URL` → + `OS_PORT` → `PORT` → the literal `http://127.0.0.1:3000`. Those are **environment** + reads. The CLI's `-p` flag is only a *default sourced from* `OS_PORT` + (`packages/cli/src/commands/serve.ts`: `port: Flags.string({ char: 'p', … default: + readEnvWithDeprecation('OS_PORT','PORT') ?? '3000' })`) and nothing writes the variable + back, so the dogfood §0 isolated boot — `objectstack dev … -p ` with the env + untouched — listens on `` while every declarative connector dials **3000**. Every + dispatch then fails `connector_action(showcase_status_api.request) failed: fetch failed`. + ⚠️ **That string is not evidence of a sandbox network restriction**, and reading it as + one is the expensive mistake: the module's own header records that #7516 burned a run + proving it was an address problem by putting a TCP forwarder on 3000, and #7538 fixed + the resolution but could not fix a boot that never exports the variable. **Check:** + before scoring any connector clause, `GET /api/v1/automation/connectors` and read the + instance `baseUrl` — if it says `:3000` and your server does not listen there, the + fixture is misconfigured, not blocked. **Fix:** boot with `OS_PORT=` exported (or + `SHOWCASE_SELF_URL=http://127.0.0.1:`) *in addition to* `-p `. ⛔ Do not + record a connector clause as `blocked(environment)` on the strength of `fetch failed` + alone. - **`ss` is not installed in these containers — read liveness with `curl`, never a socket table.** `ss` and `netstat` are both absent (`command not found`); `lsof` and `fuser` @@ -250,6 +302,27 @@ contradicts it, and correct it here when it does. by name and exits **1**, where it used to exit **0** in silence — the entry point is `packages/cli/bin/run.js`. +- **23 items carry an `automated.ref` that lives ENTIRELY in the `objectui` repo — from + this checkout they are neither runnable nor pin-evidenced.** Their refs name only + objectui paths (`e2e/live/*.spec.ts`, `e2e/import-console/*`, `e2e/import-harness/*`, + and unit suites under `packages/plugin-grid`, `plugin-gantt`, `plugin-calendar`, + `plugin-form`, `fields`, `core`, `app-shell`). This repository builds no console bundle + and holds none of those specs, so a run here can neither execute the pin nor cite its + output. **Every one of those items now says so in its own `fixtures.knownGaps`** — the + rediscovery this fact exists to stop (each sweep re-derived it from scratch). ⚠️ The + distinction that matters is *exclusive*: items whose `automated.ref` names an + objectui spec **alongside** a framework pin (`packages/qa/dogfood/**`, + `packages/objectql/**`) are partly runnable here — run the framework half and score the + objectui half separately, never one for the other. + + **Two honest options, and the run record must say which it took:** (a) drive the item + **by hand** in the browser against a live boot and score it as a manual browser run — + the `automated` field does not make it exempt; or (b) run the pin inside an **objectui + checkout at the pinned revision** and cite that output, **naming the revision** (the + 17.1.0 sweep subject was `9a3daf8d37ad`). ⛔ Never record such an item as covered on the + strength of the `automated` field alone: an unrun pin is a claim, not evidence — which + is the same "ticking on a label" failure clause-grained acceptance exists to prevent. + ### Trap vocabulary (`traps` field) | trap | what it fakes | counter | diff --git a/docs/qa/platform-checklist/areas/api-backend.json b/docs/qa/platform-checklist/areas/api-backend.json index 4ff7fd4711..14d86f9507 100644 --- a/docs/qa/platform-checklist/areas/api-backend.json +++ b/docs/qa/platform-checklist/areas/api-backend.json @@ -441,7 +441,7 @@ "title": "Sampled endpoints return the standard error envelope with ledgered codes — no invented codes, no retired keys, statuses match the map", "since": "v16", "status": "active", - "revision": 2, + "revision": 3, "priority": "P1", "surface": "api", "personas": [ @@ -453,7 +453,7 @@ "app": "showcase", "requires": [ "showcase_invoice.name is required:true — the cheapest deterministic VALIDATION_FAILED", - "a member persona lacking showcase_announcement create (member_default: create false) — the deterministic 403" + "a member persona lacking showcase_announcement create (member_default: create false) — the deterministic 403. Stock showcase SEEDS two loginable non-admin personas for exactly this: Mei Phone (phone.demo@example.com) and Ada Auditor (auditor.demo@example.com), both with DEMO_PERSONA_PASSWORD = 'showcase123' (examples/app-showcase/src/security/demo-personas.ts, provisioned by seed-approval-demo.ts on every dev boot). Driven as Mei, POST /api/v1/data/showcase_announcement answers 403 PERMISSION_DENIED — a ledgered code — so the 403 family is reachable with no provisioning at all. POST /api/v1/auth/sign-up/email also works on a stock dev boot if a THIRD identity is wanted" ] }, "steps": [ @@ -533,7 +533,13 @@ "change": "new — error-envelope conformance sampling grounded in the two-tier code ledger, per the deep-test contract", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 1 and steps 8/9 at ErrorCode, the canonical ADR-0112 D4 union export. The item asked the runner to hand-union StandardErrorCode with ERROR_CODE_LEDGER, but the latter is a Record of package name to code array rather than a flat code list, and three of the item's own sampled codes (VALIDATION_FAILED, UNSUPPORTED_QUERY_PARAM, BATCH_NOT_ATOMIC) are not in StandardErrorCode at all — they resolve only through REGISTERED_ERROR_CODES. error-code-ledger.zod.ts already exports ErrorCode as exactly that union, so the clause now names one export instead of prescribing a union the runner has to rebuild (#9417)", "ref": "#9386" } + { "revision": 2, "date": "2026-08-18", "change": "re-pointed clause 1 and steps 8/9 at ErrorCode, the canonical ADR-0112 D4 union export. The item asked the runner to hand-union StandardErrorCode with ERROR_CODE_LEDGER, but the latter is a Record of package name to code array rather than a flat code list, and three of the item's own sampled codes (VALIDATION_FAILED, UNSUPPORTED_QUERY_PARAM, BATCH_NOT_ATOMIC) are not in StandardErrorCode at all — they resolve only through REGISTERED_ERROR_CODES. error-code-ledger.zod.ts already exports ErrorCode as exactly that union, so the clause now names one export instead of prescribing a union the runner has to rebuild (#9417)", "ref": "#9386" }, + { + "revision": 3, + "date": "2026-08-21", + "change": "named the seeded demo personas in fixtures.requires. The 17.1.0 sweep recorded this item's 403 family as unreachable on stock fixtures and scored the item partial, having assumed a runner gets only the dev admin; the filer withdrew that in the same wave after driving Mei to a real 403 PERMISSION_DENIED (6/6 families, superseding #10233 on that point). The personas were always there — the item just never said so, which is what made the wrong assumption cheap. Requirement text now carries both persona emails, the well-known password constant and its source file, so the next runner does not repeat it (#10236 A5)", + "ref": "#10236" + } ] }, { @@ -1046,7 +1052,7 @@ "title": "Filter comparand conformance: only the accepted comparand TYPES pass the where-door, a bigint past the exact-integer limit is refused, and a dotted head is classified rather than guessed", "since": "v17", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "api", "personas": ["seeded admin"], @@ -1056,14 +1062,15 @@ "an object with a scalar field, a relation (lookup/master-detail) field, and — if the showcase ships one — a formula/virtual field, so all three dotted-head classes are reachable" ], "knownGaps": [ - "If no formula/virtual field exists on a filterable object in the stock seeds, the `virtual` head class is blocked(fixture) — record it rather than scoring the class from the relation case." + "If no formula/virtual field exists on a filterable object in the stock seeds, the `virtual` head class is blocked(fixture) — record it rather than scoring the class from the relation case.", + "The bigint past-limit refusal (clause 2) has NO REST arm at all — it is not a thin fixture, it is a door property. `normalizeFilterComparandTypes` refuses on `typeof value === 'bigint'`, and JSON.parse produces `number` for every numeric literal, so the arm is unreachable through any JSON transport regardless of magnitude. Score clause 2 at the engine/driver door and record the REST verdict as not-applicable; a REST 200 there is neither evidence of enforcement nor of a leak." ] }, "steps": [ "read ACCEPTED_FILTER_COMPARAND_TYPES in packages/spec/src/data/filter-comparand-type.ts and enumerate the current members — the variants matrix below is pinned to that export, so a mismatch is a validator failure, not a judgement call", "for EACH accepted comparand type, issue a data-API read whose `where` carries a comparand of that type against a compatible field, and record the answer set", "issue a read whose comparand is a type NOT in the accepted set (e.g. a plain object, an array where the operator takes a scalar, undefined, a function-shaped value) and capture the refusal", - "probe the bigint boundary: a bigint comparand at FILTER_COMPARAND_BIGINT_EXACT_LIMIT (2n**53n) and one past it", + "probe the bigint boundary AT AN IN-PROCESS DOOR: pass a real JS bigint at FILTER_COMPARAND_BIGINT_EXACT_LIMIT (2n**53n) and one past it through objectql/engine (or replay FILTER_COMPARAND_TYPE_CASES against a driver) — not over REST, where JSON.parse has already turned the literal into a number before the door is consulted", "issue reads with dotted filter heads of each class — a relation head, a virtual/formula head, and a scalar head (a dot on a field that is not traversable) — and record how each is classified", "cross-check the SAME filters through a second door (the engine/objectql path or a view filter) so the refusal is proven at the contract, not at one transport" ], @@ -1081,10 +1088,10 @@ "evidence": "status + message + the returned row count vs the unfiltered count" }, { - "clause": "the bigint exact-integer limit is enforced at the boundary: a value inside the limit is accepted, one past it is refused rather than silently losing precision", - "oracle": "api", - "verify": "both requests; the past-limit case must refuse — a 200 whose comparand was rounded is a silent-wrong-answer FAIL", - "evidence": "both responses" + "clause": "the bigint exact-integer limit is enforced at the boundary — a value inside the limit narrows and is accepted, one past it is refused rather than silently losing precision. ⚠️ DOOR-SPECIFIC: assertable only at an IN-PROCESS door (objectql/engine, or a driver replaying the conformance table); it is structurally unreachable over REST", + "oracle": "test", + "verify": "run packages/objectql/src/engine-comparand-type-door.test.ts (the engine door) plus the driver suites that replay FILTER_COMPARAND_TYPE_CASES (driver-memory / driver-sql / driver-sqlite-wasm / driver-mongodb / driver-turso, each `*-comparand-type-conformance.test.ts`), and cite the past-limit case by its name in the table: 'a bigint beyond ±2^53 is refused — precision loss must not answer silently' (packages/spec/src/data/filter-comparand-type-conformance.ts). ⛔ Do NOT score this over HTTP, and do NOT read a REST 200 as either a pass or a FAIL: the refusal arm is guarded by `typeof value === 'bigint'` (judgeLiteralComparand in filter-comparand-type.ts) and JSON.parse never yields a bigint at any magnitude, so no JSON body can reach it. Measured: the literal 9007199254740993 (2^53 + 1) parses to 9007199254740992, which IS FILTER_COMPARAND_BIGINT_EXACT_LIMIT and therefore also inside the inclusive `<=` bound — two independent reasons the REST arm cannot fail. The REST door's honest verdict for this clause is not-applicable, recorded as such", + "evidence": "the engine-door + driver-conformance output naming the past-limit case; if a REST probe was issued at all, the parsed comparand value showing why its status proves nothing" }, { "clause": "a dotted filter head is CLASSIFIED (relation | virtual | scalar | null) and each class is handled per contract — a scalar head with a dot is refused rather than treated as a relation traversal", @@ -1135,6 +1142,12 @@ "date": "2026-08-17", "change": "new — the comparand-typing contract landed as a cluster of seven spec modules plus three semantic migrations at 17.0.0-rc.6 and had no item. api-backend.query-contract-matrix covers OPERATORS ('every filter operator gives known answers') but says nothing about the TYPE of the right-hand side, which is where the silent-widening failure lives. Variants pinned to ACCEPTED_FILTER_COMPARAND_TYPES so a seventh accepted type flags the matrix stale", "ref": "#9299" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "made clause 2 (the bigint exact-integer boundary) name the door it is actually assertable at, and marked it door-specific. As written it prescribed 'both requests' against the api oracle, but the refusal arm is guarded by `typeof value === 'bigint'` and JSON.parse never yields a bigint, so the clause was unreachable over REST at any magnitude — a runner following it collected a 200 that proves nothing in either direction. Re-derived rather than assumed: 9007199254740993 parses to 9007199254740992, which is exactly the limit and so also inside the inclusive bound, giving two independent reasons the REST arm cannot fail. Clause 2 now points at engine-comparand-type-door.test.ts and the five driver conformance suites with oracle: test, step 3 says to pass a real bigint in-process, and a knownGap records that the REST verdict is not-applicable rather than blocked (#10236 A3)", + "ref": "#10236" } ] }, diff --git a/docs/qa/platform-checklist/areas/attachments-storage.json b/docs/qa/platform-checklist/areas/attachments-storage.json index a9a90732de..c2cd964004 100644 --- a/docs/qa/platform-checklist/areas/attachments-storage.json +++ b/docs/qa/platform-checklist/areas/attachments-storage.json @@ -661,7 +661,7 @@ "title": "Per-line Receipt upload cells in the invoice inline grid: auto-derived file column, real upload control, opaque sys_file reference in the atomic batch", "since": "v15.1", "status": "active", - "revision": 2, + "revision": 3, "priority": "P1", "surface": "browser", "personas": ["seeded admin (admin@objectos.ai)"], @@ -670,6 +670,9 @@ "requires": [ "showcase_invoice_line.receipt (Field.file) and the seeded product 'Widget A' (sku WIDGET-A) for the line's product pick", "the storage service live (the upload rides the console's UploadProvider adapter into /api/v1/storage)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -727,7 +730,13 @@ ], "history": [ { "revision": 1, "date": "2026-08-07", "change": "new item transcribed from the live e2e pin (grid-file-upload.spec.ts) with seeded names verified (Widget A / showcase_invoice_line.receipt); ADR-0059 form-side guard cross-referenced to records-forms instead of duplicated", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 2, "date": "2026-08-11", "change": "clause 3 restated against ADR-0104 D3 after run #7635 scored it `partial` on a product that is CORRECT: the clause demanded a resolved stored-file object with receipt.name and an absolute http(s) url, but D3 makes the stored form the bare opaque sys_file id and the resolved shape the read/expand form only — so the item was asserting a contract the ADR had already retired from the write path. The clause now asserts the opaque id plus the field-ownership stamp (ref_object/ref_id/ref_field/acl) that proves the reference is managed rather than a placeholder, the step text and title follow, a negative pins the drift itself so a future run does not re-score the shipped shape as a defect, and source cites the ADR + the storage-routes.ts line documenting the id. Checklist mirrors the product; no product change", "ref": "#7669" } + { "revision": 2, "date": "2026-08-11", "change": "clause 3 restated against ADR-0104 D3 after run #7635 scored it `partial` on a product that is CORRECT: the clause demanded a resolved stored-file object with receipt.name and an absolute http(s) url, but D3 makes the stored form the bare opaque sys_file id and the resolved shape the read/expand form only — so the item was asserting a contract the ADR had already retired from the write path. The clause now asserts the opaque id plus the field-ownership stamp (ref_object/ref_id/ref_field/acl) that proves the reference is managed rather than a placeholder, the step text and title follow, a negative pins the drift itself so a future run does not re-score the shipped shape as a defect, and source cites the ADR + the storage-routes.ts line documenting the id. Checklist mirrors the product; no product change", "ref": "#7669" }, + { + "revision": 3, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, { diff --git a/docs/qa/platform-checklist/areas/automation.json b/docs/qa/platform-checklist/areas/automation.json index 9d6e5582b0..5bd59f3a4d 100644 --- a/docs/qa/platform-checklist/areas/automation.json +++ b/docs/qa/platform-checklist/areas/automation.json @@ -2,6 +2,74 @@ "$comment": "Standing platform test checklist — area ledger. Hand-edited, append-only; validated by scripts/check-platform-checklist.mjs (pnpm check:platform-checklist). Authoring rules: docs/qa/platform-checklist/README.md · execution protocol: RUNNER.md.", "area": "automation", "title": "Automation — flows, triggers, roll-ups", + "fixtures": { + "$comment": "AREA-LEVEL standing fixtures. An item opts in with fixtures.provisioning = { use: \"\", why: \"…\" }; the recipe is written ONCE here so several items do not carry drifting copies of the same call sequence. Recipes are runtime-provisioned (no repo file is touched, nothing to seed) and every call is grounded in framework source at the cited file:line — replay them literally, and if one 4xxs, re-read the citation before assuming the recipe rotted. Shape copied from qa-scratch-authz in areas/attachments-storage.json (#7716/#7670).", + "qa-flow-status-doors": { + "title": "Two scratch probe flows (qa_no_start_node / qa_always_fails) in a scratch package, so every never-dispatched and ran-and-failed row of the #9378 status table has a target", + "why": "Stock showcase can reach four rows of the trigger table on its own — 404 (any unheld name), 409 (toggle showcase_urgent_task_alert OFF), 200 success, and 200 PAUSED (showcase_reassign_wizard) — but NOT the two that need a purpose-built definition. 422 FLOW_NO_START_NODE needs a flow whose `nodes` array holds no `start` node, and 400 FLOW_FAILED needs one that reaches a node and fails TERMINALLY: the showcase's only deterministic failure, showcase_resilient_sync, is wrapped in a try_catch that HANDLES it, so the run succeeds and the 400 row is never exercised. Without this recipe those two rows are blocked(fixture) and a regression that reverted either one leaves the area green.", + "provenance": "authored from source in the 17.1.0 post-release sweep (#10236); the scratch-package authoring path is the one automation.flow-error-handling already declares ('a writable/scratch package to author the UNPROTECTED probe flow into — the showcase itself ships read-only') and the call shapes are qa-scratch-authz's (#7670). NOT yet proven by a live run — the four stock rows were hand-verified in #10225, these two were not.", + "app": "showcase", + "requires": [ + "an admin session holding `manage_metadata` — PUT /api/v1/meta/:type/:name is capability-gated per ADR-0066 D1 (#6603), so a bare authenticated session is NOT enough", + "an isolated boot (own port + file DB, dogfood §0) — this recipe authors metadata into the running app", + "⚠️ export OS_PORT= as well as passing -p , or the showcase's self-pinging connectors dial 3000 and every unrelated flow run in the same boot fails `fetch failed` (RUNNER.md, the isolated-boot/OS_PORT standing fact)" + ], + "sequence": [ + { + "step": 1, + "call": "POST /api/v1/packages", + "body": { + "manifest": { "id": "com.objectstack.qa.automation", "name": "QA automation status fixture", "version": "1.0.0", "type": "app" }, + "enableOnInstall": true + }, + "expect": "2xx with the installed package echoed back. The id is deliberately distinct from the other QA recipes so this one installs and tears down independently. A re-run against a live DB 409s on the duplicate id — send `overwrite: true` deliberately, never by reflex.", + "source": "POST /api/v1/packages is the dispatcher install route (packages/rest/src/rest-route-ledger.ts:312 note; client `packages.install`); body shape { manifest, enableOnInstall?, overwrite? } pinned in packages/client/src/client.test.ts:2144-2166" + }, + { + "step": 2, + "call": "PUT /api/v1/meta/flow/qa_no_start_node?package=com.objectstack.qa.automation", + "body": { + "name": "qa_no_start_node", + "label": "QA — no start node", + "type": "autolaunched", + "status": "active", + "nodes": [{ "id": "n1", "type": "end", "label": "End" }], + "edges": [] + }, + "expect": "2xx. The definition is deliberately AUTHORABLE-but-unexecutable: FlowSchema declares `nodes: z.array(FlowNodeSchema)` with no `.min(1)` and no refinement requiring a `start` member (packages/spec/src/automation/flow.zod.ts:658), and nothing rejects a start-less flow at publish, so this lands. It is the engine that refuses it at DISPATCH — `flow.nodes.find(n => n.type === 'start')` misses and execute() returns { success:false, code:'FLOW_NO_START_NODE' } (packages/services/service-automation/src/engine.ts:3212-3220). ⚠️ If a publish-time structural check lands later this step will start 4xx-ing — that is a FINDING about where the refusal moved, not a rotted recipe.", + "source": "flow is NOT in the org-overlay-allowed set (RUNNER.md's overlay fact — view/dashboard/report/translation/email_template only), so the `?package=` route is required exactly as qa-scratch-authz uses it for objects; `?package=` is read as query.package and threaded to saveMetaItem as packageId (packages/runtime/src/domains/meta.ts:262,319)" + }, + { + "step": 3, + "call": "PUT /api/v1/meta/flow/qa_always_fails?package=com.objectstack.qa.automation", + "body": { + "name": "qa_always_fails", + "label": "QA — always fails terminally", + "type": "autolaunched", + "status": "active", + "nodes": [ + { "id": "start", "type": "start", "label": "Start" }, + { "id": "boom", "type": "http", "label": "Unroutable", "config": { "method": "POST", "url": "https://api.example.com/v1/qa-always-fails" } } + ], + "edges": [{ "id": "e1", "source": "start", "target": "boom" }] + }, + "expect": "2xx on the author. On trigger the run DISPATCHES, the http node fails against the unroutable host, and — with no try_catch region and no errorHandling.strategy — the run ends `status: 'failed'`, which is the 400 FLOW_FAILED row. ⚠️ Do NOT wrap it in try_catch: a handled failure is a SUCCESSFUL run and scores the wrong row. The unroutable-host technique (api.example.com) is the one automation.flow-error-handling already relies on for a deterministic failure.", + "source": "the 400 row keys on the engine's lifecycle verdict `status: 'failed'`, never on message text (packages/runtime/src/flow-dispatch-status.ts, classifyFlowRefusal); examples/app-showcase/src/automation/flows/index.ts (showcase_resilient_sync) is the precedent for the unroutable host" + }, + { + "step": 4, + "call": "toggle a stock record-change flow OFF for the 409 row: POST /api/v1/automation/showcase_urgent_task_alert/toggle {\"enabled\": false}", + "expect": "2xx. No authoring needed — the 409 FLOW_DISABLED row is reachable on stock fixtures. ⚠️ Toggle it back ON in teardown: it is the fixture automation.flow-toggle-kill-switch drives, and leaving it off silently changes that item's baseline.", + "source": "packages/runtime/src/domains/automation.ts (POST /:name/toggle → engine.toggleFlow); automation.flow-toggle-kill-switch names showcase_urgent_task_alert as the only type:'record_change' flow" + } + ], + "teardown": "POST /api/v1/automation/showcase_urgent_task_alert/toggle {\"enabled\": true} to undo step 4, then DELETE /api/v1/packages/com.objectstack.qa.automation (client `packages.uninstall`, ledgered at rest-route-ledger.ts:317) — or discard the isolated file DB, the cheaper path an isolated boot makes free. The toggle is NOT covered by discarding the DB if you reused a boot, so undo it explicitly.", + "knownGaps": [ + "Same SDK sharp edge as qa-scratch-authz: `meta.saveItem(type, name, item)` does NOT send `?package=` (packages/client/src/index.ts:701-707) — issue steps 2 and 3 as raw HTTP with the query string appended, or the flows land outside the package and the teardown misses them.", + "This recipe has NOT been proven by a live run. If step 2 or 3 4xxs on the flow body, re-read the cited schema line before assuming the recipe rotted — a publish-time refusal of a start-less flow would itself be a finding (it would mean the 422 row has moved from dispatch time to author time, which changes what the item is asserting)." + ] + } + }, "items": [ { "id": "automation.flow-run-step-nesting", @@ -629,7 +697,7 @@ "title": "Screen flow round-trip: action trigger → paused run → rendered dialog → resume with inputs → persisted write", "since": "v16", "status": "active", - "revision": 2, + "revision": 3, "priority": "P1", "surface": "browser", "personas": [ @@ -639,6 +707,9 @@ "app": "showcase", "requires": [ "the Tasks list row action showcase_bulk_reassign wired to showcase_reassign_wizard (type screen, runAs user; screen node 'collect' with required field new_assignee; downstream update_record)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -729,6 +800,12 @@ "date": "2026-08-08", "change": "added the FlowRunner cancel-mid-flow clause — dismissing the screen dialog parks the run (still paused, resumable) rather than orphaning/cancelling it; API run state is the oracle", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 3, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -822,7 +899,7 @@ "title": "connector_action dispatches through every registered connector kind, and the registry feeds the designer pickers", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "mixed", "personas": [ @@ -832,10 +909,12 @@ "app": "showcase", "requires": [ "plugin connectors rest + slack (objectstack.config.ts) and declarative instances showcase_status_api (provider rest) + showcase_mcp_tools (provider mcp, in-repo stdio fixture server) from src/system/connectors/", - "the three dispatching flows: showcase_task_completed_rest_ping (rest), showcase_declarative_connector_ping (declarative rest, ADR-0097), showcase_mcp_connector_echo (declarative MCP, #3056)" + "the three dispatching flows: showcase_task_completed_rest_ping (rest), showcase_declarative_connector_ping (declarative rest, ADR-0097), showcase_mcp_connector_echo (declarative MCP, #3056)", + "⚠️ a boot that EXPORTS OS_PORT= (or SHOWCASE_SELF_URL) as well as passing -p — the declarative instances resolve their baseUrl from the environment, not from the flag, so an isolated boot without it dials 3000 and every declarative dispatch fails `fetch failed` (see knownGaps)" ], "knownGaps": [ - "slack DELIVERY is not assertable on stock fixtures — TaskCompletedSlackFlow points at a placeholder channel with no real bot token, so the slack variant is registry-enumeration only" + "slack DELIVERY is not assertable on stock fixtures — TaskCompletedSlackFlow points at a placeholder channel with no real bot token, so the slack variant is registry-enumeration only", + "`showcase_declarative_connector_ping` failing every record-change firing with `connector_action(showcase_status_api.request) failed: fetch failed` is a BOOT MISCONFIGURATION, not a blocked environment and not a missing stub. The showcase's declarative connector instances dial the running server itself via resolveShowcaseSelfUrl() (examples/app-showcase/src/system/self-url.ts), which reads SHOWCASE_SELF_URL -> OS_PORT -> PORT -> the literal http://127.0.0.1:3000 from the ENVIRONMENT. The CLI's `-p` flag only takes its DEFAULT from OS_PORT and never writes it back (packages/cli/src/commands/serve.ts), so a dogfood-§0 isolated boot on a non-3000 port leaves the connectors dialing 3000 while the server listens elsewhere. Export OS_PORT= (or SHOWCASE_SELF_URL) alongside -p and the variant runs. ⚠️ The string is indistinguishable from a sandbox egress block — that is what makes it expensive: #7516 burned a run proving it was an address problem with a TCP forwarder on 3000, and #7538 fixed the resolution but cannot fix a boot that never exports the variable. Verify by reading the instance baseUrl out of GET /api/v1/automation/connectors BEFORE scoring the variant." ] }, "variants": [ @@ -847,6 +926,7 @@ "steps": [ "boot showcase isolated (dogfood §0); sign in as the dev admin", "GET /api/v1/automation/connectors — record the descriptor list and each connector's action list (the MCP instance's actions must come from its tools/list handshake)", + "premise guard, before scoring any dispatch clause: read the two declarative instances' resolved baseUrl out of that same response and confirm it names the port this boot is listening on — a baseUrl of http://127.0.0.1:3000 against a server on another port means the fixture is misconfigured, and every dispatch below will fail `fetch failed` for that reason alone", "POST a showcase_task — fires showcase_declarative_connector_ping and showcase_mcp_connector_echo on create", "PATCH the task status→'done' — fires showcase_task_completed_rest_ping", "GET each of the three flows' newest run detail; capture the connector step's recorded request and response", @@ -898,6 +978,12 @@ "date": "2026-08-07", "change": "initial — connector dispatch matrix over the plugin / declarative-rest / declarative-MCP kinds the showcase seeds, with the unregistered-id negative", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded the self-URL boot fact that makes the declarative-rest variant look permanently broken. The 17.1.0 sweep observed showcase_declarative_connector_ping failing `fetch failed` on every firing and read it as a network-restricted runner, i.e. as a fixture/environment gap needing a declared stub. Re-derived from source: it is an ADDRESS problem an isolated boot creates for itself — resolveShowcaseSelfUrl reads OS_PORT/PORT from the environment while the CLI's -p flag only reads that variable as a default and never exports it, so every non-3000 isolated boot dials 3000. The variant is fully runnable with OS_PORT exported; recording it as blocked(environment) or building a stub would have waived a working clause. Added the boot requirement and a pre-scoring baseUrl check, plus a knownGap carrying the discriminator so the next runner does not re-derive it (#10236 A5/B3)", + "ref": "#10236" } ] }, @@ -1064,6 +1150,128 @@ "ref": "claude/platform-test-checklist-ocwugl" } ] + }, + { + "id": "automation.trigger-status-contract", + "title": "The flow-dispatch HTTP status contract: every never-dispatched, ran-and-failed and non-terminal outcome answers its own status + ADR-0112 code, at every door that dispatches a flow", + "since": "v17", + "status": "active", + "revision": 1, + "priority": "P1", + "surface": "api", + "personas": ["seeded admin (admin@objectos.ai / admin123)"], + "fixtures": { + "app": "showcase", + "requires": [ + "showcase_reassign_wizard (examples/app-showcase/src/automation/flows/index.ts) — the screen flow whose trigger PAUSES, giving the non-terminal 200 row and every resume arm a live suspension to work against", + "showcase_urgent_task_alert — the record-change flow the 409 row toggles OFF (and back ON in teardown)" + ], + "provisioning": { + "use": "qa-flow-status-doors", + "why": "unblocks the 422 FLOW_NO_START_NODE and 400 FLOW_FAILED rows of clause 0 and their per-door repeats in clause 2. Without it those two rows are blocked(fixture): stock showcase has no start-less flow, and its one deterministic failure (showcase_resilient_sync) is wrapped in a try_catch that HANDLES the failure, so the run SUCCEEDS and the 400 row is never exercised. The item would score partial with the two most-regressible rows unproven." + }, + "knownGaps": [ + "The MCP `run_action` bridge shares action-execution.ts's mapper with POST /api/v1/actions/... but is not an HTTP door, so it has no status to observe — clause 2 covers it by asserting the two share ONE mapper (source-level), not by driving MCP." + ] + }, + "steps": [ + "read the table first, from its single definition: packages/runtime/src/flow-dispatch-status.ts (FLOW_NOT_FOUND_STATUS, classifyFlowRefusal, isPausedRun) — it is a module rather than a route mapper precisely because three doors must agree on it, and a second copy of it is a defect by construction. The resume arms are a separate list, read them from packages/runtime/src/domains/automation.ts (the `POST /:name/runs/:runId/resume` branch and its code→status arms)", + "boot showcase isolated (dogfood §0) with OS_PORT exported as well as -p (RUNNER.md's connector self-URL fact), sign in as the dev admin, and run the qa-flow-status-doors recipe", + "TRIGGER, per row, against BOTH doors — POST /api/v1/automation/:name/trigger and the legacy POST /api/v1/automation/trigger/:name (the shape client.automation.trigger() calls): a name the deployment does not hold; qa_always_fails; the disabled showcase_urgent_task_alert; qa_no_start_node; a flow that succeeds; showcase_reassign_wizard (pauses)", + "record for every response the HTTP status AND error.code together — neither alone is the contract, and a run that captured only the status cannot tell 409 FLOW_DISABLED from any other 409", + "on the 400 FLOW_FAILED responses, additionally record error.details.errorMessage and error.details.summary (the flow author's own text and the per-node accounting) — they are the documented payload of that row and the console reads them from exactly there", + "RESUME: trigger showcase_reassign_wizard to obtain a live paused runId, then drive each refusal arm against POST /api/v1/automation/:name/runs/:runId/resume — a bodyless/valid resume (200, next screen or completion), a resume missing a required screen input (400 INVALID_SCREEN_INPUT), a resume whose signal writes the engine's `$` namespace (400 INVALID_SIGNAL), a resume of an unknown runId (404 RUN_NOT_FOUND), and a resume as a caller who is neither the trigger identity nor a resumeAuthority holder (403 PERMISSION_DENIED)", + "after each RETRYABLE refusal, re-read the run and confirm the suspension is STILL PAUSED and a later valid resume still works — 'the refusal left the suspension intact' is half of what those arms mean, and a status alone does not show it", + "cross-door: dispatch the SAME disabled flow through POST /api/v1/actions/... with a type:'flow' action and through a declared type:'flow' endpoint, and record what each answers", + "capture the per-row × per-door verdict table" + ], + "acceptance": [ + { + "clause": "every TRIGGER row answers its own status and code, on BOTH trigger routes: 404 (no such flow), 409 FLOW_DISABLED, 422 FLOW_NO_START_NODE, 400 FLOW_FAILED — and a successful run answers 200", + "oracle": "api", + "verify": "one request per row per route; the observed (status, error.code) pair equals the row in flow-dispatch-status.ts. ⚠️ The dangerous direction is a 200 whose body carries an inner {success:false}: that is the pre-17.1.0 double envelope these rows replaced, and a caller branching on HTTP status alone reads a failed or refused run as a successful one — so a 200 on any of the four refusal rows is a FAIL, not a formatting difference", + "evidence": "the per-row per-route status + code table, with one full response body per row" + }, + { + "clause": "a PAUSED run is served as the non-terminal state it is — 200 with the engine result carrying runId (and screen) — and is never classified as a refusal", + "oracle": "api", + "verify": "trigger showcase_reassign_wizard: 200, body carries a runId the resume door accepts. This row is written down on purpose (isPausedRun, #9510): a pause reaching a reader that knows only 'succeeded' and 'failed' is how one defect becomes another, and since #9510 a pause can also arrive through the RETRY path, so it has two producers", + "evidence": "the paused trigger response + the runId proven resumable" + }, + { + "clause": "the table is a property of the flow-dispatch CONTRACT, not of the trigger route: the other doors that dispatch a flow answer the same (status, code) for the same engine exit", + "oracle": "api", + "verify": "drive the disabled flow through POST /api/v1/actions/... (type:'flow' action) and a declared type:'flow' endpoint and compare against the trigger doors. This is the exact drift the ruling (maintainer 2026-08-18, 「同意」 on #9446) converged: one exit once had three answers — 409 FLOW_DISABLED at the trigger door, 400 FLOW_FAILED at /actions ('the flow ran and rejected', a false statement about a dispatch that never happened), and 200 for every outcome at the endpoint door. ⚠️ Two divergences are DECLARED and are not findings: an UNCLASSIFIED success:false stays 200 at the trigger door but is refused at /actions, and /actions THROWS where the trigger door RETURNS. Read the module note before scoring a difference", + "evidence": "the per-door comparison table + the source citation showing which mapper each door reads" + }, + { + "clause": "the 400 FLOW_FAILED row carries the author's own message and the run's per-node accounting where the door documents them — error.details.errorMessage and error.details.summary — and not only a generic message", + "oracle": "api", + "verify": "the trigger door's 400 body carries both keys; ADR-0112 envelopes carry no `data`, so a producer building its message out of result.error alone drops the author's words silently. #9585 extended this to /actions for this row only, via the typed FlowActionRefusal carrier — so both doors ship the pair HERE while the mechanism stays each door's own", + "evidence": "the 400 bodies from both doors, keys shown" + }, + { + "clause": "every RESUME refusal answers its own status and code, and the retryable ones LEAVE THE SUSPENSION INTACT: 403 PERMISSION_DENIED, 400 INVALID_SIGNAL, 400 INVALID_SCREEN_INPUT, 409 RESUME_IN_PROGRESS, 503 STORE_UNAVAILABLE all keep the run paused; 404 RUN_NOT_FOUND and 400 FLOW_FAILED are terminal", + "oracle": "api", + "verify": "per reachable arm: the (status, code) pair, THEN a re-read of the run proving it is still paused and a subsequent valid resume still succeeds. The intact-suspension half is what separates these arms from the terminal ones and no status shows it. 503 STORE_UNAVAILABLE has no stock trigger — record it blocked(fixture) rather than inferring it from the code being registered", + "evidence": "per-arm response + the post-refusal run state + the successful later resume" + }, + { + "clause": "the two STALE-suspension exits are reported as terminal 404 RUN_NOT_FOUND, not as a business rejection — the flow was deregistered, or the node the run was parked on was edited away under a live pause", + "oracle": "api", + "verify": "obtain a live pause, then remove the flow (or edit the paused node away) and resume: 404 RUN_NOT_FOUND, with the engine's message naming WHICH of the two it was. The engine classifies both (#8684, producer-first) — the route must not be sniffing summary/durationMs to tell the classes apart", + "evidence": "the 404 response + its message" + }, + { + "clause": "the SDK surfaces the contract rather than flattening it: client.automation.trigger()/.execute() and .resume() REJECT on every non-2xx, and the rejection carries httpStatus and code", + "oracle": "test", + "verify": "drive the same rows through @objectstack/client and assert the catch: err.httpStatus ∈ {400,403,404,409,422,503} and err.code is the row's code. Callers used to inspect a RESOLVED {success:false}; the 17.1.0 changesets moved them to a catch, so a resolve on a refusal row is a FAIL of the SDK half of the same contract", + "evidence": "the SDK rejection captures, per row" + } + ], + "negative": [ + "any refusal row answered 200 with an inner {success:false} — the double envelope #3962 ruled out and 17.1.0 removed; this is the load-bearing negative, because it is the shape that reads as success to every status-branching caller", + "a never-dispatched exit answered 400 FLOW_FAILED — it states that a run ran and was rejected, which is false, and it tells an operator to debug a run that does not exist", + "409 and 422 collapsed into one status — a disabled flow is reversible operational state (enable it and the identical request succeeds, which is what 409 means) while a start-less definition cannot be executed as stored and no retry helps (422); collapsing them tells an operator to flip a switch that will not help", + "a PAUSED run answered as a refusal (or as a completed run) — the continuation is persisted and the caller needs the runId", + "a retryable resume refusal that CONSUMED the suspension — the status may be right while the run is destroyed", + "a door answering from its own reading of the engine result rather than from flow-dispatch-status.ts — even when today's answer happens to match, a second copy of the table is the defect the module exists to prevent" + ], + "variants": [ + "trigger · 404 — no such flow (registry probe, before dispatch)", + "trigger · 409 FLOW_DISABLED — never dispatched, reversible", + "trigger · 422 FLOW_NO_START_NODE — never dispatched, unretryable", + "trigger · 400 FLOW_FAILED — ran and was rejected", + "trigger · 200 — succeeded", + "trigger · 200 PAUSED — non-terminal, runId returned (#9510)", + "resume · 403 PERMISSION_DENIED — service-owned suspension, retryable", + "resume · 400 INVALID_SIGNAL — signal wrote the engine's $ namespace, retryable", + "resume · 400 INVALID_SCREEN_INPUT — bag violates the screen's field contract, retryable", + "resume · 409 RESUME_IN_PROGRESS — a concurrent resume holds the run, retryable", + "resume · 503 STORE_UNAVAILABLE — durable store unreadable, retryable", + "resume · 404 RUN_NOT_FOUND — no such suspension, or a STALE one; terminal", + "resume · 400 FLOW_FAILED — the resumed run consumed its pause and failed; terminal", + "resume · 200 — completed, or paused again at the next screen" + ], + "traps": ["wrong-persona", "single-datapoint"], + "source": [ + "packages/runtime/src/flow-dispatch-status.ts — the ONE definition of the trigger-side table (FlowRefusalCode, classifyFlowRefusal, isPausedRun, FLOW_NOT_FOUND_STATUS) and the note on why it is a module rather than a route mapper", + "packages/runtime/src/domains/automation.ts — both trigger routes and the resume route's code→status arms", + "packages/runtime/src/action-execution.ts and packages/runtime/src/endpoint-executor.ts — the other two doors reading the same table", + "packages/services/service-automation/src/engine.ts:3212-3220 — the FLOW_NO_START_NODE exit; the disabled-flow exit stamps FLOW_DISABLED, and neither carries `status`, which is what lets a transport tell a never-dispatched exit from a run that dispatched and failed", + "packages/spec/src/contracts/automation-service.ts — AutomationResult.code, the closed union the two trigger-time refusals were added to", + "packages/spec/src/api/error-code-ledger.zod.ts — ADR-0112 registration of FLOW_DISABLED / FLOW_NO_START_NODE / FLOW_FAILED to @objectstack/runtime", + "packages/client/CHANGELOG.md 17.1.0 (bc6434b resume, 9aa8890 disabled/no-start-node, 48032c9 ran-and-failed) — the three BREAKING changesets whose own status table this matrix is derived from", + "packages/runtime/src/domains/automation-trigger-route-status.test.ts, automation-trigger-paused-run.test.ts, endpoint-flow-dispatch-status.test.ts, actions-flow-dispatch-status.test.ts — the per-door pins" + ], + "history": [ + { + "revision": 1, + "date": "2026-08-21", + "change": "new — the automation area asserted flow BEHAVIOUR but never the status a caller observes, so a regression reverting any of 17.1.0's three BREAKING status changes left the whole area green (found by the 17.1.0 post-release sweep, #10225). The matrix is derived from the three changesets' own status tables and reconciled against the single runtime definition in flow-dispatch-status.ts, which is wider than the changesets: the same table is now read by /actions, the MCP run_action bridge and declared endpoints, and it carries a sixth NON-TERMINAL row (PAUSED, #9510) the changesets do not tabulate. The 422 arm was expected to be a knownGap; it is not — FlowSchema requires no start node (flow.zod.ts:658, no .min(1), no refinement) and nothing refuses a start-less flow at publish, so the row is authorable through a scratch package and is provisioned by qa-flow-status-doors instead of waived", + "ref": "#10236" + } + ] } ] } \ No newline at end of file diff --git a/docs/qa/platform-checklist/areas/dashboards.json b/docs/qa/platform-checklist/areas/dashboards.json index 23c2d65c33..c9a39f82da 100644 --- a/docs/qa/platform-checklist/areas/dashboards.json +++ b/docs/qa/platform-checklist/areas/dashboards.json @@ -279,7 +279,7 @@ "title": "Every ChartTypeSchema member renders a real chart with the correct marks and series count", "since": "v15", "status": "active", - "revision": 3, + "revision": 4, "priority": "P1", "surface": "browser", "personas": [ @@ -293,7 +293,7 @@ "a scratch draft dashboard for the period-over-period compareTo probe (via PUT /api/v1/meta/dashboard/qa_compareto_probe?mode=draft) — NO showcase widget authors compareTo, so this is authored fresh exactly like the strict-widget probes; bind a metric/line widget to showcase_task_metrics with a time dimension (a month-bucketed created_at) and carry compareTo: { kind: 'previousPeriod' }" ], "knownGaps": [ - "kpi / gauge / solid-gauge / bullet have NO distinct renderer today — they render the same single value as `metric` (chart.zod.ts NOTE: 'honest single-value synonyms … gain a dial when a gauge renderer lands'; coverage.test.ts SAME_AS_METRIC). For these four the demonstrable claim is 'renders the value, not blank', NOT 'renders a dial' — a run must not tick a dial that does not exist" + "kpi / gauge / solid-gauge / bullet have NO distinct renderer today — all four route to the SAME single-value renderer as `metric` (chart.zod.ts NOTE: '`metric`/`kpi` are kept as honest single-value synonyms; `gauge`/`solid-gauge`/`bullet` render a value today and gain a dial when a gauge renderer lands'; examples/app-showcase/test/coverage.test.ts SAME_AS_METRIC = {kpi, gauge, solid-gauge, bullet}). Two consequences a run must hold BOTH of: (a) the demonstrable claim is 'renders the value, not blank', NOT 'renders a dial' — do not tick a dial that does not exist; and (b) ⚠️ the four scratch tiles are expected to be VISUALLY IDENTICAL to each other and to the metric tile. Four indistinguishable tiles is the CORRECT observation, not evidence that the renderer ignored the declared type and not a per-variant FAIL — re-confirmed on 17.1.0 (#10560). The type IS being honoured; there is simply one renderer behind five names, which is what makes them synonyms rather than families" ] }, "steps": [ @@ -337,7 +337,7 @@ { "clause": "the four single-value synonyms (kpi, gauge, solid-gauge, bullet) render the SAME value as metric — a number, never a blank or an error", "oracle": "screenshot", - "verify": "scratch-dashboard tiles each show the task_count value; evidence must note these are value-only renders (no dial) per the spec's own NOTE", + "verify": "scratch-dashboard tiles each show the task_count value; evidence must note these are value-only renders (no dial) per the spec's own NOTE, and that the four tiles are EXPECTED to look identical to one another and to the metric tile — one renderer, five names. ⛔ Do not score 'the four tiles are indistinguishable' against the negative below: that negative is about a variant rendering as a DIFFERENT FAMILY (sankey drawn as a bar), which is the opposite failure", "evidence": "scratch-dashboard screenshot annotated with the honest-synonym caveat" }, { @@ -366,7 +366,7 @@ } ], "negative": [ - "any variant rendering as a DIFFERENT family than named (e.g. sankey falling back to a bar) is a FAIL — advertising a type that renders as something else is the exact failure the trimmed enum exists to prevent", + "any variant rendering as a DIFFERENT family than named (e.g. sankey falling back to a bar) is a FAIL — advertising a type that renders as something else is the exact failure the trimmed enum exists to prevent. ⚠️ EXPLICITLY EXEMPT: the four single-value synonyms (kpi/gauge/solid-gauge/bullet) rendering as `metric` is DECLARED, not a fallback — it is written into the spec NOTE and waived by name in coverage.test.ts. Filing it under this negative is a false FAIL", "a blank widget with no error for any variant is a FAIL", "a compareTo widget rendering its base numbers with the comparison silently absent (the pre-#5011 dropped-string-arm bug) is a FAIL, not a thin-data caveat" ], @@ -423,6 +423,12 @@ "date": "2026-08-08", "change": "pinned enumSource for the variants-freshness ratchet — spec enum drift is caught by the manual check on this item directly", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 4, + "date": "2026-08-21", + "change": "made the single-value-synonym carve-out say what a runner will actually SEE. The item already recorded that kpi/gauge/solid-gauge/bullet render 'the same single value as metric', which stops the wrong tick (a dial) but not the wrong FAIL: a runner putting the four scratch tiles side by side sees four indistinguishable tiles and can read that as the renderer ignoring the declared type — i.e. as the wrong-family negative two lines down. Re-confirmed at source (chart.zod.ts's synonym NOTE; coverage.test.ts's SAME_AS_METRIC = {kpi, gauge, solid-gauge, bullet}): there is ONE renderer behind five names. The knownGap now states that identical tiles are the expected observation, clause 4's verify says so at the point of scoring, and the wrong-family negative carries an explicit exemption so the two failures cannot be confused (#10236 A5, observed #10560)", + "ref": "#10236" } ], "enumSource": { diff --git a/docs/qa/platform-checklist/areas/platform-core.json b/docs/qa/platform-checklist/areas/platform-core.json index d339930e69..7c776ac866 100644 --- a/docs/qa/platform-checklist/areas/platform-core.json +++ b/docs/qa/platform-checklist/areas/platform-core.json @@ -8,7 +8,7 @@ "title": "Showcase boots clean: health + ready 200, no degraded startup banners, console + app metadata served", "since": "v15", "status": "active", - "revision": 3, + "revision": 4, "priority": "P0", "surface": "mixed", "preconditions": [ @@ -37,9 +37,9 @@ "evidence": "the curl output" }, { - "clause": "the `Flows:` startup banner reports no ⚠ misauthored flows and no ERROR-level lines appear anywhere in the boot log", + "clause": "the `Flows:` startup banner reports no ⚠ misauthored flows and no ERROR-level lines appear IN THE BOOT WINDOW — the window is part of the clause, because ordinary caller-error 4xx traffic also logs at ERROR", "oracle": "log", - "verify": "grep the boot log for '⚠' in the Flows banner and for ERROR lines; seed rejections count as failures (see #3415 — SeedLoader rejections were silent)", + "verify": "grep the boot log for '⚠' in the Flows banner and for ERROR lines, SCOPED to the boot window — from process start to the first 200 on /api/v1/health (the same instant clause 0 records as time-to-healthy). ⚠️ A caller-error REFUSAL logs at ERROR level with a full stack BEFORE answering 4xx — measured on 17.1.0 with a `$fn` filter probe answering 400 INVALID_FILTER (#10257). So an unscoped grep over a log that also carries the run's own probe traffic reports ERROR lines for requests the platform refused CORRECTLY, and a clean boot reads as a boot failure. Cut the log at the health-green line before grepping, or capture the boot log to its own file before issuing the first request; seed rejections still count as failures (see #3415 — SeedLoader rejections were silent)", "evidence": "the grepped log excerpt" }, { @@ -57,7 +57,7 @@ ], "negative": [ "any SeedLoader rejection line in the boot log is a FAIL against the seed (RUNNER rule 3: a defect in the fixture is a fail, not a block — #3408/#3415)", - "a health 200 with ERROR lines in the log is NOT a clean boot — the log clause fails independently of the probe" + "a health 200 with ERROR lines in the BOOT WINDOW is NOT a clean boot — the log clause fails independently of the probe. ⚠️ The converse is equally a defect in the RUN, not the platform: ERROR lines logged AFTER health went green by the run's own 4xx-provoking probes are correct refusals being logged loudly, and filing them as a boot failure is a false P0" ], "traps": ["seed-data-thin", "stale-dist"], "source": [ @@ -69,7 +69,13 @@ "history": [ { "revision": 1, "date": "2026-08-07", "change": "initial — standing P0 smoke distilled from the dogfood boot protocol", "ref": "#3358" }, { "revision": 2, "date": "2026-08-07", "change": "expanded to deep-test contract: concrete steps, multi-clause acceptance, negatives, variants", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 3, "date": "2026-08-18", "change": "corrected the ?id= spelling from the package id com.example.showcase to the app NAME showcase_app in step 6 and clause 3. rest-server.ts filters app documents on a.name === id and App declares no id of its own, so the package-id spelling returns an empty items array — a false-P0 shape (#9334, re-confirmed #9453 CF-1)", "ref": "#9386" } + { "revision": 3, "date": "2026-08-18", "change": "corrected the ?id= spelling from the package id com.example.showcase to the app NAME showcase_app in step 6 and clause 3. rest-server.ts filters app documents on a.name === id and App declares no id of its own, so the package-id spelling returns an empty items array — a false-P0 shape (#9334, re-confirmed #9453 CF-1)", "ref": "#9386" }, + { + "revision": 4, + "date": "2026-08-21", + "change": "scoped clause 2's ERROR-line grep to the boot window and said why. A caller-error refusal logs at ERROR level with a full stack before answering its 4xx — measured on 17.1.0 with a `$fn` filter probe answering 400 INVALID_FILTER — so any clause grepping the whole log for ERROR reads ordinary, correctly-refused 4xx traffic as a boot failure. The clause now names the window (process start to the first health 200, the instant clause 0 already records), prescribes cutting the log there or capturing it before the first request, and the negatives carry both directions so the false P0 is recognisable from either side. The security property is unchanged: a genuine ERROR-during-boot still fails independently of the probe (#10236 A5, measured #10257)", + "ref": "#10236" + } ] }, { @@ -459,10 +465,10 @@ }, { "id": "platform-core.builtin-apps-nav-render", - "title": "The three built-in apps (Setup / Studio / Account) render every merged-nav destination; app-level and entry-level gates are enforced, not errored", + "title": "The built-in apps a stock boot REGISTERS (Setup / Account) render every merged-nav destination; app-level and entry-level gates are enforced, not errored — and Studio's deliberate absence is asserted, never hunted", "since": "v15", "status": "active", - "revision": 2, + "revision": 3, "priority": "P0", "surface": "browser", "personas": [ @@ -472,42 +478,43 @@ "fixtures": { "app": "showcase", "requires": [ - "the three platform apps ship with @objectstack/platform-objects and register as one-app packages @objectstack/{setup,studio,account} (ADR-0048), so their PACKAGE ids are com.objectstack.setup / com.objectstack.studio / com.objectstack.account — but the App documents themselves are NAMED setup / studio / account (setup.app.ts, studio.app.ts, account.app.ts), and every ?id= below matches the NAME — present on any stock boot, no showcase-specific fixture", + "the platform apps ship with @objectstack/platform-objects and are packaged one-app-per-package (ADR-0048) as com.objectstack.setup / com.objectstack.studio / com.objectstack.account — but only TWO of the three are loaded on a stock boot. The ADR-0048 registration loop registers exactly @objectstack/setup and @objectstack/account (packages/plugins/plugin-dev/src/dev-plugin.ts for os dev; packages/cli/src/commands/serve.ts for os serve), so a stock GET /api/v1/meta/app answers showcase_app + setup + account. The App documents are NAMED setup / account (setup.app.ts, account.app.ts) and every ?id= below matches the NAME, never the package id — present on any stock boot, no showcase-specific fixture", "a second, non-admin persona: sign up a fresh user in-run to drive the app-level and entry-level permission gates (do not reuse the admin — wrong-persona masks the guard)" ], "knownGaps": [ - "the SSO Providers entry (nav_sso_providers → sys_sso_provider) is contributed ONLY when the external-IdP RP is wired (OS_SSO_ENABLED self-host / cloud planAllowsSso — plugin-auth auth-plugin.ts isSsoWired()). On a stock open-framework boot it is ABSENT by design (not gated-and-erroring); assert its absence, never treat it as a missing surface" + "the SSO Providers entry (nav_sso_providers → sys_sso_provider) is contributed ONLY when the external-IdP RP is wired (OS_SSO_ENABLED self-host / cloud planAllowsSso — plugin-auth auth-plugin.ts isSsoWired()). On a stock open-framework boot it is ABSENT by design (not gated-and-erroring); assert its absence, never treat it as a missing surface", + "Studio is NOT registered on a stock boot, by product decision — this is a carve-out to ASSERT, not a fixture to provision. STUDIO_APP is defined (packages/platform-objects/src/apps/studio.app.ts) and packaged (packages/apps/studio, com.objectstack.studio), but both boot paths deliberately skip it with the identical note: '@objectstack/studio is intentionally NOT default-loaded — the console ships a dedicated Studio surface at /_console/studio//, so Studio no longer needs to exist as a navigable app tile' (plugin-dev/src/dev-plugin.ts and cli/src/commands/serve.ts, the ADR-0048 app-package loop in each — both register only setup and account). So ?id=studio answering 200 {\\\"items\\\":[]} is the CORRECT observation on every stock boot, not a showcase-specific gap and not a missing built-in. ⛔ Do not install the package to 'restore' it: that changes what the boot ships and is a maintainer decision, not a run's. The prior wave already walked everything reachable here — Setup 43/43 and Account 7/7 destinations, 50/50 ok, zero pageerror (#9453). Studio's own surfaces are covered by area studio-authoring against /_console/studio, not through a nav tile." ] }, "steps": [ "as admin, GET /api/v1/meta/app?id=setup and extract every merged nav destination — Setup is a SHELL of empty group anchors (packages/platform-objects/src/apps/setup.app.ts: group_overview/apps/people_org/access_control/approvals/configuration/diagnostics/integrations/advanced) filled by SETUP_NAV_CONTRIBUTIONS (setup-nav.contributions.ts) plus capability-plugin contributions (plugin-security Positions/Permission Sets, plugin-sharing Sharing Rules/Record Shares, plugin-approvals, plugin-webhooks)", - "GET /api/v1/meta/app?id=studio (studio.app.ts — static nav: Overview, Data Model, User Experience, Logic, Automation, AI, Developer, Integration) and GET /api/v1/meta/app?id=account (account.app.ts — Profile + Inbox/Security/Developer groups)", + "GET /api/v1/meta/app with NO query first and record the names actually served — a stock boot answers exactly showcase_app, setup, account. Then GET /api/v1/meta/app?id=account (account.app.ts — Profile + Inbox/Security/Developer groups). ⚠️ GET /api/v1/meta/app?id=studio answers 200 {\"items\":[]} on every stock boot and that is the expected result (knownGap): record the absence as an assertion and move on — do NOT install @objectstack/studio to make the step runnable", "as admin, hand-walk each destination in each app: navigate, wait for render, SCREENSHOT FIRST, then read the DOM — assert no pageerror, no 'failed to load', no blank
, no 'no actions configured' placeholder", "Setup detail: confirm each settings URL entry (nav_settings_* → /apps/setup/system/settings/) opens the settings namespace form (localization/company/branding/auth/mail/storage/ai/knowledge/feature_flags), and Users / Organization / Business Units / Teams / Sessions / OAuth Applications / Identity Links / User Preferences render", - "Studio detail: confirm each metadata:resource list (object/app/view/page/dashboard/report/dataset/action/hook/flow/agent/tool/skill/email_template) renders, and the component surfaces render (App Builder studio:builder, All Metadata Types metadata:directory, Packages developer:packages, API Console developer:api-console, Flow Runs developer:flow-runs, Public Forms developer:public-forms)", + "Studio detail is NOT walked from here — the app is not registered on a stock boot (knownGap), so there is no merged nav to render. Its metadata:resource lists and component surfaces (App Builder, All Metadata Types, Packages, API Console, Flow Runs, Public Forms) are reached through the console surface at /_console/studio// and are covered by area studio-authoring. What THIS item asserts about Studio is the absence itself, per clause 6", "Account detail: confirm Profile (account:profile_card), Notifications (sys_inbox_message/mine), Approvals (sys_approval_request/my_pending), Linked Accounts (sys_account), Active Sessions (sys_session/mine), API Keys (sys_api_key/mine), OAuth Applications (sys_oauth_application/mine) each render", "confirm the gated entries resolve to ABSENT-not-erroring for the admin: nav_organizations (requiresService org-scoping) is absent in single-org mode; nav_jwks Signing Keys is ABSENT for EVERY persona, admin included — the entry was removed in #7544 because sys_jwks declares apiEnabled:false, so its list answers OBJECT_API_DISABLED (404) for all callers and no permission gate could prune it; SSO Providers is absent unless OS_SSO_ENABLED (knownGap)", - "sign in as the plain member: attempt to open Setup (app name setup) and Studio (app name studio) — capture the app-level refusal (App.requiredPermissions setup.access / studio.access); open Account (declares no requiredPermissions) — capture it opening", + "sign in as the plain member: attempt to open Setup (app name setup) — capture the app-level refusal (App.requiredPermissions setup.access); open Account (declares no requiredPermissions) — capture it opening. Studio is not part of this comparison: it is absent for EVERY persona including the admin, so a member failing to reach it proves nothing about the gate (the same reasoning #7544 applied to nav_jwks)", "as the member, GET /api/v1/meta/app for a reachable app and confirm every manage_platform_settings-gated entry (API Keys, the manage_platform_settings settings URLs) is ABSENT from the member's payload — the server prunes, the client does not merely hide. JWKS is deliberately NOT in this list any more: it is absent for admins too, so finding it absent for a member proves nothing about the gate (#7544)", "cross-check served-vs-rendered: diff each app's meta/app nav destinations against what actually rendered so nothing is silently outside the walk" ], "acceptance": [ { - "clause": "every merged-nav destination in all three apps renders a real surface for the admin — no pageerror, no 'failed to load', no blank
, no placeholder leak; this closes the gap platform-core.nav-surfaces-render leaves (it only sweeps the showcase app)", + "clause": "every merged-nav destination in each REGISTERED built-in app (Setup, Account) renders a real surface for the admin — no pageerror, no 'failed to load', no blank
, no placeholder leak; this closes the gap platform-core.nav-surfaces-render leaves (it only sweeps the showcase app)", "oracle": "screenshot", - "verify": "per-destination screenshot (screenshot-first) then DOM mark-check, across Setup + Studio + Account", + "verify": "per-destination screenshot (screenshot-first) then DOM mark-check, across Setup + Account — the two built-ins a stock boot registers. Scoring this clause over two apps is FULL coverage of what boots, not a partial: Studio has no merged nav to walk (knownGap)", "evidence": "per-app per-destination screenshot set + verdict table" }, { - "clause": "the served merged nav matches each app's authored shell + contributions: Setup's group anchors (setup.app.ts) are filled by SETUP_NAV_CONTRIBUTIONS and capability plugins; Studio's static groups (studio.app.ts) are all present; Account's Profile/Inbox/Security/Developer groups present", + "clause": "the served merged nav matches each registered app's authored shell + contributions: Setup's group anchors (setup.app.ts) are filled by SETUP_NAV_CONTRIBUTIONS and capability plugins; Account's Profile/Inbox/Security/Developer groups present", "oracle": "api", - "verify": "the three GET /meta/app payloads list the expected group ids + entries per the app source files", + "verify": "the two GET /meta/app payloads list the expected group ids + entries per the app source files", "evidence": "the three nav payloads" }, { - "clause": "as a plain member, Setup and Studio REFUSE (app-level requiredPermissions setup.access / studio.access) with a named access-denied surface — never a blank shell — while Account OPENS (declares no requiredPermissions, RLS scopes its rows)", + "clause": "as a plain member, Setup REFUSES (app-level requiredPermissions setup.access) with a named access-denied surface — never a blank shell — while Account OPENS (declares no requiredPermissions, RLS scopes its rows)", "oracle": "screenshot", - "verify": "run the two refusals + the Account open as the member persona (wrong-persona trap — do it as the member, not the admin)", + "verify": "run the Setup refusal + the Account open as the member persona (wrong-persona trap — do it as the member, not the admin). Studio is excluded deliberately: it is absent for the admin too, so a member-side failure there is not attributable to studio.access", "evidence": "three screenshots" }, { @@ -517,10 +524,10 @@ "evidence": "the forged response" }, { - "clause": "permission-gated ENTRIES are absent from the member's merged nav, not present-and-erroring: JWKS (nav_jwks), API Keys (nav_api_keys) and the manage_platform_settings settings entries do not appear in the member's payload", + "clause": "the entry-level gate for Setup's permission-gated entries (nav_api_keys, the manage_platform_settings settings URLs) is enforced ONE LEVEL UP as a whole-app refusal: the member cannot obtain the setup app at all, so those entries are unreachable rather than pruned-from-a-served-nav. Assert the app-level absence; do NOT assert an admin-vs-member entry diff on stock fixtures, because there is no member-side Setup nav to diff", "oracle": "api", - "verify": "diff of admin vs member /meta/app nav destinations — the gated entries are only in the admin set", - "evidence": "the admin-vs-member diff" + "verify": "as the member, GET /api/v1/meta/app?id=setup → 200 {\"items\":[]}: no App document is served, so every entry inside it (including nav_api_keys and the manage_platform_settings settings URLs) is unreachable. ⚠️ Do NOT substitute the account app for this: account is the one app a member CAN obtain and it carries no permission-gated entries at all — every one of its entries is gated by requiresObject/requiresService (nav_account_api_keys → sys_api_key, nav_account_sessions → sys_session, nav_account_linked → sys_account, nav_account_approvals → requiresService approvals, nav_account_memberships → requiresService org-scoping; account.app.ts), which are PRESENCE gates that resolve identically for both personas. Admin and member account navs measured identical (7 vs 7) — that is the contract behaving as authored and it proves nothing about permission gating, so scoring the clause off it is a false pass. The prior wave recorded the same shape as a named limitation on its own c4 (#9453)", + "evidence": "the member's empty ?id=setup payload, plus the admin-vs-member account nav comparison shown as the CONTROL that carries no gated entries (not as the gate's proof)" }, { "clause": "service/object-gated entries resolve to ABSENCE on stock fixtures rather than an entry that can only error: nav_organizations (requiresService org-scoping) absent in single-org mode; nav_business_units (requiresObject sys_business_unit) present only when the object is registered", @@ -551,7 +558,13 @@ ], "history": [ { "revision": 1, "date": "2026-08-08", "change": "initial — the big built-in-apps nav sweep (Setup/Studio/Account merged nav render + app-level and entry-level gates both-sides), complementing nav-surfaces-render which only walks the showcase app", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 2, "date": "2026-08-18", "change": "corrected every ?id= from the package id (com.objectstack.setup/.studio/.account) to the App document NAME (setup/studio/account) across steps 0, 1, 7 and clause 4. rest-server.ts matches a.name === id, so the package-id spelling returns an empty items array on every one of them — indistinguishable from a missing built-in app (#9334). Kept the package ids where they are genuinely the subject (the ADR-0048 one-app-package note), spelled as PACKAGE ids", "ref": "#9386" } + { "revision": 2, "date": "2026-08-18", "change": "corrected every ?id= from the package id (com.objectstack.setup/.studio/.account) to the App document NAME (setup/studio/account) across steps 0, 1, 7 and clause 4. rest-server.ts matches a.name === id, so the package-id spelling returns an empty items array on every one of them — indistinguishable from a missing built-in app (#9334). Kept the package ids where they are genuinely the subject (the ADR-0048 one-app-package note), spelled as PACKAGE ids", "ref": "#9386" }, + { + "revision": 3, + "date": "2026-08-21", + "change": "two corrections from the 17.1.0 sweep, both about premises that fail one level up. (1) B2/Studio: the item named THREE built-in apps, so on stock fixtures it could never exceed partial — c0/c1/c2 were blocked(fixture) for one reason. Verified at source rather than assumed: Studio's absence is a deliberate product decision at the BOOT layer, not a showcase choice — both os dev and os serve carry the identical note that @objectstack/studio is intentionally not default-loaded because the console ships a dedicated Studio surface at /_console/studio//, and each registers only setup and account. Title, fixtures, steps 1/4/7 and clauses 0/1/2 now describe the two apps a stock boot registers; Studio's absence became an assertion with the carve-out recorded in knownGaps, and its surfaces stay covered by area studio-authoring. (2) A4/clause 4: the clause asked for permission-gated ENTRIES to be absent from the member's merged nav, but the member cannot obtain the setup app at all (GET /meta/app?id=setup → 200 items:[]), so no member-side Setup nav exists to diff — the gate is a whole-app refusal, which clause 3 already proves. Reworded for that case, with the account app named explicitly as a non-substitute: every account entry is gated by requiresObject/requiresService, so admin and member navs are identical (7 vs 7) by construction and scoring the clause off that comparison is a false pass (#10236 A4+B2, measured #10257, same shape as #9453's c4)", + "ref": "#10236" + } ] }, { diff --git a/docs/qa/platform-checklist/areas/records-forms.json b/docs/qa/platform-checklist/areas/records-forms.json index bd78d607b7..2bb76959d0 100644 --- a/docs/qa/platform-checklist/areas/records-forms.json +++ b/docs/qa/platform-checklist/areas/records-forms.json @@ -8,7 +8,7 @@ "title": "Create → read → update → delete a record through the console UI", "since": "v15", "status": "active", - "revision": 4, + "revision": 5, "priority": "P0", "surface": "browser", "personas": [ @@ -18,6 +18,9 @@ "app": "showcase", "requires": [ "showcase_account — writable standard object (sharingModel public_read_write), required name + status, format validations tax_id_format / billing_email_format (examples/app-showcase/src/data/objects/account.object.ts)" + ], + "knownGaps": [ + "Clause 7 (clone is RLS-gated) cannot be scored on showcase_account, the object every other clause here drives: it is public_read_write, so there is no persona for whom a source row is invisible. It needs showcase_invoice plus a contributor-bound member — the recipe qa-contributor-bound-member in areas/search.json. ⚠️ The recipe mechanism is AREA-SCOPED (provisioning.use must name a key in this area's own fixtures block, README.md; unresolved by the validator either way, deferred at #7716 / tracked #7720), so this item cross-references the recipe by name rather than opting in. Replay it from there; do not fork a second copy into this area." ] }, "steps": [ @@ -78,8 +81,8 @@ { "clause": "clone is RLS-gated: the source is fetched in the caller's context (engine.findOne with context), so cloning a record the caller cannot SEE is refused with 404 RECORD_NOT_FOUND — never a silent duplicate of an invisible row", "oracle": "api", - "verify": "as a persona without read on the source row, POST the clone route → 404 RECORD_NOT_FOUND and no new row lands (recordNotFoundError from the null findOne); an enable.clone:false object refuses with 403 CLONE_DISABLED", - "evidence": "the RLS refusal + a post-attempt count showing no new row" + "verify": "⚠️ NOT on showcase_account — it is public_read_write, so no persona can fail to read a source row there and the clause is unscoreable on the object the rest of this item drives. Drive it on showcase_invoice with a contributor-bound member (recipe qa-contributor-bound-member in areas/search.json; the area-scoped recipe mechanism means this item cannot `use` it — replay it from there): as that member, POST the clone route against INV-1003 (linus@example.com's, invisible to them) → 404 RECORD_NOT_FOUND and no new row lands (recordNotFoundError from the null findOne). ⛔ Distinguish the two refusals before scoring: 404 RECORD_NOT_FOUND means the caller cannot SEE the source (this clause); 403 PERMISSION_DENIED means they can see it but hold no create grant — a DIFFERENT gate, and the one a persona with read-but-not-create hits. The 17.1.0 sweep hit exactly that confusion by cloning showcase_account as Mei, who reads it fine (#10236). Separately, an enable.clone:false object refuses with 403 CLONE_DISABLED", + "evidence": "the 404 RLS refusal with its code shown (not merely its status), the premise read proving the source row was invisible to that persona first, and a post-attempt count showing no new row" } ], "negative": [ @@ -120,7 +123,13 @@ "change": "added the /data/:object/:id/clone clauses (new id + copied field values, cloner ownership, engine/readonly column re-derivation, RLS-invisible source refused RECORD_NOT_FOUND, enable.clone gate); inline-edit two-surface behavior placed in concurrent-edit-conflict with a cross-ref here", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 4, "date": "2026-08-18", "change": "corrected a negative that can never hold for the object the item drives. showcase_account.status is required:true, but its 'prospect' option carries default:true (examples/app-showcase/src/data/objects/account.object.ts), so a POST omitting status is created with status:'prospect' — correct behaviour that the negative as written scores as a server-side validation failure (#9334)", "ref": "#9386" } + { "revision": 4, "date": "2026-08-18", "change": "corrected a negative that can never hold for the object the item drives. showcase_account.status is required:true, but its 'prospect' option carries default:true (examples/app-showcase/src/data/objects/account.object.ts), so a POST omitting status is created with status:'prospect' — correct behaviour that the negative as written scores as a server-side validation failure (#9334)", "ref": "#9386" }, + { + "revision": 5, + "date": "2026-08-21", + "change": "named the object and persona clause 7 actually needs, and separated its refusal from the one next to it. The clause asks for a clone of a row the caller cannot SEE to be refused 404 RECORD_NOT_FOUND, but every other clause in this item drives showcase_account, which is public_read_write — no persona can fail to read a row there, so the clause was unscoreable on its own item's object. The 17.1.0 sweep hit this directly: cloning showcase_account as a seeded demo persona produced 403 PERMISSION_DENIED (a missing CREATE grant), a different gate that a status-only reading would have scored as the RLS refusal. Clause 7 now names showcase_invoice + INV-1003 + a contributor-bound member (recipe qa-contributor-bound-member, areas/search.json), requires the code and not just the status, requires the invisibility premise to be read first, and states the 404-vs-403 discriminator. A knownGap records the cross-area recipe reference and why `use` is not available here (#10236 B1)", + "ref": "#10236" + } ] }, { @@ -632,7 +641,7 @@ "title": "Header-level conditional rules: visibleWhen / requiredWhen / readonlyWhen on invoice fields, enforced on BOTH ends", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "mixed", "personas": [ @@ -643,6 +652,9 @@ "requires": [ "showcase_invoice conditional fields (examples/app-showcase/src/data/objects/invoice.object.ts): issued_on requiredWhen status in ['sent','paid']; tax_rate readonlyWhen status=='paid'; paid_on visibleWhen+requiredWhen status=='paid'; userActions edit disabledWhen / delete visibleWhen on paid", "seeded invoices: INV-1002 (draft), INV-1001 (sent), INV-1003 (paid)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -718,6 +730,12 @@ "date": "2026-08-07", "change": "initial — deep-test both-ends contract for header-level conditional rules on the seeded invoices", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -726,7 +744,7 @@ "title": "Grid-level conditional rules: row-scoped requiredWhen and parent-scoped readonlyWhen in the inline line-item grid", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "browser", "personas": [ @@ -737,6 +755,9 @@ "requires": [ "showcase_invoice_line rules (examples/app-showcase/src/data/objects/invoice.object.ts): description requiredWhen record.quantity >= 100 (ROW-scoped); product/quantity/unit_price readonlyWhen parent.status == 'paid' (PARENT-scoped); receipt Field.file upload cell (objectui#2360)", "a draft invoice with editable lines (INV-1002) and a paid one (INV-1003)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -802,6 +823,12 @@ "date": "2026-08-07", "change": "initial — deep-test grid conditional rules split from the header item (different scopes, different pins)", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -810,7 +837,7 @@ "title": "Cascading and gated select options: dependsOn + per-option visibleWhen, client narrows / server rejects", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "mixed", "personas": [ @@ -822,6 +849,9 @@ "requires": [ "showcase_cascade (examples/app-showcase/src/data/objects/cascading-select.object.ts): province dependsOn ['country'] with per-option visibleWhen (cn → zj/gd, us → ca/tx); tier option 'restricted' gated on 'admin' in current_user.positions; sharingModel public_read_write", "showcase_invoice.contact dependsOn ['account'] — the dependent LOOKUP twin (contact picker scoped to the chosen account's people; seed gives Northwind 26 contacts, Contoso 2, Fabrikam 1)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -889,6 +919,12 @@ "date": "2026-08-07", "change": "initial — deep-test dynamic options: cascade, role gate, dependent lookup, server-side rejection", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -897,7 +933,7 @@ "title": "Master + line items save as one atomic batch, with server-side roll-up", "since": "v16", "status": "active", - "revision": 2, + "revision": 3, "priority": "P1", "surface": "browser", "personas": [ @@ -908,6 +944,9 @@ "requires": [ "showcase_invoice + showcase_invoice_line with inlineEdit: 'grid' on the master_detail field (examples/app-showcase/src/data/objects/invoice.object.ts) — every New/Edit Invoice form renders the editable Line Items grid", "seeded products for the catalog lookup (SKUs incl. SERVICE-HR, WIDGET-A; picking one auto-fills description + unit_price)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -986,6 +1025,12 @@ "date": "2026-08-07", "change": "expanded to deep-test contract: concrete steps, multi-clause acceptance, negatives, variants", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 3, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -994,7 +1039,7 @@ "title": "Related lists derive from the relationship and page on the server — never fetch every child row — and are read-gated on both ends", "since": "v16", "status": "active", - "revision": 3, + "revision": 4, "priority": "P1", "surface": "browser", "personas": [ @@ -1010,7 +1055,8 @@ "the permission-zoo sets examples/app-showcase/src/security/permission-sets.ts: showcase_manager grants showcase_contact read (line 124), showcase_contributor omits it (lines 36-45) — the both-sides read-gate probe binds a user to each" ], "knownGaps": [ - "the child-read-gate clause needs two signed-up users bound to showcase_manager and showcase_contributor — sys_user rows come from sign-up, not seeds — so on a single-user stock boot record that clause blocked(fixture); the paging clauses run on the seeded admin alone" + "the child-read-gate clause needs two signed-up users bound to showcase_manager and showcase_contributor — sys_user rows come from sign-up, not seeds — so on a single-user stock boot record that clause blocked(fixture); the paging clauses run on the seeded admin alone", + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -1099,6 +1145,12 @@ "date": "2026-08-08", "change": "added the §E3 read-gating both-sides clause (child-object read gate: related section absent in the UI AND child query 403s server-side) with the permission-zoo personas (manager reads child, contributor does not); objectui#2359/#2565", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 4, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -1283,7 +1335,7 @@ "title": "CSV import: GBK decode, required-field hint, legacy-fallback notice", "since": "v16", "status": "active", - "revision": 3, + "revision": 4, "priority": "P2", "surface": "browser", "personas": [ @@ -1293,6 +1345,9 @@ "app": "showcase", "requires": [ "an importable showcase object (e.g. showcase_contact) and a GBK-encoded CSV fixture; the pinned unit suites carry their own byte fixtures" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -1356,7 +1411,13 @@ "change": "expanded to deep-test contract: concrete steps, multi-clause acceptance, negatives, variants", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 3, "date": "2026-08-18", "change": "spelled the two companion refs as full paths. They were named bare, and they do not sit beside importParsers.test.ts — they live one directory down in packages/plugin-grid/src/__tests__/ with a .tsx extension, so the bare names cost the next runner a search in the wrong place (#9401)", "ref": "#9386" } + { "revision": 3, "date": "2026-08-18", "change": "spelled the two companion refs as full paths. They were named bare, and they do not sit beside importParsers.test.ts — they live one directory down in packages/plugin-grid/src/__tests__/ with a .tsx extension, so the bare names cost the next runner a search in the wrong place (#9401)", "ref": "#9386" }, + { + "revision": 4, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, { @@ -1364,7 +1425,7 @@ "title": "Action buttons surface at exactly their declared locations — list toolbar, list row, record header/more/related/section — and each dispatches for real", "since": "v15", "status": "active", - "revision": 4, + "revision": 5, "priority": "P1", "surface": "browser", "personas": [ @@ -1375,6 +1436,9 @@ "requires": [ "the per-location action fleet on showcase_task (examples/app-showcase/src/ui/actions/index.ts): showcase_bulk_reassign (list_item+list_toolbar, flow), showcase_quick_view (list_item, modal), showcase_mark_done (list_item+record_header+record_section, script, visible '!record.done'), showcase_log_time (record_header+record_related+record_section, form), showcase_open_docs (record_more, url), showcase_recalc_selection (record_more, api — declares record_more ONLY, which is what keeps it off the toolbar; objectui#3142), showcase_new_task (locations: [] — the fleet's headless specimen, read by the empty-locations probe)", "seeded tasks in both done and not-done states for the CEL visibility both-sides check" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "variants": [ @@ -1472,6 +1536,12 @@ "date": "2026-08-10", "change": "inverted the empty-locations probe to the post-objectui#3142 contract (#7323). The variant, its step and its acceptance clause all asserted that a locations-less action lands on EVERY location; objectui#3142 collapsed four disagreeing renderers onto one membership predicate (actionRendersAt), so such an action now lands on NONE. The probe was therefore grading correct behavior as a FAIL, and contradicted this same item's negative list, which already encoded the post-#3142 contract. The probe is kept rather than deleted — both halves stay observable end-to-end: no `locations` key → nowhere (the inert shape packages/lint's action-no-placement warns about), and `locations: []` → nowhere deliberately (read off the fleet's showcase_new_task, which revision 3 made headless). Re-anchored the recalc_selection note — record_more is an explicit placement, not an opt-out of an everywhere-default — reworded the objectui source line to name the predicate, and added the in-repo codification as a source", "ref": "claude/issue-7323-action-location-matrix-probe" + }, + { + "revision": 5, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ], "enumSource": { @@ -1564,7 +1634,7 @@ "title": "Gantt is interactive, not a picture: drag persists, locked tasks survive auto-schedule, host veto restores", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ @@ -1576,7 +1646,8 @@ "the task gantt view (task.view.ts 'Schedule (Gantt)': startDateField/endDateField/titleField/progressField)" ], "knownGaps": [ - "the stock view declares NO dependenciesField / lockField / parentField, so auto-schedule + locked-task + subtree-drag surfaces are unreachable on stock fixtures — needs the view extended (objectui carries unit pins for those behaviors: GanttView.autoscheduledlg / summaryedit / deptypes tests)" + "the stock view declares NO dependenciesField / lockField / parentField, so auto-schedule + locked-task + subtree-drag surfaces are unreachable on stock fixtures — needs the view extended (objectui carries unit pins for those behaviors: GanttView.autoscheduledlg / summaryedit / deptypes tests)", + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "variants": [ @@ -1641,6 +1712,12 @@ "date": "2026-08-07", "change": "initial — #3358 §8 interaction rows were never imported; render-only coverage existed in view-type-gallery", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -1712,12 +1789,18 @@ "title": "Dirty forms guard navigation: discard prompts, save proceeds, nothing is lost silently", "since": "v15", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ "admin" ], + "fixtures": { + "app": "showcase", + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." + ] + }, "steps": [ "open a record edit form (modal or drawer); change one field WITHOUT saving", "attempt to close the dialog / navigate away; capture the guard prompt", @@ -1766,6 +1849,12 @@ "date": "2026-08-07", "change": "initial — the dirty-state guard existed (with a unit pin) but no checklist item asserted it", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -1836,7 +1925,7 @@ "title": "User-built FilterBuilder/SortBuilder toolbar merges with the view filter, restores across nav, and reproduces rows from the URL", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "browser", "personas": [ @@ -1848,6 +1937,9 @@ "requires": [ "showcase_task with its saved views (examples/app-showcase/src/ui/views/task.view.ts) — the in_progress view carries a stored filter (status equals in_progress) so a toolbar condition can be proven to MERGE with, not replace, the view filter", "10 seeded tasks spanning statuses/priorities/estimate_hours so a 2-condition toolbar filter returns a non-trivial subset (examples/app-showcase/src/data/seed/index.ts)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -1920,6 +2012,12 @@ "date": "2026-08-08", "change": "initial — deep-test the ad-hoc FilterBuilder/SortBuilder toolbar: view-filter merge + $orderby, localStorage nav-restore (user-scoped), uf_* URL reproduction, ADR-0053 leak negative", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -1928,7 +2026,7 @@ "title": "Calendar is interactive: drag reschedules via PATCH, a failing PATCH reverts with an error, mode switches re-render, empty-day click quick-creates", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "browser", "personas": [ @@ -1939,6 +2037,9 @@ "requires": [ "the showcase_task Calendar view (examples/app-showcase/src/ui/views/task.view.ts: named 'Calendar' view + allowedVisualizations calendar, calendar.startDateField due_date, titleField title, colorField status)", "10 seeded tasks carrying due_date values so cards land on distinct days (examples/app-showcase/src/data/seed/index.ts)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "variants": [ @@ -2006,6 +2107,12 @@ "date": "2026-08-08", "change": "initial — calendar was render-only in view-type-gallery; this deep-tests the drag→PATCH→revert chain, mode switches, and empty-day quick-create", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2014,7 +2121,7 @@ "title": "Concurrent edits collide loudly (OCC 409 + conflict dialog), never silent last-write-wins; inline two-surface edit is ONE atomic OCC-guarded save", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "mixed", "personas": [ @@ -2025,6 +2132,9 @@ "requires": [ "any writable object whose rows carry an updated_at OCC token (e.g. showcase_account or showcase_task) — the form reads updated_at and sends it as If-Match (objectui plugin-form/src/occSave.tsx; framework packages/metadata-protocol/src/protocol.ts updateData If-Match handling)", "a record with both a details-body field and a header-highlight field for the two-surface inline-edit fold-in (RecordDetailView highlight fields)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -2102,6 +2212,12 @@ "date": "2026-08-08", "change": "initial — deep-test OCC conflict (409 + dialog, reload/overwrite) and fold in the §E1 inline-edit atomic two-surface behavior (ONE save bar / ONE PATCH / ifMatch)", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2199,7 +2315,7 @@ "title": "Grid personalization persists across reload: column resize/reorder/pin, row-height, group-by (with API-true totals), row-color", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ @@ -2209,6 +2325,9 @@ "app": "showcase", "requires": [ "showcase_task grid view with a groupable field (status) and a summarizable field (estimate_hours) — examples/app-showcase/src/ui/views/task.view.ts; 10 seeded tasks across statuses (examples/app-showcase/src/data/seed/index.ts)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "variants": [ @@ -2284,6 +2403,12 @@ "date": "2026-08-08", "change": "initial — deep-test grid personalization persistence (columns/row-height/grouping/row-color) with API-true grouping totals", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2381,7 +2506,7 @@ "title": "Select-all-matching covers every matching id across pages (server total, not the rendered page); clear resets both selection sources", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ @@ -2392,6 +2517,9 @@ "requires": [ "a list whose filtered result exceeds one page: showcase_contact filtered to Northwind's account has 26 rows (2 named + 24 'Prospect NN'), authored to exceed a page (examples/app-showcase/src/data/seed/index.ts, objectui#2711)", "a bulk action registered on the object so the BulkActionBar and its aggregate/per-record dispatch are exercisable (objectui plugin-grid BulkActionBar)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -2459,6 +2587,12 @@ "date": "2026-08-08", "change": "initial — deep-test cross-page select-all-matching: server total, across-pages id coverage vs a filtered API count, clear-resets-both (extends the page-local bulk in list-view-capabilities)", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2467,7 +2601,7 @@ "title": "Lookup picker quick-create: a user-facing zero-hit lookup opens the referenced create form and adopts the new id; system references offer no quick-create", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ @@ -2478,6 +2612,9 @@ "requires": [ "a user-facing lookup with allowCreate: showcase_business_unit.parent (Field.lookup('showcase_business_unit', {allowCreate:true})) or showcase_category.parent (allowCreate:true) — examples/app-showcase/src/data/objects/{business-unit,category}.object.ts", "a system/user-directory reference for the negative side (any sys_/cloud_/ai_ reference, or a user/users directory field) — the picker must NOT offer quick-create there" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -2539,6 +2676,12 @@ "date": "2026-08-08", "change": "initial — deep-test lookup quick-create: create-new opens the referenced form + id adoption + parent re-read, scoped search, and the sys_/user-directory no-quick-create both-sides", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2547,7 +2690,7 @@ "title": "Record edit undo: toast Undo / Ctrl+Z restores the prior value through the API and logs a revert; undo after a concurrent change must not silently clobber", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "browser", "personas": [ @@ -2558,6 +2701,9 @@ "app": "showcase", "requires": [ "a writable record and an undoable edit path (an inline edit or an action flagged undoable) — objectui react/src/hooks/useGlobalUndo.ts + RecordDetailView toast Undo action; core globalUndoManager" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -2617,6 +2763,12 @@ "date": "2026-08-08", "change": "initial — deep-test edit undo: API-true restore via toast/Ctrl+Z, auditable revert, and the unguarded concurrent-clobber observation", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, @@ -2625,7 +2777,7 @@ "title": "Async import job: terminal state + per-row results, undo removes exactly the imported rows, cancel mid-job leaves a coherent partial", "since": "v17", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "mixed", "personas": [ @@ -2638,7 +2790,8 @@ "the objectui import specs, which self-gate on IMPORT_CONSOLE_LIVE=1 (console) or a reachable import harness /live.html" ], "knownGaps": [ - "the async import + undo/cancel path is gated: e2e/import-console/import-console-undo.spec.ts skips unless IMPORT_CONSOLE_LIVE=1 with an import-job-capable client wired; e2e/import-harness/import-undo.spec.ts skips unless the harness origin serves /live.html — on a stock showcase boot these do not run, so record blocked(fixture)" + "the async import + undo/cancel path is gated: e2e/import-console/import-console-undo.spec.ts skips unless IMPORT_CONSOLE_LIVE=1 with an import-job-capable client wired; e2e/import-harness/import-undo.spec.ts skips unless the harness origin serves /live.html — on a stock showcase boot these do not run, so record blocked(fixture)", + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "blocked": { @@ -2702,6 +2855,12 @@ "date": "2026-08-08", "change": "initial — deep-test async import jobs (terminal + per-row results, exact-set undo, coherent cancel); blocked(fixture) on the IMPORT_CONSOLE_LIVE / harness gate, pinned to the self-gating specs", "ref": "claude/platform-test-checklist-ocwugl" + }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" } ] }, diff --git a/docs/qa/platform-checklist/areas/search.json b/docs/qa/platform-checklist/areas/search.json index b98f2a5518..fbd4fd2a7f 100644 --- a/docs/qa/platform-checklist/areas/search.json +++ b/docs/qa/platform-checklist/areas/search.json @@ -2,6 +2,59 @@ "$comment": "Standing platform test checklist — area ledger. Hand-edited, append-only; validated by scripts/check-platform-checklist.mjs (pnpm check:platform-checklist). Authoring rules: docs/qa/platform-checklist/README.md · execution protocol: RUNNER.md.", "area": "search", "title": "Record search — $search executor, field scoping, RLS composition, pinyin recall, freshness", + "fixtures": { + "$comment": "AREA-LEVEL standing fixtures. An item opts in with fixtures.provisioning = { use: \"\", why: \"…\" }; the recipe is written ONCE here so several items do not carry drifting copies of the same call sequence. Recipes are runtime-provisioned (no repo file is touched, nothing to seed) and every call is grounded in framework source at the cited file:line — replay them literally, and if one 4xxs, re-read the citation before assuming the recipe rotted. Shape copied from qa-scratch-authz in areas/attachments-storage.json (#7716/#7670).", + "qa-contributor-bound-member": { + "title": "A CONTRIBUTOR-BOUND member — an identity that reads a strict SUBSET of showcase_invoice rows (fresh sign-up + one sys_user_position row)", + "why": "Both-sides RLS needs a persona for whom a specific row is invisible while the object itself is readable. ⚠️ Neither seeded demo persona is that persona, and reaching for them is the trap this recipe closes — measured on 17.1.0 (#10257): Mei (phone.demo@example.com) holds NO position, so showcase_contributor never merges and showcase_invoice answers PERMISSION_DENIED — the OBJECT gate, no rows at all, not a subset; Ada (auditor.demo@example.com) holds `auditor` → showcase_auditor, which carries `showcase_invoice: { allowRead: true, viewAllRecords: true }` — VAMA bypasses record scope entirely, so she sees all 12 — also not a subset. The persona that IS a subset is a member bound to the `contributor` position: showcase_contributor grants read/create/edit on showcase_invoice while the `invoice_own_rows` RLS narrows selects to `owner == current_user.email`, and none of the seeded invoices are owned by a fresh sign-up. Nothing here needs a seed change or a repo file — the whole gap was that the recipe existed only as prose on one item, which is why a runner reached for the demo personas instead and recorded blocked(fixture).", + "provenance": "run #7629 (framework, 2026-08-11) drove search.rls-both-personas to 4/4 PASS on exactly this persona — 3 invoices visible to the member vs 15 to admin. Promoted from prose on that one item to a replayable area recipe by the 17.1.0 sweep (#10236 B1); the Mei/Ada measurements above are from #10257.", + "app": "showcase", + "requires": [ + "a stock dev boot — the position→permission-set bindings are created automatically at startup on `kernel:bootstrapped`, including ['contributor','showcase_contributor'] (examples/app-showcase/src/security/bind-position-sets.ts, POSITION_PERMISSION_SET_BINDINGS). ⛔ Do NOT hand-create a sys_position_permission_set row: it already exists, and a duplicate proves nothing about the merge", + "system-context write access for the position binding (the sys_user_position insert) — the same pattern access-security.json's permission-set items already replay", + "an isolated boot (own port + file DB, dogfood §0); export OS_PORT= as well as passing -p (RUNNER.md's connector self-URL fact) so unrelated flow noise does not pollute the run" + ], + "sequence": [ + { + "step": 1, + "call": "POST /api/v1/auth/sign-up/email", + "body": { "email": "qa-contributor@verify.test", "password": "", "name": "QA Contributor" }, + "expect": "2xx with a session for a brand-new principal. This works on a stock dev boot — three identities were minted in one call each in #10257 — so the identity half needs no seed change. Use a UNIQUE email per run: a re-run against a live DB collides on the existing account.", + "source": "packages/plugins/plugin-auth (better-auth email sign-up); exercised end-to-end by access-security.record-share-grant-revoke, which was driven on minted A/B/C identities in #10257" + }, + { + "step": 2, + "call": "resolve the new principal's sys_user id with a system-context read of sys_user by email", + "expect": "one row. Needed because step 3 binds by user id, not by email.", + "source": "the house pattern access-security.json's items already use ('fresh sign-ups whose sys_user ids are resolved system-side')" + }, + { + "step": 3, + "call": "insert a sys_user_position row in system context binding that user id to the `contributor` position", + "expect": "one row. This is THE step that makes the persona a subset reader: `contributor` resolves to showcase_contributor (auto-bound at boot, see requires), which grants showcase_invoice allowRead/allowCreate/allowEdit, and the RLS rule `invoice_own_rows` (using: `owner == current_user.email`, positions: ['contributor']) narrows every select to rows the caller owns.", + "source": "examples/app-showcase/src/security/permission-sets.ts — ContributorPermissionSet.objects.showcase_invoice and rowLevelSecurity['invoice_own_rows']; examples/app-showcase/src/security/positions.ts (the `contributor` position); bind-position-sets.ts for the position→set edge" + }, + { + "step": 4, + "call": "PROVE THE SUBSET before scoring anything: as the new member, GET /api/v1/data/showcase_invoice (list) and GET /api/v1/data/showcase_invoice/", + "expect": "the list answers 200 with ZERO of the twelve seeded invoices — not 403. A 403 means the position binding did not take (the OBJECT gate answered first, which is Mei's shape); a full 12 means it resolved to a VAMA-carrying set (Ada's shape). Both are mis-provisioning, and both would make a later verdict prove the opposite of what it reads. INV-1003 is linus@example.com's and is the canonical invisible row.", + "source": "examples/app-showcase/src/data/seed — INV-1001..INV-1012, owners ada@example.com ×5 / linus@example.com ×4 / grace@example.com ×3; ADR-0090 D5: every authenticated member also holds the `everyone` baseline additively, so compute the union before calling a row invisible" + }, + { + "step": 5, + "call": "give the member a row of their OWN: as the member, POST /api/v1/data/showcase_invoice (contributors hold allowCreate)", + "expect": "2xx, and that row IS visible to them on a re-read — turning the subset from empty into a proper non-empty one. This is what makes the restriction provably SUBTRACTIVE rather than a blackout; without it, zero-hits and no-access are indistinguishable.", + "source": "ContributorPermissionSet.objects.showcase_invoice.allowCreate === true (permission-sets.ts); the seeded owner emails hold no credential, so borrowing a seeded owner is not an option — this is why the own-row is CREATED" + } + ], + "teardown": "delete the sys_user_position row and the invoice created in step 5, or simply discard the isolated file DB — the cheaper path an isolated boot makes free. The signed-up principal is not otherwise cleaned up; use a unique per-run email so a leftover account never collides.", + "knownGaps": [ + "⚠️ The recipe mechanism is AREA-SCOPED: `provisioning.use` must name a key in the item's OWN area fixtures block (README.md), and the validator does not resolve it either way (deferred at #7716, tracked #7720). records-forms.crud-roundtrip clause 7 needs this same persona and lives in another area file, so it cross-references this recipe by name in its knownGaps instead of `use`-ing it. Replay it from here; do not fork a second copy.", + "The persona is a subset reader of showcase_invoice ONLY. showcase_account is public_read_write and showcase_contributor grants plain allowRead on it with no RLS, so a contributor still reads every account row — an item needing an invisible SOURCE row must drive showcase_invoice, not showcase_account.", + "showcase_private_note is an EMPTY table for everyone and showcase_account reads identically (15 rows) for admin, Mei and Ada (#10236) — neither is usable as a both-sides RLS control. Recorded so the next runner does not reach for them." + ] + } + }, "items": [ { "id": "search.cross-field-object-search", @@ -155,16 +208,20 @@ "title": "Search honors RLS both ways: a restricted member gets no hits — and no total leakage — from rows they cannot see; the entitled persona finds the same rows", "since": "v15", "status": "active", - "revision": 3, + "revision": 4, "priority": "P1", "surface": "api", - "personas": ["seeded admin (sees all invoices)", "signed-up member bound to the contributor position (invoice_own_rows RLS: owner == current_user.email)"], + "personas": ["seeded admin (sees all invoices)", "signed-up member bound to the contributor position via the qa-contributor-bound-member recipe (invoice_own_rows RLS: owner == current_user.email) — ⛔ NOT one of the seeded demo personas: Mei holds no position (PERMISSION_DENIED, object gate) and Ada holds auditor with viewAllRecords (all 12). See the recipe."], "fixtures": { "app": "showcase", "requires": [ "seeded invoices INV-1001..INV-1012 (12 rows) with owner emails (ada@example.com ×5 / linus@example.com ×4 / grace@example.com ×3) none of which belong to the test member — INV-1003 is linus's, which is why it is the invisible row the guard uses", - "a signed-up member holding the contributor position — positions are seeded, users are not: sign the member up, then bind contributor (sys_user_position) before the run" + "a signed-up member holding the contributor position — positions are seeded, users are not: sign the member up, then bind contributor (sys_user_position) before the run. The full replayable sequence, its premise guard and the two personas that do NOT work now live in the area recipe qa-contributor-bound-member (see fixtures.provisioning)" ], + "provisioning": { + "use": "qa-contributor-bound-member", + "why": "supplies the one persona every clause of this item rests on — a reader of a strict SUBSET of showcase_invoice. Without it all four clauses are unscoreable: clause 0 needs a row that is invisible to the member and visible to admin, clause 1 needs a caller-scoped population to reconcile against, and clause 2 needs the member to still find their own row. The 17.1.0 sweep recorded this item blocked(fixture) after driving the seeded demo personas instead — Mei is refused at the object gate and Ada carries viewAllRecords, so neither is a subset — which is the rediscovery this reference exists to stop (#10257)." + }, "knownGaps": [ "the seeded owner emails are metadata-only (users cannot be seeded; they sign up) — no stock login exists for ada@example.com, so the 'restricted member owns SOME rows' side is produced by having the member CREATE an invoice of their own (contributors hold allowCreate), not by borrowing a seeded owner" ] @@ -217,7 +274,13 @@ "history": [ { "revision": 1, "date": "2026-08-07", "change": "new item: RLS × $search both-personas contract; total-leakage clause grounded in the protocol's page-local total for searched lists and the #2737 count middleware fix, persona provisioning gap recorded honestly", "ref": "claude/platform-test-checklist-ocwugl" }, { "revision": 2, "date": "2026-08-11", "change": "stale fixture count corrected against the seed (run #7629): the showcase now ships 12 invoices INV-1001..INV-1012, not 8. Three places carried the stale number — the fixtures requirement, the no-count-leakage clause's 'true 8-row population', and the seed source citation. The leakage clause no longer hard-codes a population at all (a run reconciles against the seed and its own created rows, which is what #7629 actually did: 3 visible to the member, 15 to admin); the owner spread is now stated as counts (ada ×5 / linus ×4 / grace ×3) with INV-1003 named as linus's, so the invisible-row premise is checkable without opening the seed. Item substance and all four acceptance clauses are otherwise unchanged — #7629 scored this item 4/4 PASS", "ref": "#7647" }, - { "revision": 3, "date": "2026-08-18", "change": "bounded clause 2's reconciliation to skip <= the caller's visible count. The page-local total is derived from the RLS-filtered find, so paging past the visible set leaves nothing to reconcile and a runner comparing totals there reads a non-result as a leak. The security property the clause exists for (no count leakage of rows the caller cannot see) is untouched and was observed intact (#9417)", "ref": "#9386" } + { "revision": 3, "date": "2026-08-18", "change": "bounded clause 2's reconciliation to skip <= the caller's visible count. The page-local total is derived from the RLS-filtered find, so paging past the visible set leaves nothing to reconcile and a runner comparing totals there reads a non-result as a leak. The security property the clause exists for (no count leakage of rows the caller cannot see) is untouched and was observed intact (#9417)", "ref": "#9386" }, + { + "revision": 4, + "date": "2026-08-21", + "change": "promoted the contributor-bound-member provisioning from prose on this item to the replayable area recipe qa-contributor-bound-member, and named the two personas that do NOT satisfy it. The item was recorded blocked(fixture) by the 17.1.0 sweep, which reached for the seeded demo personas: Mei holds no position so showcase_invoice answers PERMISSION_DENIED at the OBJECT gate (no rows at all), and Ada holds auditor whose set carries viewAllRecords so she sees all 12 — neither is the strict subset every clause here rests on. Nothing was actually missing from the fixtures: #7629 drove this item 4/4 PASS on a fresh sign-up bound to `contributor`, whose invoice_own_rows RLS narrows selects to owner == current_user.email. The recipe now carries that sequence with a premise guard that distinguishes all three outcomes (403 = binding did not take, 12 rows = a VAMA set, 0 of 12 = correct), so the mis-provisioning is caught before any verdict rests on it (#10236 B1)", + "ref": "#10236" + } ] }, { diff --git a/docs/qa/platform-checklist/areas/studio-authoring.json b/docs/qa/platform-checklist/areas/studio-authoring.json index 37d5e5d0c2..28c54f49e6 100644 --- a/docs/qa/platform-checklist/areas/studio-authoring.json +++ b/docs/qa/platform-checklist/areas/studio-authoring.json @@ -93,7 +93,7 @@ "title": "Object designer round-trip: field add/edit/reorder persists to metadata, identifiers derive and survive typing", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "mixed", "personas": ["admin"], @@ -101,6 +101,9 @@ "app": "showcase", "requires": [ "a writable package to author into (create one via the Studio wizard or POST /api/v1/packages — the source-loaded com.example.showcase is read-only and rejects designer saves server-side)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -150,7 +153,13 @@ "docs/audits/2026-07-studio-package-create-ux-dogfood.md (finding 1 — read-only package rejects designer saves server-side, which is why a writable package is a fixture requirement)" ], "history": [ - { "revision": 1, "date": "2026-08-07", "change": "new item: designer→metadata round-trip with the meta API as oracle, pinning the four objectui live-e2e regression classes (F1–F4) and the audit's field_N identifier finding", "ref": "claude/platform-test-checklist-ocwugl" } + { "revision": 1, "date": "2026-08-07", "change": "new item: designer→metadata round-trip with the meta API as oracle, pinning the four objectui live-e2e regression classes (F1–F4) and the audit's field_N identifier finding", "ref": "claude/platform-test-checklist-ocwugl" }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, { @@ -228,13 +237,16 @@ "title": "Record-page authoring round-trip: created bound to its object, seeded from the default layout, block-edited, published, rendered", "since": "v16", "status": "active", - "revision": 2, + "revision": 3, "priority": "P1", "surface": "mixed", "personas": ["admin (authors)", "end user (opens the record)"], "fixtures": { "app": "showcase", - "requires": ["seeded showcase_invoice records to render the published page against (stock showcase seed)"] + "requires": ["seeded showcase_invoice records to render the published page against (stock showcase seed)"], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." + ] }, "steps": [ "open /_console/apps/showcase_app/metadata/page/new and fill the create form: Label 'Invoice Page ' (slugifies into Name), Object showcase_invoice; Save", @@ -295,7 +307,13 @@ ], "history": [ { "revision": 1, "date": "2026-08-07", "change": "new item: record-page authoring round-trip pinned to the two objectui live e2e specs (create-seeded draft, block picker) and extended to the publish + end-user render sides they do not cover", "ref": "claude/platform-test-checklist-ocwugl" }, - { "revision": 2, "date": "2026-08-11", "change": "marked both pinning specs STALE (run #7695): studio-record-page.spec.ts fills the Object control as an input when it is now a role=combobox button, and studio-editor.spec.ts targets a 'Layout' heading that no longer exists plus a shipped page the editor correctly locks. Neither reports a product defect — the annotation tells a runner not to cite their output and not to score their failure as this item's FAIL. Re-pointing the specs lives in objectui (cross-repo half of #7753 item 6); this repo only records the staleness", "ref": "#7753" } + { "revision": 2, "date": "2026-08-11", "change": "marked both pinning specs STALE (run #7695): studio-record-page.spec.ts fills the Object control as an input when it is now a role=combobox button, and studio-editor.spec.ts targets a 'Layout' heading that no longer exists plus a shipped page the editor correctly locks. Neither reports a product defect — the annotation tells a runner not to cite their output and not to score their failure as this item's FAIL. Re-pointing the specs lives in objectui (cross-repo half of #7753 item 6); this repo only records the staleness", "ref": "#7753" }, + { + "revision": 3, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, { @@ -591,7 +609,7 @@ "title": "The Studio CEL editors reach the SAME verdict as the engine: formula result-type inference, previous. completion in conditional rules, and RLS lint + test-run — the editor UI, not a second grammar", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P1", "surface": "mixed", "personas": ["admin"], @@ -603,7 +621,8 @@ "the @objectstack/formula engine present in the console bundle — celAuthoring lazy-loads it" ], "knownGaps": [ - "the CEL editors lazy-load @objectstack/formula and FEATURE-DETECT every entry point (celAuthoring.ts) — a missing/older engine degrades to no lint / no suggestions / test-run 'unavailable', never an exception. If the vendored bundle lacks the engine, record the affected clauses blocked(dependency) and verify the graceful-degradation clause instead" + "the CEL editors lazy-load @objectstack/formula and FEATURE-DETECT every entry point (celAuthoring.ts) — a missing/older engine degrades to no lint / no suggestions / test-run 'unavailable', never an exception. If the vendored bundle lacks the engine, record the affected clauses blocked(dependency) and verify the graceful-degradation clause instead", + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -690,7 +709,13 @@ "examples/app-showcase/src/data/objects/invoice.object.ts + project.object.ts + field-zoo.object.ts (stock formula-field expressions to author against)" ], "history": [ - { "revision": 1, "date": "2026-08-08", "change": "new item: the Studio CEL editor UI (formula result-type inference, previous. completion in conditional rules, RLS lint + test-run) round-tripping to the SAME @objectstack/formula verdicts the engine gives — grounded in celAuthoring.ts + CelPredicateField/CelTestRunDialog and cross-referencing the engine-parity items rather than re-proving them", "ref": "claude/platform-test-checklist-ocwugl" } + { "revision": 1, "date": "2026-08-08", "change": "new item: the Studio CEL editor UI (formula result-type inference, previous. completion in conditional rules, RLS lint + test-run) round-tripping to the SAME @objectstack/formula verdicts the engine gives — grounded in celAuthoring.ts + CelPredicateField/CelTestRunDialog and cross-referencing the engine-parity items rather than re-proving them", "ref": "claude/platform-test-checklist-ocwugl" }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, { @@ -698,7 +723,7 @@ "title": "Permission-matrix editor UX: field-filter + bulk apply to EXACTLY the visible fields, and the Bulk column never clips at narrow widths", "since": "v16", "status": "active", - "revision": 1, + "revision": 2, "priority": "P2", "surface": "mixed", "personas": ["admin"], @@ -708,6 +733,9 @@ "a writable permission set to edit in the Permission Matrix editor (env-scope metadata-admin, or a package permission set in the Studio Access pillar) — a read-only source-loaded set can be opened to verify the read-only gate but not to exercise bulk writes", "an object with MORE THAN 6 fields to expand in the matrix, so the field sub-table's filter renders (FieldsSubTable shows the filter only when fields.length > 6) — a stock showcase object with a wide field set (showcase_account / showcase_invoice)", "a narrow viewport for the Bulk-column clip check (B3)" + ], + "knownGaps": [ + "AUTOMATION IS PINNED IN THE objectui REPO, NOT HERE — `automated.ref` names objectui specs exclusively, so from this checkout the item is neither runnable nor pin-evidenced (no console bundle is built here and none of those specs exist here). Drive it by hand in the browser and score it as a manual run, OR run the pin in an objectui checkout and cite the output naming the revision (17.1.0 sweep subject: 9a3daf8d37ad). ⛔ Never score it as covered on the strength of the `automated` field alone — an unrun pin is a claim, not evidence. Full protocol: RUNNER.md, the objectui-pinned-automation standing fact." ] }, "steps": [ @@ -777,7 +805,13 @@ "access-security.permission-matrix-edit-loop (the grant→access flip — cross-referenced, not duplicated)" ], "history": [ - { "revision": 1, "date": "2026-08-08", "change": "new item: permission-matrix editor UX — field-filter + bulk apply scoped to exactly the visible fields (wrong scope writes wrong grants) and the Bulk column's anti-clip min-width, grounded in PermissionMatrixEditor.tsx + its fieldBulk/readonly tests; cross-refs the permission-matrix-edit-loop for the grant→access side", "ref": "claude/platform-test-checklist-ocwugl" } + { "revision": 1, "date": "2026-08-08", "change": "new item: permission-matrix editor UX — field-filter + bulk apply scoped to exactly the visible fields (wrong scope writes wrong grants) and the Bulk column's anti-clip min-width, grounded in PermissionMatrixEditor.tsx + its fieldBulk/readonly tests; cross-refs the permission-matrix-edit-loop for the grant→access side", "ref": "claude/platform-test-checklist-ocwugl" }, + { + "revision": 2, + "date": "2026-08-21", + "change": "recorded on the item that its automation is pinned exclusively in the objectui repo and is therefore neither runnable nor pin-evidenced from this checkout — one of 23 items in that position, each of which every sweep had been re-deriving from scratch. The knownGap names the two honest ways to score it (hand-drive it as a manual run, or run the pin in an objectui checkout and cite the revision) and forbids the third: treating the `automated` field itself as coverage. The protocol and the exclusive-vs-mixed distinction live once in RUNNER.md; this line is the pointer (#10236 A5)", + "ref": "#10236" + } ] }, {