Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 43 additions & 22 deletions docs/qa/platform-checklist/README.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -142,7 +142,9 @@ copy its shape rather than inventing a second one.
"app": "showcase",
"requires": ["…"], // item-specific needs stay here
"provisioning": {
"use": "qa-scratch-authz", // must match a key in the AREA's fixtures block
"use": "qa-scratch-authz", // a key in THIS area's fixtures block — or
// "<area>:<recipe>" for one another area owns
// (see "the qualified `use`" below)
"why": "which of THIS item's clauses the recipe unblocks, and what they'd score without it"
},
"knownGaps": ["CLOSED by the qa-scratch-authz recipe (#7670): … ; fall back to <pin> only if …"]
Expand All@@ -165,27 +167,46 @@ Why this shape:
*CLOSED-by-recipe*, naming any pinned fallback and asking the run to record **which**
of the two its verdict rests on. Deleting the gap loses the reason the recipe exists.

The validator **resolves `provisioning.use` against its own area's `fixtures` keys**: a
`use` naming a key that area does not define fails `check:platform-checklist`, naming the
item, the key that resolved to nothing, and the recipes the area does offer. This is
option C on #7716's open question — deferred at #7720 while the recipe shape lived in a
single area, landed at #10593 on its own stated condition, once the shape had spread to
three areas and six references.

Two things the resolve deliberately does **not** do:

- **It does not flag a recipe no item references.** Cross-area reuse has no spelling yet
(below), so a recipe whose only consumer lives in another area is referenced from that
item's `knownGaps` prose — invisible to the check. Redding the unreferenced direction
would answer the cross-area question by accident, in the direction of "recipes are
area-local", and that is a convention decision rather than a mechanical one.
- **It does not reach across areas.** Resolution is area-scoped because the mechanism is:
`use` names a key in the item's *own* file. A `use` pointing at another area's recipe
key is therefore a dangling pointer and fails — there is no qualified spelling
(`search:qa-contributor-bound-member` or similar) and no shared recipe file. A
cross-area consumer still cites the recipe **by name in `knownGaps`** and does not fork
a second copy; `records-forms.crud-roundtrip` clause 7 is the worked instance. Giving
that pointer a spelling the tooling can see is the open half of #10593.
#### Referencing a recipe another area owns — the qualified `use`

A recipe is proved by one area and sometimes needed by a second. `use` therefore has
**two spellings**, and the validator resolves both:

```jsonc
"use": "qa-scratch-authz" // a recipe of THIS item's own area
"use": "search:qa-contributor-bound-member" // "<area>:<recipe>" — one another area owns
```

The area half is the **filename stem** (`search` → `areas/search.json`). Prefer the bare
key when the recipe is your area's; reach for the qualifier when it is not — and **never
fork a second copy** of a recipe into your area, which is the drift this spelling exists
to prevent. `records-forms.crud-roundtrip` clause 7 is the worked instance: it needs
`qa-contributor-bound-member`, which `areas/search.json` owns.

This is option A on #10593's second gap, ruled by the maintainer on 2026-08-22 over a
shared `areas/_fixtures.json` (which would move a recipe away from the area that proved
it, and make a hot shared file out of one the area sharding deliberately keeps cold) and
over "recipes stay area-local, duplicate with a back-reference" (which accepts the drift).
Every reference already written keeps working: the qualifier is an addition, not a
migration.

The validator **resolves `provisioning.use` to a real recipe** — its own area's key, or
the named area's — and a reference that resolves to nothing fails `check:platform-checklist`,
naming the item and distinguishing the three ways it can fail: the named area does not
exist, the area exists but does not define that recipe, or the reference is malformed
(`a:b:c`, `search:`, a padded half). A bare key that some *other* area defines is told
the exact qualified spelling to write. This is option C on #7716's open question,
deferred at #7720 while the recipe shape lived in a single area and landed at #10593
once it had spread to three areas and six references.

One thing the resolve deliberately does **not** do:

- **It does not flag a recipe no item references.** It was left out while cross-area
reuse had no spelling, because redding it would have answered that question by
accident in the direction of "recipes are area-local". With the qualified spelling
ruled, that reason has expired and an unreferenced recipe is unambiguously dead text —
but turning the direction on is its own change with its own blast radius, tracked at
#11506 rather than folded in here.

⚠️ Remember the cadence: this gate is **not** CI-wired (above), so it catches a typo'd
`use` at the next manual run, not on the PR that introduced it. Copy the key, don't retype it.
Expand Down
16 changes: 13 additions & 3 deletions docs/qa/platform-checklist/areas/records-forms.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,7 +8,7 @@
"title": "Create → read → update → delete a record through the console UI",
"since": "v15",
"status": "active",
"revision": 6,
"revision": 7,
"priority": "P0",
"surface": "browser",
"personas": [
Expand All@@ -19,8 +19,12 @@
"requires": [
"showcase_account — writable standard object (sharingModel public_read_write), required name + status, format validations tax_id_format / billing_email_format (examples/app-showcase/src/data/objects/account.object.ts)"
],
"provisioning": {
"use": "search:qa-contributor-bound-member",
"why": "supplies the persona clause 7 (clone is RLS-gated) rests on and that no other clause here needs — a reader for whom a SOURCE row is invisible. showcase_account, the object the rest of this item drives, is public_read_write, so no persona can fail to read a row on it and the clause is unscoreable there; clause 7 is driven on showcase_invoice against INV-1003 (linus@example.com's) instead. Without the recipe the clause scores blocked(fixture) — which is what the 17.1.0 sweep recorded after reaching for the seeded demo personas and cloning showcase_account as Mei, who reads it fine (#10236)."
},
"knownGaps": [
"Clause 7 (clone is RLS-gated) cannot be scored on showcase_account, the object every other clause here drives: it is public_read_write, so there is no persona for whom a source row is invisible. It needs showcase_invoice plus a contributor-bound member — the recipe qa-contributor-bound-member in areas/search.json. ⚠️ The recipe mechanism is AREA-SCOPED (provisioning.use must name a key in this area's own fixtures block, README.md) and the validator now ENFORCES that scoping (#10593): a same-area `use` RESOLVES, and a `use` naming another area's recipe key FAILS check:platform-checklist as a dangling pointer, naming the item and the key that resolved to nothing. ⛔ So this reference cannot be spelled as `use` today — no cross-area spelling exists that the tooling accepts, and giving that pointer one is the open half of #10593 — and this item cross-references the recipe by name rather than opting in. Replay it from there; do not fork a second copy into this area."
"Clause 7 (clone is RLS-gated) cannot be scored on showcase_account, the object every other clause here drives: it is public_read_write, so there is no persona for whom a source row is invisible. It needs showcase_invoice plus a contributor-bound member — the recipe qa-contributor-bound-member, which areas/search.json owns. That reference now lives in fixtures.provisioning as the AREA-QUALIFIED `search:qa-contributor-bound-member`, and check:platform-checklist RESOLVES it (#10593 gap 2, maintainer ruling 2026-08-22 — option A): both spellings are valid, an own-area key and `<area>:<recipe>` for one another area owns. So this is a real opt-in rather than a prose pointer — rename or move the recipe and this item fails the gate by name, instead of leaving a sentence aimed at nothing. ⛔ Replay it from areas/search.json; do not fork a second copy into this area."
]
},
"steps": [
Expand DownExpand Up@@ -81,7 +85,7 @@
{
"clause": "clone is RLS-gated: the source is fetched in the caller's context (engine.findOne with context), so cloning a record the caller cannot SEE is refused with 404 RECORD_NOT_FOUND — never a silent duplicate of an invisible row",
"oracle": "api",
"verify": "⚠️ NOT on showcase_account — it is public_read_write, so no persona can fail to read a source row there and the clause is unscoreable on the object the rest of this item drives. Drive it on showcase_invoice with a contributor-bound member (recipe qa-contributor-bound-member in areas/search.json; the area-scoped recipe mechanism means this item cannot `use` it — replay it from there): as that member, POST the clone route against INV-1003 (linus@example.com's, invisible to them) → 404 RECORD_NOT_FOUND and no new row lands (recordNotFoundError from the null findOne). ⛔ Distinguish the two refusals before scoring: 404 RECORD_NOT_FOUND means the caller cannot SEE the source (this clause); 403 PERMISSION_DENIED means they can see it but hold no create grant — a DIFFERENT gate, and the one a persona with read-but-not-create hits. The 17.1.0 sweep hit exactly that confusion by cloning showcase_account as Mei, who reads it fine (#10236). Separately, an enable.clone:false object refuses with 403 CLONE_DISABLED",
"verify": "⚠️ NOT on showcase_account — it is public_read_write, so no persona can fail to read a source row there and the clause is unscoreable on the object the rest of this item drives. Drive it on showcase_invoice with a contributor-bound member (recipe qa-contributor-bound-member, owned by areas/search.json and opted into by this item as `search:qa-contributor-bound-member` — see fixtures.provisioning; replay it from there rather than forking a copy): as that member, POST the clone route against INV-1003 (linus@example.com's, invisible to them) → 404 RECORD_NOT_FOUND and no new row lands (recordNotFoundError from the null findOne). ⛔ Distinguish the two refusals before scoring: 404 RECORD_NOT_FOUND means the caller cannot SEE the source (this clause); 403 PERMISSION_DENIED means they can see it but hold no create grant — a DIFFERENT gate, and the one a persona with read-but-not-create hits. The 17.1.0 sweep hit exactly that confusion by cloning showcase_account as Mei, who reads it fine (#10236). Separately, an enable.clone:false object refuses with 403 CLONE_DISABLED",
"evidence": "the 404 RLS refusal with its code shown (not merely its status), the premise read proving the source row was invisible to that persona first, and a post-attempt count showing no new row"
}
],
Expand DownExpand Up@@ -135,6 +139,12 @@
"date": "2026-08-21",
"change": "corrected a knownGap that had gone from stale FACT to stale PERMISSION. Clause 7's cross-area note said `provisioning.use` was 'unresolved by the validator either way' — true when written (deferred at #7716 / tracked #7720), false since the area-scoped resolve landed at #10593: a same-area `use` now resolves and a cross-area one FAILS check:platform-checklist, naming the item. The prose therefore invited exactly the edit the gate rejects. The note now states what is enforced today; the cross-area SPELLING is still undecided (the open half of #10593) and is deliberately NOT predicted here, so this correction cannot itself become the next stale permission. Item substance, steps, personas, fixtures and all acceptance clauses are unchanged — no run verdict is invalidated by this revision",
"ref": "#10809"
},
{
"revision": 7,
"date": "2026-08-24",
"change": "clause 7's cross-area recipe pointer became a real reference. The persona it needs — qa-contributor-bound-member — is owned by areas/search.json, and until now no spelling existed that the validator accepted, so the pointer lived in knownGaps prose and clause 7's verify text: readable by a runner, invisible to check:platform-checklist, and free to rot if the recipe were ever renamed or moved. The maintainer ruled option A on 2026-08-22 (#10593 gap 2, area-qualified references), so this item now carries fixtures.provisioning.use = `search:qa-contributor-bound-member` and the gate resolves it. What a RUN does is unchanged in every particular: the same recipe, replayed from the same file, unblocking the same single clause on the same object and row — clause 7 was already instructed to drive showcase_invoice with this persona, and no step, clause, oracle, persona or acceptance text changed. The revision is bumped because the item's fixtures did, and because a run record pinning revision 6 was scored against an item whose recipe reference the tooling could not check",
"ref": "#10593"
}
]
},
Expand Down
10 changes: 8 additions & 2 deletions docs/qa/platform-checklist/areas/search.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -49,7 +49,7 @@
],
"teardown": "delete the sys_user_position row and the invoice created in step 5, or simply discard the isolated file DB — the cheaper path an isolated boot makes free. The signed-up principal is not otherwise cleaned up; use a unique per-run email so a leftover account never collides.",
"knownGaps": [
"⚠️ The recipe mechanism is AREA-SCOPED: `provisioning.use` must name a key in the item's OWN area fixtures block (README.md), and the validator now RESOLVES it (#10593): a same-area `use` resolves, and one naming another area's recipe key FAILS check:platform-checklist as a dangling pointer, naming the item and the key that resolved to nothing. records-forms.crud-roundtrip clause 7 needs this same persona and lives in another area file, so ⛔ it CANNOT `use` this recipe — no cross-area spelling exists that the tooling accepts today, and giving that pointer one is the open half of #10593 — and it cross-references this recipe by name in its knownGaps instead. Replay it from here; do not fork a second copy.",
"⚠️ This recipe has consumers in TWO areas, and the validator resolves both (#10593, maintainer ruling 2026-08-22 — option A, area-qualified references). `provisioning.use` accepts either spelling: a bare key naming a recipe of the item's own area (search.rls-both-personas uses `qa-contributor-bound-member`), or `<area>:<recipe>` naming one another area owns — records-forms.crud-roundtrip clause 7 needs this same persona and opts in from records-forms.json as `search:qa-contributor-bound-member`. Both FAIL check:platform-checklist loudly if this recipe is renamed, moved, or removed, naming the item and what resolved to nothing; the cross-area pointer used to be knownGaps prose the tooling could not see, which is the drift the qualified spelling closes. ⛔ Replay it from here; do not fork a second copy into another area — this block is the one home, and that is what the qualifier buys.",
"The persona is a subset reader of showcase_invoice ONLY. showcase_account is public_read_write and showcase_contributor grants plain allowRead on it with no RLS, so a contributor still reads every account row — an item needing an invisible SOURCE row must drive showcase_invoice, not showcase_account.",
"showcase_private_note is an EMPTY table for everyone and showcase_account reads identically (15 rows) for admin, Mei and Ada (#10236) — neither is usable as a both-sides RLS control. Recorded so the next runner does not reach for them."
]
Expand DownExpand Up@@ -208,7 +208,7 @@
"title": "Search honors RLS both ways: a restricted member gets no hits — and no total leakage — from rows they cannot see; the entitled persona finds the same rows",
"since": "v15",
"status": "active",
"revision": 5,
"revision": 6,
"priority": "P1",
"surface": "api",
"personas": ["seeded admin (sees all invoices)", "signed-up member bound to the contributor position via the qa-contributor-bound-member recipe (invoice_own_rows RLS: owner == current_user.email) — ⛔ NOT one of the seeded demo personas: Mei holds no position (PERMISSION_DENIED, object gate) and Ada holds auditor with viewAllRecords (all 12). See the recipe."],
Expand DownExpand Up@@ -286,6 +286,12 @@
"date": "2026-08-21",
"change": "pins the knownGaps correction made on the area recipe this item `use`s. qa-contributor-bound-member's first knownGap claimed the validator 'does not resolve it either way', which the area-scoped resolve at #10593 made false — a same-area `use` resolves, a cross-area one fails. Area-level recipes carry no `revision`/`history` of their own (the lifecycle check in scripts/check-platform-checklist.mjs is item-scoped, and none of the four recipes in this checklist has either field), so the revision of the item that `use`s the recipe is the ONLY pin a run record can hold that text against — which is why this item is bumped although none of its own fields changed. Steps, clauses, personas, fixtures and provisioning are all unchanged and no run verdict is invalidated",
"ref": "#10809"
},
{
"revision": 6,
"date": "2026-08-24",
"change": "pins the second knownGaps correction on the area recipe this item `use`s, for the same reason revision 5 pinned the first: area-level recipes carry no revision/history of their own, so the revision of a consuming item is the only thing a run record can hold that text against (#10885 tracks giving recipes their own, and is deliberately not pre-empted here). qa-contributor-bound-member's first knownGap said records-forms.crud-roundtrip ⛔ CANNOT `use` this recipe because no cross-area spelling existed — true when written, false since the maintainer ruled option A on 2026-08-22 (#10593 gap 2): `provisioning.use` now also accepts `<area>:<recipe>`, and that item opts in as `search:qa-contributor-bound-member`. The note now describes a recipe with consumers in two areas, both machine-resolved. THIS item's own reference is untouched and still the bare same-area `qa-contributor-bound-member` — every existing spelling stays valid, which is part of the ruling. Steps, clauses, personas, fixtures and provisioning are all unchanged and no run verdict is invalidated",
"ref": "#10593"
}
]
},
Expand Down
Loading
Loading