Skip to content

finding: five checklist-text corrections for the approvals area — item text, oracles and fixtures that do not match the build (from the #7517 run) #7530

Description

@huangyiirene

Symptom

Five authoring defects in docs/qa/platform-checklist/ surfaced by the full approvals area run. None of these is a product defect — in every case the build behaves correctly (or correctly refuses) and the checklist is what is wrong: an item asserts an outcome the platform deliberately does not produce, prescribes an oracle that is not readable, or requires a fixture that cannot be constructed. Consolidated into one card because they share a single fix face (the approvals item files) and should land as one authoring pass.

Two of them are the sole reason their items were scored PARTIAL rather than PASS, so leaving them unfixed keeps re-manufacturing false amber on every future run.

1. decision-action-matrix — the resubmit clause asserts the wrong state transition

The item's verify text says resubmit moves the original request returned → pending. In fact a new request is minted per approval-node entry while the original stays returned. This was source-confirmed as by design, so the item text is what must change, not the engine.

Revise the item text to assert the mint-a-new-request shape (and, if the original's terminal returned state matters, assert that too as an explicit clause).

2. dynamic-approver-routing — the prescribed oracle is not readable while the run is paused

The clause asks that the routing outputs be observed landing "in run variables". The run-detail read carries no variables key at all while the run is paused — structural, not a data gap: the engine's suspend path calls recordLog without variables, and there is no side door to read them. The clause was therefore satisfied functionally (stage-2 pending_approvers equals precisely the picked set — 1 pick → 1, 2 picks → 2, same order) rather than read directly.

Reword the clause to name the functional oracle it can actually use — or, if reading the snapshot is genuinely wanted, file the product-side ask that run-detail carry the suspended-run variables snapshot, and gate the clause on it.

3. ooo-delegation-reroute — the prescribed persona shape is not constructible

The item prescribes setting up the delegation as a third party on behalf of the named delegator. That cannot be done on this build: sys_approval_delegation is self-service-only (writes are scoped to the caller's own delegator row — correctly, and proven by the security sweep in the same run), and the named delegator has no auth account, so nobody can create the row as themselves either.

The run instead drove the mirror image — a second real runtime account as the delegate — which is the stronger test. ⇒ Rewrite the fixture and persona shape around that, rather than the unconstructible one.

4. Provisioning a seeded demo persona as a real login needs more than a password hash

Worth writing into the approvals items' knownGaps so future runs drive real second personas instead of recording blocked(fixture):

better-auth 1.7.0-rc.2 requires the credential account's issuer to equal local:credential. Without it, sign-in fails with INVALID_EMAIL_OR_PASSWORD behind a misleading "User not found" warn — the user exists and the hash is right, so the warn sends you looking in exactly the wrong place. Three items in this area were previously at risk of being scored blocked(fixture) for what is a one-field provisioning detail.

5. notification-deep-link — its fixtures.requires is false on this build, and its clause-1 evidence line is unmeetable

Two separate problems in one item:

  • fixtures.requires is false on this build. It rests on opening a request notifying somebody. Opening a request notifies nobody — there is no approval.opened topic. ⇒ Name the remind / reassign path instead, which does produce a real deep-linked notification (the run used exactly that and the item otherwise passed 4/4).
  • Clause 1's evidence line — "two screenshots with the URL visible" — cannot be met literally, because the console consumes the query param: by the time the drawer is open and screenshottable, the URL no longer shows it. ⇒ Restate the evidence requirement in terms of something observable (the drawer resolving to the right request, field-matched against the API — which is what was actually captured).

Root cause

Checklist authoring drift, not product behaviour. Items 1–3 and 5 were written against an assumed shape of the approvals subsystem that the implementation does not have (and in cases 1 and 3 deliberately does not have); item 4 is undocumented environment knowledge that each run currently has to rediscover. The fix face is entirely docs/qa/platform-checklist/ — the approvals area item files and their knownGaps / fixtures.requires blocks.

Reproduction

Full approvals area run of the checklist-test skill against a live showcase, framework a86db175, console 09987b68 — 8 runnable items, isolated boot per batch. Items 1 and 2 above are the recorded PARTIAL verdicts; items 3–5 are the run's "fixture corrections for the next sweep". Each is evidenced in the run record at #7517.

Source

Extracted from the QA run #7517 (framework a86db17, console 09987b68).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions