The since:vN half of the 17.1.0 release sweep — the capabilities the v17 line introduced. Companions: #10224 (priority:P0) · #10225 (area:automation) · #10233 (area:api-backend) · #10245 (repo-wide pin sweep).
Environment
framework 19f98fa1fffbeb305bdcb6af64bc826fb25a46b1 · .objectui-sha = console dist stamp = 9a3daf8d37ad973a621e5edd276fe32467f90684 · showcase on 3456 against a file DB · 2026-08-20.
Execution mode: SEQUENTIAL, declared.
Scope
since:v17 → 17 runnable items (3 blocked hidden). 7 adjudicated, 10 not reached.
Per-item verdicts
| item | rev | verdict | evidence |
|---|
ai.skill-instructions-mcp-prompts | 1 | pass | pin mcp/src/skill-prompts.test.ts, 12 assertions |
identity-auth.invitation-scope-gates | 1 | pass | pin delegated-admin-invite.dogfood.test.ts, 4 assertions |
api-backend.declarative-endpoint-execution | 1 | pass | pin showcase-declarative-endpoints.dogfood.test.ts, 17 assertions |
cli.qa-suite-execution | 4 | partial 7/8 | below |
api-backend.filter-comparand-conformance | 1 | partial 4/5 | detail in #10233 |
api-backend.date-range-preset-matrix | 1 | partial 2/4 | detail in #10233 |
integration-system.datasource-credential-refusal-matrix | 1 | partial | detail in #10224 |
No fail recorded.
cli.qa-suite-execution (rev 4) — os test, the Quality Protocol runner
Driven against the live showcase with a minted admin token.
| clause | verdict | evidence |
|---|
| [0] green path closes | pass | both scenarios ✅ with id + duration; SUCCESS: All 2 scenarios passed.; exit 0 |
[1] state threads (capture → {{var}}) | pass | the suite carries 5 capture sites and 4 interpolation points, and category-create-read-roundtrip passes end to end |
| [2] a false assertion fails the run | pass | ❌ Scenario: false-assert, Assertion failed: data.status expected degraded, got ok, failing step named, exit 1 |
| [3] #7256 — an unevaluable assertion FAILS | pass | contains against an absent path: "cannot be evaluated by 'contains' — expected an array or a string at that path, got undefined. The path resolved to nothing — the field is absent from the result, or the path is misspelled. Use 'is_null' if asserting absence is what you meant." — exit 1, and the message prescribes the correct operator |
| [4] #6247 — load-site enforcement | pass | a suite with steps misspelled stpes is refused before execution, naming the file, the offending path (scenarios.0.steps), the expected shape, and content/docs/references/qa/testing.mdx; exit 1 |
| [5] teardown runs and leaves nothing | pass | showcase_category 4 rows before → 4 after; a second identical run is green again (repeatable, no leftover) |
[6] all 8 TestActionTypeSchema members get their own verdict | not-run | the variant sweep was not staged |
| [7] #7363 glob discipline | pass | qa/**/*.test.json completes in 3s and finds the same suite; the zero-match glob prints "Exiting 0 — an empty match is not a failure. Pass --fail-on-empty to make it one." with exit 0 |
Worth recording beyond the verdict: clause [3] is the runner refusing to let an assertion that cannot be computed pass silently. That is the same failure mode that produced several false readings inside this very sweep when driven by hand — the shipped runner is stricter about it than an ad-hoc script is, which is an argument for expressing more checklist clauses as Quality Protocol suites where the shape allows.
Not reached — handoff
access-security.capability-declaration-lifecycle (rev 1, mixed) · api-backend.api-console-discovery-execute (rev 1, browser) · api-backend.retired-def-refusal (rev 1, build) · approvals.account-app-entry (rev 1, P0, browser) · approvals.setup-nav-entry (rev 1, browser) · approvals.pending-count-surfaces (rev 1, browser) · cli.scaffold-console-first-paint (rev 1, mixed) · i18n.strict-translation-key-rejection (rev 2, mixed) · identity-auth.org-membership-team-management (rev 1, mixed) · identity-auth.identity-import-wizard (rev 1, mixed).
Hidden as blocked by the selector: access-security.no-active-org-session-semantics · identity-auth.oauth-app-consent-loop · records-forms.import-job-undo-cancel.
Parked-on: #10236 · cli.qa-suite-execution · clause 6 — TestActionTypeSchema variant sweep not staged
Carried-forward: #10233 · api-backend.filter-comparand-conformance · partial 4/5
Carried-forward: #10233 · api-backend.date-range-preset-matrix · partial 2/4
Carried-forward: #10224 · integration-system.datasource-credential-refusal-matrix · partial
The
since:vNhalf of the 17.1.0 release sweep — the capabilities the v17 line introduced. Companions: #10224 (priority:P0) · #10225 (area:automation) · #10233 (area:api-backend) · #10245 (repo-wide pin sweep).Environment
framework
19f98fa1fffbeb305bdcb6af64bc826fb25a46b1·.objectui-sha= console dist stamp =9a3daf8d37ad973a621e5edd276fe32467f90684· showcase on 3456 against a file DB · 2026-08-20.Execution mode: SEQUENTIAL, declared.
Scope
since:v17→ 17 runnable items (3 blocked hidden). 7 adjudicated, 10 not reached.Per-item verdicts
ai.skill-instructions-mcp-promptsmcp/src/skill-prompts.test.ts, 12 assertionsidentity-auth.invitation-scope-gatesdelegated-admin-invite.dogfood.test.ts, 4 assertionsapi-backend.declarative-endpoint-executionshowcase-declarative-endpoints.dogfood.test.ts, 17 assertionscli.qa-suite-executionapi-backend.filter-comparand-conformanceapi-backend.date-range-preset-matrixintegration-system.datasource-credential-refusal-matrixNo
failrecorded.cli.qa-suite-execution(rev 4) —os test, the Quality Protocol runnerDriven against the live showcase with a minted admin token.
✅with id + duration;SUCCESS: All 2 scenarios passed.; exit 0{{var}})capturesites and 4 interpolation points, andcategory-create-read-roundtrippasses end to end❌ Scenario: false-assert,Assertion failed: data.status expected degraded, got ok, failing step named, exit 1containsagainst an absent path: "cannot be evaluated by 'contains' — expected an array or a string at that path, got undefined. The path resolved to nothing — the field is absent from the result, or the path is misspelled. Use 'is_null' if asserting absence is what you meant." — exit 1, and the message prescribes the correct operatorstepsmisspelledstpesis refused before execution, naming the file, the offending path (scenarios.0.steps), the expected shape, andcontent/docs/references/qa/testing.mdx; exit 1showcase_category4 rows before → 4 after; a second identical run is green again (repeatable, no leftover)TestActionTypeSchemamembers get their own verdictqa/**/*.test.jsoncompletes in 3s and finds the same suite; the zero-match glob prints "Exiting 0 — an empty match is not a failure. Pass --fail-on-empty to make it one." with exit 0Worth recording beyond the verdict: clause [3] is the runner refusing to let an assertion that cannot be computed pass silently. That is the same failure mode that produced several false readings inside this very sweep when driven by hand — the shipped runner is stricter about it than an ad-hoc script is, which is an argument for expressing more checklist clauses as Quality Protocol suites where the shape allows.
Not reached — handoff
access-security.capability-declaration-lifecycle(rev 1, mixed) ·api-backend.api-console-discovery-execute(rev 1, browser) ·api-backend.retired-def-refusal(rev 1, build) ·approvals.account-app-entry(rev 1, P0, browser) ·approvals.setup-nav-entry(rev 1, browser) ·approvals.pending-count-surfaces(rev 1, browser) ·cli.scaffold-console-first-paint(rev 1, mixed) ·i18n.strict-translation-key-rejection(rev 2, mixed) ·identity-auth.org-membership-team-management(rev 1, mixed) ·identity-auth.identity-import-wizard(rev 1, mixed).Hidden as blocked by the selector:
access-security.no-active-org-session-semantics·identity-auth.oauth-app-consent-loop·records-forms.import-job-undo-cancel.