Skip to content

fix(qa-checklist): resolve provisioning.use against its own area's recipes (#10593 gap 1) - #10810

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-10593-provisioning-use-resolve
Aug 21, 2026
Merged

fix(qa-checklist): resolve provisioning.use against its own area's recipes (#10593 gap 1)#10810
os-zhuang merged 2 commits into
mainfrom
claude/issue-10593-provisioning-use-resolve

Conversation

@claude

@claudeclaudeBot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Part of #10593gap 1 only. ⚠️ Deliberately NOT a closing keyword: gap 2 (cross-area recipe reuse) is still open on that card, and Fixes would close it on merge regardless of this sentence. Gap 2 (cross-area recipe reuse) is untouched and is left open on purpose; the reasoning and a proposed shape are at the bottom.

The defect

docs/qa/platform-checklist/README.md recorded option C from #7716 as deliberately deferred (tracked at #7720, which landed only the documentation half):

The validator does not yet resolve provisioning.use against the area's fixtures keys — that was deliberately deferred … to be revisited if the recipe shape spreads to more areas.

That condition is met: recipes live in three area files and six items carry a provisioning.use.

Reproduced on origin/main at 112a8c6731, before touching the validator. Two shapes, injected one at a time into an otherwise clean tree:

injectedvalidator said
"use": "qa-media-constraint" (a typo of qa-media-constraints, attachments-storage.json)exit 0OK — 15 areas, 205 items (205 active); …, byte-identical to the untouched run
"use": "qa-contributor-bound-member" on records-forms.crud-roundtrip (a real recipe key, owned by another area)exit 0 — same OK line

Nothing reported, either time. An item that reads as provisioned and is not costs the run the clauses the recipe was meant to unblock, and it discovers that mid-run, on a live boot.

What this PR does

scripts/check-platform-checklist.mjs now resolves every fixtures.provisioning.use against the recipe keys of its own area's area-level fixtures block. It mirrors the supersededBy check that already lives in the file — same err(file, item.id, …) reporting, same "points at unknown" wording — and reuses the trap checker's did-you-mean, because a typo of a real recipe is the drift shape review is worst at. $-prefixed keys ($comment) are annotations, not recipes.

Same input, after:

check-platform-checklist: 1 problem(s)
✗ attachments-storage.json · attachments-storage.field-accept-maxsize-server-enforced: "fixtures.provisioning.use"
names "qa-media-constraint", which is not a recipe in this area's area-level "fixtures" block — did you mean
`qa-media-constraints`? — this area offers `qa-scratch-authz`, `qa-media-constraints`. References are
AREA-SCOPED: a recipe another area owns cannot be opted into by key (cross-area reuse has no spelling yet,
#10593) — cite that recipe by name in this item's "knownGaps" instead, and do not fork a second copy.

The clean tree stays green, and the OK line now states what the new check ran on, so a green says something:

check-platform-checklist: OK — 15 areas, 205 items (205 active); coverage: 30 kinds mapped, 0 waived;
traps: 19 documented, 19 in use; provisioning: 4 area recipes, 6 item references resolved
(self-checks: 22 trap-vocabulary + 14 provisioning-resolve assertions).

The README paragraph that documented the deferral is rewritten to document the check, both of its deliberate limits, and the cadence caveat below.

⚠️ This gate is not CI-wired — read the green accordingly

.github/workflows/lint.yml states it plainly, and this PR honours it rather than changing it:

NOTE: the standing platform test checklist (docs/qa/platform-checklist/) is validated by pnpm check:platform-checklist, but by MAINTAINER DECISION it is NOT wired into CI — it runs on a periodic manual cadence … Run it by hand.

So adding this check does not make it run on pull requests. A typo'd use is now caught at the next manual sweep — before a release, after a large platform surface lands — not on the PR that introduces it. That is a real narrowing of the win versus what "a green check" normally implies, and it is why the check carries its own inline positive-control battery: on a gate nobody runs per-PR, a check that quietly stopped firing would restore exactly the green it replaced, and no CI run would ever contradict it. CI-wiring it would reverse a recorded maintainer decision and is not on this card.

Evidence

Everything below on 20d61f7e55 (the tip of this branch).

Census. Six provisioning.use values across all 15 area files, every one resolving on the base — matching the count on the card:

area filerecipes defineditems referencing
attachments-storage.jsonqa-scratch-authz, qa-media-constraints4
automation.jsonqa-flow-status-doors1
search.jsonqa-contributor-bound-member1

The gate's own OK line reports the same numbers independently: 4 area recipes, 6 item references resolved.

records-forms.crud-roundtrip clause 7 stays green, by running. The item has no provisioning block at all (#10236 cross-referenced the recipe by name in knownGaps instead), so there is nothing for the resolve to dangle on: node scripts/check-platform-checklist.mjs exits 0 with all 205 items, the item included, and node scripts/checklist-select.mjs records-forms.crud-roundtrip still selects it. The green is not vacuous — the second repro row above is that exact item, and it does red when a cross-area use is added to it, which is what proves the check visits it.

Ablation. Mutated provisioningProblems() to return [] immediately, confirmed on disk by anchor count (old anchor 1 to 0, marker 0 to 1, grep -c = 1 at line 269), not by an editor exit code. There is no rebuild leg here: the gate runs node scripts/check-platform-checklist.mjs against the source file itself, so no dist/ can hold a stale copy.

  • Predicted: 10 of the 14 provisioning assertions red — U4-U13, every case that asserts the check FIRES; U1, U2, U3 (assert zero problems) and U14 (exercises areaRecipeKeys only) stay green. And the gate refusing on the real tree rather than going quietly green.
  • Observed: exactly that. --self-test exit 1, 10 failure(s), the named set U4-U13. The full gate exit 1 with the provisioning-resolve check's own positive control FAILED — a "use" that resolves to nothing would pass, which is the exact defect this check was added to close.
  • Restored with git checkout --: marker absent (grep -c = 0), sha256 identical to the pre-mutation file (39d95f98…6e746e), tree clean, gate green again.

Gates. Derived with node scripts/pm/dispatch-gates.mjs (no paths passed — it takes the change set from the merge base itself), re-run on the final commit. Every exit code captured with cmd > file 2>&1; ec=$?, never through a pipe:

gateexit
pnpm check:cross-package-test-inputs (and node scripts/check-cross-package-test-inputs.mjs, --self-test)0
pnpm check:doc-authoring (+ --self-test)0
pnpm --filter @objectstack/lint run check:doc-formula-expressions (+ --self-test)0
pnpm check:entry-guard (+ --self-test) — by hand, dispatch-gates cannot reach scripts/** gates0
pnpm check:parse-guard (+ --self-test) — likewise; dispatch-gates prints it as unreachable by construction0
pnpm check:nul-bytes (+ --self-test)0
pnpm check:platform-checklist (the alias this PR changes, both halves)0
eslint scripts/check-platform-checklist.mjs0

What is deliberately left open — gap 2

Cross-area reuse still has no spelling. records-forms.crud-roundtrip clause 7 needs the persona qa-contributor-bound-member provisions; that recipe lives in areas/search.json; use names a key in the item's own area — so the item cannot reference it, and #10236 handled it by naming the recipe in knownGaps prose with an explicit instruction not to fork a copy.

This PR does not decide that, and is built so it cannot decide it by accident. Two things the resolve deliberately does not do:

  1. It does not reach across areas. A use pointing at another area's recipe key fails, and the message says why rather than pretending the reference works.
  2. It does not flag a recipe no item references — the direction the trap-vocabulary check does enforce. With cross-area reuse unspelled, a recipe whose only consumer lives in another area is cited from prose the tooling cannot see. Redding the unreferenced direction would answer gap 2 by accident, in the direction of "recipes are area-local".

Open question for triage — the three candidate answers, and what this PR learned about them. The card lists a qualified use (search:qa-contributor-bound-member), a shared areas/_fixtures.json, or ruling that recipes stay area-local with a pinned back-reference.

Doing gap 1 sharpened the choice in one respect worth recording: the resolve's universe is now a per-area key set, computed per file. A qualified use needs only that universe widened to a two-level lookup and would keep every existing spelling valid — the smallest diff of the three, and the only one that makes the existing cross-area pointer machine-checkable where it already sits. A shared _fixtures.json moves recipes away from the area that proved them, which is where the provenance field points and where a runner looks. The area-local ruling is the cheapest to write and the only one that leaves the prose pointer permanently unverifiable — the drift the card was filed about. My recommendation is the qualified use, but it is a convention decision and belongs to the maintainer, so it is stated here rather than implemented. Two references would use it today.

Also not in this PR

Filed as #10809: the same two cross-area pointers carry knownGaps prose asserting the validator does not resolve use "either way" — true when written, inverted by this change, and now reading as permission to write the one spelling the gate rejects. Correcting it is a semantic edit to two items, so it needs a revision bump and a history entry each, which is not a rider on a gate PR; it also wants to be written once, after gap 2 lands.

No changeset: this changes gate tooling and a QA ledger README, and publishes nothing.


Generated by Claude Code


Generated by Claude Code

… recipes
`docs/qa/platform-checklist/README.md` recorded option C from #7716 as
deliberately deferred (tracked at #7720, which landed only the documentation
half), "to be revisited if the recipe shape spreads to more areas". It has:
recipes now live in three area files and six items carry a
`provisioning.use`. Until now a `use` naming a key its area does not define
validated clean — measured on 112a8c6, a typo'd `qa-media-constraint` and a
cross-area `qa-contributor-bound-member` each exited 0 with the untouched OK
line. An item that reads as provisioned and is not costs the run the clauses
the recipe was meant to unblock, mid-run and on a live boot.
The resolve mirrors the existing `supersededBy` check — same `err(file, id, …)`
reporting, same "points at unknown" wording — and reuses the trap checker's
did-you-mean, since a typo of a real recipe is the drift shape review is worst
at. It carries its own inline positive-control battery (14 assertions) for the
reason the trap battery does: this gate is not CI-wired, so a check that
quietly stopped firing would restore exactly the green it replaced.
Scope is gap 1 of #10593 only. Cross-area reuse still has no spelling, so the
resolve is same-area only and the unreferenced-recipe direction is deliberately
NOT checked — redding it would settle that convention question by accident.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 21, 2026
@github-actionsgithub-actionsBot added the documentation Improvements or additions to documentation label Aug 21, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review August 21, 2026 13:18
@os-zhuang
os-zhuang added this pull request to the merge queueAug 21, 2026
Merged via the queue into main with commit 9b9a16bAug 21, 2026
29 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-10593-provisioning-use-resolve branch August 21, 2026 13:45
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@os-zhuang@claude