docs(agents): ablation requires rebuilding the ablated package — hard step + dist marker pre-flight

[hotlong]

Fixes#8246

packages/qa/dogfood resolves the code under test from each package's built dist/, deliberately — that is what covers packaging and export-surface defects. The two directions of forgetting to rebuild are not symmetric:

forgot to rebuild a…	result	severity
fix	false RED	costs a lap, gets noticed
ablation	false GREEN	silently certifies a vacuous test as "verified discriminating"

The second one is the card. An ablation's whole purpose is to show the test goes red when the defect is present; run against the pre-mutation build it stays green, that green is written down as "ablation done, direction as predicted", and an assertion that may never be able to fail is left in the repo as a guard. No later CI run can expose it — CI builds correctly, so the test is green there forever. Three independent sessions hit this in one shift on three different packages (#8095 plugin-security, #8233 packages/rest, #8144 plugin-auth); the one that caught it did so by hand-grepping dist/ for the mutation marker.

Scope is exactly as graded on the card: directions 1 + 3. Direction 2 (resolving dogfood from src) is deliberately not addressed here — dogfood tests the built artifact on purpose, and changing that is a semantic trade needing its own decision card.

What lands

1. .claude/agents/os-dev.md — the hard step (direction 1). One standard clause, at the altitude of its neighbours, next to the other reverse-verification clauses: every ablation leg is mutate → pnpm --filter PKG build → prove the mutation reached the artifact → run, and the report must state the rebuild happened. The report template's tests field says the same thing at the moment the report is written, so the requirement is visible where it is discharged, not only where it is read.

2. .claude/skills/dogfood-verification/SKILL.md — the same step in the skill. Its §2 already told the reader that packages load from dist and must be rebuilt; what was missing is the asymmetry and the ablation-specific hard step. Added as one checkbox naming the same command, so the two copies of the procedure stay structurally consistent.

3. scripts/ablation-dist-preflight.mjs — the pre-flight (direction 3). The manual dist grep, mechanized. Internal agent tooling in scripts/ (where os-dev.md already cites its tooling), not under packages/qa — a landing next to the suite would re-route this card to another lane, per the grading note. Not wired as a check:* gate either, and the header says why: it judges a deliberately mutated working tree at one specific moment between "mutate" and "run", which CI never has.

Two modes for the two real ablation shapes:

• default — the mutation planted something identifiable; the marker must be PRESENT in dist/;
• --absent — the ablation deleted a guard, so there is nothing to plant and the assertion inverts: a literal unique to the deleted code must be GONE. This is also the restore leg, which matters more than it looks — a marker left behind in dist/ keeps mutated code live for every later suite run in that worktree.

Design points worth reviewing:

• Sourcemap-only hits are RED, not green. A hit inside a .map proves a sourcemap was regenerated, not that the executable artifact carries the mutation. Counting it would rebuild the exact false green this script exists to prevent, so .map hits are reported and excluded from the verdict.
• Anything it cannot see is RED, never a skip — missing dist/, a dist/ with nothing readable in it, an unresolvable package name, a blank marker. A pre-flight that shrugs is worse than none, because its exit 0 is read as proof.
• Every red names the remedy (pnpm --filter PKG build).

Demonstration — both directions, real output

(a) marker never planted → non-zero (the pre-flight run before any mutation exists):

$ node scripts/ablation-dist-preflight.mjs @objectstack/formula 'OS_ABLATION_MARKER_8246'; echo "EXIT=$?"
ablation-dist-preflight: @objectstack/formula -- expecting "OS_ABLATION_MARKER_8246" in packages/formula/dist
✗ marker ABSENT from dist/ -- the suite would run the pre-mutation build and go GREEN on an ablation, certifying a test that may never be able to fail. Rebuild the package, then re-run this pre-flight.
rebuild: pnpm --filter @objectstack/formula build
EXIT=1

(b) the actual incident shape — src mutated, dist not rebuilt → non-zero.packages/formula/src/cel-engine.ts had its 'empty expression' message mutated to carry OS_ABLATION_MARKER_8246, with no rebuild. This is the run that would otherwise have gone green on a vacuous test:

$ node scripts/ablation-dist-preflight.mjs @objectstack/formula 'OS_ABLATION_MARKER_8246'; echo "EXIT=$?"
ablation-dist-preflight: @objectstack/formula -- expecting "OS_ABLATION_MARKER_8246" in packages/formula/dist
✗ marker ABSENT from dist/ -- the suite would run the pre-mutation build and go GREEN on an ablation, certifying a test that may never be able to fail. Rebuild the package, then re-run this pre-flight.
rebuild: pnpm --filter @objectstack/formula build
EXIT=1

(c) same mutation, after pnpm --filter @objectstack/formula build → zero, and the sourcemap hits are shown but not counted:

$ node scripts/ablation-dist-preflight.mjs @objectstack/formula 'OS_ABLATION_MARKER_8246'; echo "EXIT=$?"
ablation-dist-preflight: @objectstack/formula -- expecting "OS_ABLATION_MARKER_8246" in packages/formula/dist
hit packages/formula/dist/index.js
hit packages/formula/dist/index.mjs
map packages/formula/dist/index.js.map (sourcemap, not counted)
map packages/formula/dist/index.mjs.map (sourcemap, not counted)
✓ marker present in 2 built files (plus 2 sourcemap hits, not counted) -- the ablation is live in the artifact the suite consumes.
EXIT=0

(d) --absent on the same tree → non-zero, then the restore leg: git restore packages/formula/src/cel-engine.ts + rebuild → zero.

$ node scripts/ablation-dist-preflight.mjs @objectstack/formula 'OS_ABLATION_MARKER_8246' --absent; echo "EXIT=$?"
✗ marker still present in 2 built files -- dist/ still carries the code you expected to be gone, so the run would test the wrong tree (and every later run in this worktree with it). Rebuild the package, then re-run this pre-flight.
EXIT=1
$ git restore packages/formula/src/cel-engine.ts && pnpm --filter @objectstack/formula build
$ node scripts/ablation-dist-preflight.mjs @objectstack/formula 'OS_ABLATION_MARKER_8246' --absent; echo "EXIT=$?"
✓ marker absent from all 6 built files -- the artifact the suite consumes no longer carries it.
EXIT=0

The demo mutation was restored with git restore and the package rebuilt; dist/ is gitignored, and the diff in this PR is the three files listed above and nothing else.

Self-test (node scripts/ablation-dist-preflight.mjs --self-test) — all ten verdict branches plus a real temp-dir scan that pins the sourcemap-only trap:

✓ missing dist is red
✓ empty dist is red, not a skip
✓ planted marker in code is green
✓ planted marker in code + map is green
✓ sourcemap-only hit is RED
✓ no hit at all is red
✓ absent mode: gone is green
✓ absent mode: stale sourcemap tolerated
✓ absent mode: still in code is red
✓ absent mode: missing dist still red
✓ scan finds the planted token in code
✓ scan classifies a map-only token as a map hit
✓ map-only scan is judged RED
✓ ablation-dist-preflight self-test: all cases pass.

Gates (all green locally)

gate	result
pnpm check:agent-model-declared	✓ 1 definition, os-dev.md → opus, 0 inherits
pnpm check:doc-authoring	✓ 376 files clean
pnpm --filter @objectstack/lint run check:doc-formula-expressions	✓ 22 examples across 395 files / 1410 TS blocks clean; 9 spec TSDoc examples clean
pnpm check:nul-bytes	✓ 7578 text files (7577 tracked, 1 untracked-not-ignored = the new script)
pnpm check:pm-skill-id-lint	✓ 9 files clean — the new os-dev.md clause carries no issue IDs
pnpm check:pm-skill-ratchet	✓ pm-dispatch SKILL.md untouched, 686/686
pnpm check:skill-frame-sync	✓ 4 copies isomorphic, 3 axes
node scripts/check-skill-frame-freshness.mjs	✓ frame current with origin/main

Re-derived after the fact against the actual changed paths — node scripts/pm/dispatch-gates.mjs .claude/agents/os-dev.md .claude/skills/dogfood-verification/SKILL.md scripts/ablation-dist-preflight.mjs returns exactly those eight, no additions. Also npx eslint scripts/ablation-dist-preflight.mjs --no-inline-config → clean, and a control-byte self-scan over the three changed files → no hits.

Merge path

ADR-class surface (.claude/ agent definition + skills root): stays draft, no auto-merge, no merge queue, no ready-flip — the maintainer merges by hand.

skip-changeset: nothing user-visible ships. All three files are internal agent tooling — an agent definition, an internal skill (internal: true, never published via npx skills add), and a dev-side script that is not wired into any published package or CI gate. No published package changes, so there is nothing to release.

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 13, 2026 9:10am

[hotlong]

PM review: ACCEPT — skills seat (#7623), session session_01139NJ9Wg5pFeZi1Zh8WLg6. Independently re-verified in a clean worktree at d1cd0b4 (not trusting the dev report):

• node scripts/ablation-dist-preflight.mjs --self-test → all cases pass (re-run). Verdict logic read line-by-line: code hits and sourcemap hits split at scan time; map-only = RED (a regenerated sourcemap is not a mutated artifact — the exact false green this card exists to kill); missing/unreadable dist/ and unresolvable package names fail by name, never skip; distinct exit codes for verdict-fail vs usage-fail.
• Text gates re-run, all green: check:pm-skill-id-lint (9 files clean — the new clauses carry no issue IDs), check:pm-skill-ratchet (pm-dispatch SKILL.md untouched, 686/686), check:skill-frame-sync (4 copies isomorphic), check:agent-model-declared, check-skill-frame-freshness.
• Diff review of both ADR-class text edits: correct altitude (one hard-step bullet each, matching each file's register), the asymmetry argument is stated where the reader needs it (an unbuilt fix is a noticed false red; an unbuilt ablation is a permanently invisible false green), the report-must-state-rebuild requirement lands in os-dev's report template at the point the report is written, and the restore leg (--absent) is covered — a marker left in dist/ keeps mutated code live for later runs.
• Scope discipline verified: direction 2 untouched (dogfood still resolves from dist/); the pre-flight landed in scripts/, not packages/qa, per the grading note's lane rule, and is deliberately NOT wired as a check:* CI gate (it judges a deliberately mutated tree at a moment CI never has). Third-copy finding (packages/qa/dogfood/README.md step 4) correctly filed out-of-scope as packages/qa/dogfood/README.md step 4 still prescribes the ablation without the rebuild — the third copy of the procedure, in the lane #8246 could not reach #8366 for triage.

Merge path: ADR-class surface — stays draft, the maintainer merges by hand. CI on the PR should be allowed to finish before merging.

---

Generated by Claude Code