No plugin-reachable transactional write primitive — every migration-class tool rebuilds journal + compensation

[os-zhuang]

Found while scoping the D13 promotion tool (cloud#874). Not a blocker for any single tool — the point is that four migration-class consumers are now queued behind the same missing primitive, and each is being forced into the same hand-rolled shape.

Current state

• The data-engine contract declares an optional batch?(requests, { transaction?: boolean }) (packages/spec/src/contracts/data-engine.ts:70-91) — no engine implementation, no callers.
• driver-turso has real primitives (beginTransaction / commit / rollback, turso-driver.ts:764-776; remote-transport.ts:430-443) — not surfaced through ObjectQL.
• Plugin-visible multi-row mutation is therefore a JS loop of single-row writes (or multi: true predicate updates). A failure mid-sequence strands partial state; a process crash (as opposed to a thrown error) defeats in-process compensation entirely.

The queued consumers

Tool	Status	Tables touched
ADR-0105 D13 BU-subtree → org promotion (分立)	in design, cloud	sys_organization, sys_business_unit(_member), sys_member, sys_user_position, N business objects
ADR-0117 D8 ownership backfill	Proposed	every object gaining owning_business_unit_id
ADR-0110 org lifecycle hooks（新设/分立/合并）	Proposed	memberships, grants, master data
D10 master-data distribution (#4585)	blocked on its ADR	distribution/mirror rows per policy

Each independently converges on: dry-run preflight → undo journal → LIFO compensation → re-entrant forward recovery (precedent: cloud organization-lifecycle.ts slug rename). One copy of that machinery is an engineering answer; four copies is a platform debt signal.

Decision needed (one of three)

1. Surface driver transactions through the engine. The honest cost: middleware/hook semantics inside a transaction (do post-hooks fire before commit?), cache invalidation on rollback, cross-driver support — a real design project, not a plumbing task.
2. Bless a shared migration-journal primitive — a framework-owned journal + compensation/resume runner that migration tools consume, standardizing the pattern instead of transactionalizing the engine.
3. Explicitly rule both out and document journal+compensation as the blessed pattern — then at least extract the shared helper so the four consumers stop hand-rolling it.

No urgency ranking implied between 1 and 2; what needs deciding is which debt we are choosing. D13 proceeds on the hand-rolled pattern regardless and simplifies later if 1/2 lands.

Refs: #4346 (the multi: true no-predicate footgun — evidence this write path is sharp), ADR-0110, ADR-0117, cloud#874, cloud#1006.

[os-zhuang]

Claiming this issue.

• Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx
• Branch: claude/focused-mendel-5akrna — harness-fixed name, so it deviates from the claude/issue-<n>-<slug> convention. Recording it here explicitly so the claim is still discoverable by branch.

Scoping note — two premises in the issue need correcting

1. Option 1's hard part is already paid. ADR-0034 ("Robust multi-write transactions (ambient transaction context)") is Accepted — implemented (v8.0.0). ObjectQL.transaction() exists at packages/objectql/src/engine.ts:4934-4973 with an ambient AsyncLocalStorage handle lifted into every driver call at buildDriverOptions (engine.ts:1257-1275) — so hooks, validation predicates and internal reads already join the open transaction, which is exactly the "middleware/hook semantics inside a transaction" design project the issue prices in. Nested calls JOIN rather than nest (ADR-0067 D2).

What is actually missing is contract reachability, not the mechanism: IObjectQLEngine (packages/spec/src/contracts/objectql-engine.ts) never declared transaction, so a plugin typed to the slot cannot see it. The in-repo callers reach it through structural casts — packages/metadata-protocol/src/protocol.ts:2289 (the transactionalBatch discovery probe) and :7455-7458 (inTxn in publishPackageDrafts), plus a hand-declared local member in sys-metadata-repository.ts. That contract's own evidence bar (L29-37: a member joins when a cross-package consumer already calls it through the slot) is met three times over; the evidence just never got recorded.

2. driver-turso is not in this repo. turso-driver.ts:764-776 / remote-transport.ts:430-443 don't resolve here — the only Turso artifact is docs/design/driver-turso.md (Status: Proposal). The in-repo drivers already implement beginTransaction/commit/rollback as required members of IDataDriver (packages/spec/src/contracts/data-driver.ts:221-235): driver-sql sql-driver.ts:2214+, driver-memory memory-driver.ts:595+, driver-mongodb mongodb-driver.ts:545+.

The queued-consumer pain is real regardless. Engine transactions alone cannot serve the four tools: a million-row backfill cannot hold one SQLite write-lock, driver-memory's beginTransaction deep-clones the entire database, transaction() binds only the default driver (engine.ts:4950) so multi-datasource writes silently commit outside it, and no in-process rollback survives a process crash. So journal + compensation + resume is needed on top of transactions, not instead of them.

Decision (maintainer-confirmed) — a hybrid, recorded as ADR-0118

• D1 — declare transaction() on IObjectQLEngine; delete the casts. This is option 1 reduced to what it actually costs given ADR-0034.
• D2 — bless a framework-owned migration-journal runner (option 2): preflight dry-run → persistent journal in a sys_migration_journal platform object → chunked writes, each chunk inside engine.transaction() with its chunk_done marker written inside that transaction (so done ⇔ committed) → LIFO compensation → re-entrant forward recovery after a crash → at-least-once with attempt-keyed idempotency, per the discipline already documented in packages/core/src/utils/bulk-write.ts. ADR-0118 specifies it in implementable detail; the runner itself lands in a follow-up PR.
• D3 — retire IDataEngine.batch? (packages/spec/src/contracts/data-engine.ts:97-100). Declared with a three-word doc comment, implemented by no engine, called by nobody, and its DataEngineRequest union even nests batches recursively. Enforce-or-remove (ADR-0049); D1 + D4 cover its stated purpose.
• D4 — a bug found while scoping, fixed here: batchData's atomic flag opens no transaction. It only breaks the loop on first failure (packages/metadata-protocol/src/protocol.ts:5302-5305) — every op before the failure stays committed while the response is titled "atomic". Same class as [P1] filter folds to where in engine.find only — findOne/count/update/delete silently match EVERY row, and the hook docs teach the broken call #4346: a write-path guarantee that is declared but not enforced. It becomes real (one engine.transaction() around the batch, rollback-all, response reporting zero successes) or refused with 501 NOT_IMPLEMENTED on a runtime that cannot roll back — never silent best-effort. The Zod atomic default (packages/spec/src/api/batch.zod.ts:62) is also declared true while no enforcement site delivers it; it gets aligned to the enforced value rather than the reverse, so no existing caller's failure semantics change without opting in.

This PR: ADR-0118 + D1 + D4 + tests + changeset. Follow-ups to be filed: the D2 runner implementation, the D3 mechanical removal, engine.transaction caveat hardening (silent degrade / default-driver-only), and the sibling fake-atomic in deleteManyData/updateManyData.

D13 can proceed on the hand-rolled pattern and collapse onto the runner when D2 lands, as the issue anticipated.

---

Generated by Claude Code