Uh oh!
There was an error while loading. Please reload this page.
fix(driver-sql): apply bulkUpdate as one transaction so a mid-batch refusal rolls back the rows already applied - #14170
Conversation
SqlDriver.bulkUpdate is a sequential loop of individual update() calls with no transaction around it, so each update() autocommitted its own row. A batch refused partway through left every row processed before the refusal committed permanently — the caller got an exception and the database held a state nobody declared. The loop now runs inside a transaction. With no caller transaction the driver opens its own; inside one it opens a knex nested transaction (a SAVEPOINT) on the caller's, so the batch is undone as a unit while the caller's own work and its transaction both survive. driver-turso inherits the repair through super.bulkUpdate; no subclass change. bulkDelete is untouched — one whereIn(...).delete() per shard, already atomic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q5WBDtaUnoz5XuJ6jk8pQ5
📓 Docs Drift CheckThis PR changes 1 package(s): 7 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin b78b5c5dc0bf007a10019e9acc9fc23ee8c3cb43 && git checkout b78b5c5dc0bf007a10019e9acc9fc23ee8c3cb43
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b003cf2e8b0bb59a7e262c42bbf728a82558b714 8345bb5a762923def1dea740194efb155bdf0fae && git checkout -B drift-repro b003cf2e8b0bb59a7e262c42bbf728a82558b714 && git merge --no-ff 8345bb5a762923def1dea740194efb155bdf0fae
node scripts/docs-audit/affected-docs.mjs --json b003cf2e8b0bb59a7e262c42bbf728a82558b714
|
Uh oh!
There was an error while loading. Please reload this page.
Fixes#13854
SqlDriver.bulkUpdateis a sequentialfor-awaitover individualupdate()calls with no transaction around it, so eachupdate()autocommitted its own row. A batch refused partway through left every row processed before the refusal permanently committed: the caller received an exception while the database held a state nobody declared — neither the pre-image nor the post-image — and a retry of the same array was not safe.The loop now runs inside a transaction. The loop itself is unchanged and deliberately so: each id carries its own patch, so no single statement expresses N different SET lists (which is why
bulkCreate's "send the batch as one INSERT" shape does not transfer). What changed is the boundary around it.Same defect class #13340 / #13435 closed on
driver-memory's batch doors, here on the production default SQL driver.Mechanism — the file's own, no new one and no new dependency
SqlDriveralready carries transaction support, and this reuses it rather than inventing a second mechanism:The
(caller's transaction) ?? this.knexrunner idiom is the onecollidingAutoNumberReservationsalready uses;getBuilderroutes every statement ontooptions.transactionvia.transacting(trx), so the whole loop — including the rotation-shard path throughrotatedUpdateById, which threadsoptionsthe same way — rides the one boundary.The choice of runner is also the choice between a transaction and a savepoint, which is what makes the caller-transaction case correct:
this.knex.transaction()opens the driver's own, the shapeupsert'sverifyIdentitybranch uses.trx.transaction()opens a knex nested transaction: aSAVEPOINT, released on success and rolled back to on failure. That is the primitiveattemptWithoutPoisoningis built on, already verified in this file on PG 16 and on better-sqlite3 at poolmax: 1, where the savepoint rides the parent's connection and never asks the pool for a second one. This matters concretely here: SQLite hands out exactly one connection (assertBareKnexSafe's subject), so a wrapper that reached forthis.knexwhile a caller held a transaction would dead-lock.upserttakes at its own boundary — would be wrong here, and the difference is N statements versus one.upsertreasons "the statement is already transactional", true of one statement and false of this loop: a caller that catches the refusal and commits anyway lands exactly the partial batch this card is about, reachable on SQLite and MySQL (Postgres aborts the whole transaction on any statement error, so it is the one dialect where the hole was already shut by accident). The half ofupsert's reasoning that does transfer is kept — the rollback decision for the caller's own work stays the caller's, and its transaction is left usable.An empty batch still issues no statement at all, so that path is byte-identical to
main.driver-tursois untouched, by designTursoDriver extends SqlDriverand overridesbulkUpdateonly to route the remote path toremoteTransport; its local path isreturn super.bulkUpdate(object, updates, options). It therefore inherits this repair with no subclass change — which is the point, since patching the subclass would leave the next subclass in the same hole. The remote transport path is a different door and is not addressed here.Evidence — the pin would have been red before
packages/drivers/driver-sql/src/sql-driver-13854-bulk-update-atomicity.test.ts. §1 and §4 each assert three things, and only the third is this card:Which refusal, measured rather than assumed. A missing id does not refuse on this door:
update()issues an UPDATE matching zero rows and returnsnull, which the loop skips — a pin built on one would never throw and would measure nothing. §6 pins that skip, since this card must not change it. The refusal used is the database's own: aunique: 'global'column with a later row in the batch assigned a value a third row already holds, measured on better-sqlite3 asUNIQUE constraint failed: account.code.Ablation (on the committed tree, direction predicted before running). Reverting the wrapper to
main's bare loop was confirmed on disk before any reading was taken — anchor counts moved1 to 0and0 to 1in the two directions, and the blob hash moved0bb4e792 to d8efbb46— rather than trusting the editor's exit code:Exactly assertion 3, in both sections, with the received value being the committed post-image. Assertions 1 and 2 stayed green, and §2/§3/§5/§6 stayed green — they pin what must not change. Restored with an absolute-path
git checkout HEAD -- FILE, proven by an emptygit diff HEADand a blob hash back to0bb4e792.No build is involved in that ablation and none is owed: the pin imports
../src/index.js, a relative source path, so the run reads the mutated source directly — which the red itself demonstrates.The reading #13854 owed on
driver-mongodbMeasured, and the answer is not the same shape (
mongodb-driver.ts):bulkUpdate(:505) is not a per-row loop. It maps the updates intobulkOpsand issues onecollection.bulkWrite(bulkOps, { session }).bulkDelete(:534) is onecollection.deleteMany({ id: { $in: ids } }, { session })— the same single-statement shape asSqlDriver.bulkDelete.sessionisgetSession(options)(:687) =options.transaction as ClientSession, the same parameterisation convention.So the autocommit loop repaired here does not exist there. A related exposure may still be reachable by a different mechanism —
bulkWritestops at the first failing op but does not undo earlier ones without a transaction, and mongo transactions need a replica set, so the repair is not a mechanical port of this one. Filed separately as #14169 rather than widening this PR into a third package, with the measured and not-measured halves marked apart.Verification
All at
8345bb5a, the final commit.pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2— 150 passed / 9 skipped (159 files), 2279 tests passed / 136 skipped, zero failures. The 9 skipped files are the live-dialect matrix with no Postgres/MySQL URL provisioned.pnpm --filter @objectstack/driver-sql typecheck— clean, and measured to actually cover the new test file (tsc --listFilesreports 1 hit; this package's tsconfig includessrc/**/*with no test exclusion, so the green is about the new file too).pnpm --filter '@objectstack/driver-sql^...' buildbefore the suite, so nothing read a staledist.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands(35 commands) and run with exit codes captured by redirect before any pipe: 33 measured green. Three arePREREQUISITE NOT MET/ exit 3 —check-test-completeness,check:dual-build-cjs-loads,check:type-check-debt— each of which needs a full workspace build and says in its own words that it is "NOT a pass and NOT a finding: nothing was measured". Recorded as NOT MEASURED, not as passes; CI runs them with the build in place.Scope narrowing, declared:
driver-sqlhas 48 downstream packages and the farm belongs to CI. Thedriver-tursoquestion — the one that matters, since it inherits this door — was settled statically by reading its override (local path delegates tosuper.bulkUpdate) rather than by a suite run; a downstream build sweep was attempted and abandoned after it failed on an unrelated filter artifact (@objectstack/runtime's DTS build cannot resolve@objectstack/rest, which is not a dependency of either driver and was never built by the filter).Clause-②: no — re-declared from the actual diff. No exported type, signature or public error shape changes; every input accepted before is accepted after and every refusal that fired before still fires. What changes is what survives a mid-batch refusal.
Generated by Claude Code
Generated by Claude Code