Uh oh!
There was an error while loading. Please reload this page.
fix(driver-sql): bound lock_wait_timeout on the deferred-DDL flush so a blocked os migrate refuses loudly instead of hanging for a year - #9547
Conversation
… a blocked migrate refuses loudly (#9354) The MySQL widening ALTERs in the deferred-DDL flush ran on a session inheriting MySQL's default lock_wait_timeout of 31,536,000s (one year), and the widening swallows its failures. A metadata lock held by another session therefore parked `os migrate apply` silently, with no output an operator could diagnose from. The flush now pins one connection, bounds lock_wait_timeout to 120s on that same session, and re-throws a metadata-lock timeout as an ADR-0112 envelope (DATABASE_ERROR / 500) naming the lock wait, the table and the bound. Boot sync and every non-lock-wait failure keep the behaviour they had. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017qYPmkKEsfbWY1yVg83p8F
📓 Docs Drift CheckThis PR changes 1 package(s): 5 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not seeCoarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 6cba128126ee68b093d2970aa4f99ffd959d3aed && git checkout 6cba128126ee68b093d2970aa4f99ffd959d3aed
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin b348ac2c24100e26bf88486f704e3064a2e1fee6 272bd8f9643a13b723d84814cdee524b7e1fbe59 && git checkout -B drift-repro b348ac2c24100e26bf88486f704e3064a2e1fee6 && git merge --no-ff 272bd8f9643a13b723d84814cdee524b7e1fbe59
node scripts/docs-audit/affected-docs.mjs --json b348ac2c24100e26bf88486f704e3064a2e1fee6
|
… Error overload (#9354) `new Error(msg, { cause })` needs the ES2022 lib; packages/drivers/driver-sql targets ES2020, and its tsconfig includes `src/**/*` with no test exclusion, so `tsc --noEmit` compiled the new pin file and failed: src/sql-driver-deferred-ddl-lock-wait.test.ts(57,95): error TS2554: Expected 0-1 arguments, but got 2. The fixture now defines `cause` with Object.defineProperty — the same idiom the driver's own refusals in sql-driver.ts use, reproducing what the constructor produces at runtime including non-enumerability. No tsconfig change, no cast, no baseline moved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017qYPmkKEsfbWY1yVg83p8F
Uh oh!
There was an error while loading. Please reload this page.
Fixes#9354
Implements the maintainer ruling of 2026-08-17 (recorded on the card, verbatim 「同意」): Option A, minimal version.
The behaviour
The deferred-DDL flush widens legacy MySQL
TIMESTAMPcolumns toDATETIME(3)(#3942) andTIMEtoTIME(3)(#3994) withALTER TABLE … MODIFY COLUMN, which needs an exclusive metadata lock on the table. That ALTER ran on a session inheriting MySQL's defaultlock_wait_timeout— 31,536,000 seconds, one year. One other session holding a lock on the table parks the ALTER inWaiting for table metadata lockfor that long, and nothing prints.An operator running
os migrate applyagainst a busy production table meets this as a command that simply hangs, indistinguishable from a crash. It was first measured as a CI stall: a sub-second test blew a 5000ms budget with no error at all.Two things were wrong, and bounding alone would have fixed neither
lock_wait_timeouthad zero occurrences anywhere underpackages/, withflushDeferredSchemaDdlas the positive control returning real hits.migrateMysqlDatetimeColumnscatches, logs a warn and returns. So a bound on its own would have produced a MySQL error that was immediately discarded —os migrate applywould print "Applied 0 change(s)" and exit 0, reporting success for work it did not do. The hang would have become a silent no-op, which is not obviously an improvement.The change
The flush now runs its widening ALTERs through one seam,
runWideningAlters:lock_wait_timeoutis a SESSION variable, so aSET SESSIONissued through the pool lands on a connection the ALTER never uses — a no-op that looks exactly like a fix. The seam holds a single connection (the same knex-transaction affinity every other multi-statement unit in this file already uses) so both statements provably ride one session. MySQL implicitly commits on DDL, which costs nothing here: the transaction is for connection affinity, not atomicity.DATABASE_ERROR/ 500, from the existing closed vocabulary, minting no new code — whose message names the lock wait, the table, the bound it hit, and how to find the holder. Every other failure keeps the swallow it has today.initObjectslines and must not be taken down by a migration, so it still runs unbounded and still swallows.Recognition is by errno 1205 /
ER_LOCK_WAIT_TIMEOUT, followed throughcause(knex re-throws with the driver error attached). Message text is deliberately not sniffed — MySQL and MariaDB word it differently and both translate it, so a prose match is a recognizer that fails silently in another locale, back to the year-long hang.The proposed bound: 120 seconds
The ruling delegates the value, minutes-scale, with reasoning. It is a diagnosis deadline, not a capacity knob — its job is to end the silence, not to decide how patient a migration may be. So it is the shortest wait that still clears legitimate blockers:
SHOW PROCESSLISTor killed it, so a bound firing later than that arrives after the diagnosis it was meant to provide.os migrate applyis re-runnable and the widening is idempotent, so an over-eager bound costs one re-run — against an unbounded hang as the cost of one that never fires.⛔ Not configurable and not retried, per the ruling's explicit minimality. Both wait for measured demand.
Scope decision the ruling did not fully close
The card's own sub-question asked whether the bound covers all deferred-DDL statements or only the widening ALTER. Implemented: both MySQL widening ALTERs — datetime and time — and nothing else. They are adjacent calls in the same flush, the same statement class, taking the same exclusive metadata lock on the same connection, and they share one helper here rather than two copies. Bounding one while its twin still waits a year would leave
os migratehanging exactly as before, with the operator unable to tell which ALTER hung. Create-table / add-column DDL is not bounded: that would mean routing knex's schema builder through the pinning seam, which is a materially larger change than the ruling's "a few lines plus a pin test".Tests
Ten pins in
sql-driver-deferred-ddl-lock-wait.test.ts. They assert the refusal, not that aSET SESSIONstring was emitted — a suite asserting only emission passes in full while the operator still hangs. Only the connection and theinformation_schemaprobe are faked, so the bounding, the 1205 recognition, the envelope and the escape from the swallow all execute for real.Reverse verification, direction predicted before running:
caught()go redThe first ablation going red is also what proves the suite reads this branch's source rather than a stale
dist— a stale read would have stayed green. The restore leg was verified byte-identical to the committed tree (git status --porcelainempty) and re-run green, 10/10.Gates
Re-derived against the actual diff with
node scripts/pm/dispatch-gates.mjs(no path arguments) and run at0c9a3db. All green, includingcheck:type-check-debt --re-measureon the built workspace closure (33 ledger entries re-measured in 267s, none above its recorded number) and the full@objectstack/driver-sqlsuite (1756 passed / 0 failed) plus the CLI deferred-DDL integration test.Deliberately not in this PR
Two findings were filed unassigned rather than folded in, both outside the dispatched file surface:
content/docs/data-modeling/drivers.mdxdocuments the driver's connect-timeout defaults but not this new bound. The docs-audit mapper names that page; the dispatched file surface did not includecontent/docs/**.Related: #9350 (the metadata-lock measurement — this change is independent of it and makes the failure diagnosable either way) · #3954 · ADR-0112.
Generated by Claude Code