Fill pooled databases in the background, and disable page verification - #1044
Merged
Conversation
CreatePool built all PoolSize databases before returning, so the first pooled test paid every file copy and attach up front, and any test that started concurrently queued behind poolLock for the whole build. PoolSize defaults to ProcessorCount, so that is 16-32 serial database creations before the first test runs. Only the first database is now awaited. The rest fill in on a background task while tests run, so the wait overlaps with useful work. The lease semaphore starts empty and gains a permit as each database lands, which changes a lease wait from "every pooled database is leased" to "every database built so far is leased". The fill stays serial: it now runs alongside live tests, and PoolSize concurrent file copies would just compete with them for the same disk. A background failure cannot hang the pool. FillPool records the cause and releases one permit per unbuilt database, so every parked waiter wakes, fails the dequeue, puts its permit back for the next caller, and rethrows the cause. Leases arriving afterwards fail fast on the same check. Also dispose the connection in AddPooled on the failure path, which only began to matter once a failure could happen off the caller's stack. Teardown waits for the fill, since DeleteInstance deletes the directory a background copy may still be writing into.
page_verify defaulted to checksum, so the engine computed a checksum over every 8KB page as it was written to disk and re-verified it on read, to detect storage corrupting a page underneath it. These databases are built from the template and deleted with the run, so that signal is never read. Set on the template, which persists it through detach/copy/attach into pooled, per-test and shared databases alike. Unlike auto_close, which the attach resets to the model default, page_verify survives. PageVerifyBenchmarks measures it: a checkpoint-heavy insert of ~7500 data pages is ~5% faster on average, and none won all five trials, though the between-run variance on the test machine is comparable to the effect. Bytes written are identical either way, so the saving is CPU, not I/O.
A pooled database is leased exclusively, so the row versioning RCSI pays for cannot buy anything there. Measured against the write-then-roll-back shape a pooled test has, turning it off is worth ~11% wall clock, ~29% less write I/O, ~38% less read I/O and ~44% less memory, consistently across three runs. Not applied. RCSI is what lets a second connection to a leased database read while the lease holds an uncommitted write, and that is public API: OpenNewConnection, NewConnectionOwnedDbContext and IDbContextFactory .CreateDbContext all open one. Probed both ways against a pooled lease mid-transaction: with RCSI on the read returns the last committed state, with it off it blocks on the lease's X locks until the command timeout. Trading a documented API for 11% is not worth it, so the benchmark is kept as the record of the trade rather than the change. accelerated_database_recovery was the other candidate, since every lease ends in a rollback. It cannot be used at all: LocalDB is Express edition and enabling it fails with error 12128.
ADR is unavailable on LocalDB, so measure what its absence actually costs before looking for a substitute. RollbackCostBenchmarks isolates the rollback from the write: a fixed floor of ~85us plus ~0.94us per modified row, linear as expected for log-walking undo. At the volumes a test writes that is nothing - ~93us for 10 rows, ~161us for 100 - and only reaches ~9.5ms at 10000 rows. Database snapshots, unlike ADR, do work on LocalDB, and a revert is the one available mechanism with ADR's shape: page-level undo against a sparse file rather than a walk backwards through the log. The snapshot also survives the revert, so one baseline can be reverted to repeatedly. ResetStrategyBenchmarks compares it against the transaction rollback end to end, including the reconnect a revert needs for exclusive access. Revert loses badly: 172ms against 0.47ms at 100 rows, 307ms against 27ms at 10000. Its cost is almost entirely fixed, so the gap narrows as the write grows but never closes at any volume a test reaches. Rolling back the lease transaction stays the right mechanism; both benchmarks are kept as the record.
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Sep 1, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes to the pooled database path, plus three measurements that argued against changing anything else.
Fill the pooled databases in the background
CreatePoolbuilt allPoolSizedatabases before returning, so the first pooled test paid every file copy and attach up front, and any test starting concurrently queued behindpoolLockfor the whole build.PoolSizedefaults toProcessorCount, so that is 16-32 serial database creations before the first test runs.Only the first database is now awaited. The rest fill in on a background task while tests run, so the wait overlaps with useful work. The lease semaphore starts empty and gains a permit as each database lands, which changes a lease wait from "every pooled database is leased" to "every database built so far is leased".
The fill stays serial deliberately: it now runs alongside live tests, and
PoolSizeconcurrent file copies would just compete with them for the same disk.A background failure cannot hang the pool.
FillPoolrecords the cause and releases one permit per unbuilt database, so every parked waiter wakes, fails the dequeue, puts its permit back for the next caller, and rethrows the cause. Leases arriving afterwards fail fast on the same check. Teardown waits for the fill, sinceDeleteInstancedeletes the directory a background copy may still be writing into.PooledFillFailureTestscovers that failure path by faulting the fill while a second lease is parked on the semaphore. It is mutation checked: with the wake-up release commented out it fails at the 30s timeout, and passes in 8s with it restored.Disable page verification on the template
page_verifydefaulted tochecksum, so the engine computed a checksum over every 8KB page as it was written to disk, and re-verified it on read, to detect storage corrupting a page underneath it. These databases are built from a template and deleted with the run, so that signal is never read.Set on the template, which persists it through detach/copy/attach into pooled, per-test and shared databases alike. Unlike
auto_close, which the attach resets to the model default,page_verifysurvives - verified against all three database kinds.PageVerifyBenchmarksmeasures it: a checkpoint-heavy insert of ~7500 data pages is ~5% faster on average, andnonewon all five trials, though between-run variance on the test machine is comparable to the effect. Bytes written are identical either way, so the saving is CPU, not I/O. Worth taking mainly because the thing given up is worthless here, not because the win is large.Measured and deliberately not applied
Three further settings were investigated. None resulted in a change, and the benchmarks are kept as the record so they are not re-litigated from scratch.
read_committed_snapshot offfor pooled - the biggest win of the lot, and refused. A pooled database is leased exclusively, so its row versioning buys nothing: turning it off is worth ~11% wall clock, ~29% less write I/O and ~38% less read I/O, consistently across three runs. But RCSI is what lets a second connection to a leased database read while the lease holds an uncommitted write, and that is public API -OpenNewConnection,NewConnectionOwnedDbContextandIDbContextFactory.CreateDbContextall open one. Probed both ways against a real pooled lease mid-transaction: with RCSI on the read returns the last committed state, with it off it blocks until the command timeout. Not worth trading a documented API for 11%, especially as it would surface as a mysterious timeout.accelerated_database_recovery- unusable. LocalDB is Express edition (EngineEdition4) and enabling it fails with error 12128, regardless of syntax or connection.Substitutes for ADR - none worth having, and none needed.
RollbackCostBenchmarkssizes the gap ADR would close: a ~85us floor plus ~0.94us per modified row, so a test touching 10-100 rows spends under 0.2ms on rollback. Database snapshots do work on LocalDB and are the one available mechanism with ADR's shape, butResetStrategyBenchmarksshows a revert is 370x slower than the rollback at 100 rows and 11x slower at 10000 - its cost is almost entirely fixed, so the gap narrows with write size but never closes at any volume a test reaches.Verification
Full suite green, 0 failures:
Solution builds clean across net8.0/net9.0/net10.0/net48 with
TreatWarningsAsErrors. Pooled tests additionally run green atLocalDBPoolSizeof 1, 2 and 8 - size 1 exercising the case where the fill loop never runs, size 2 putting four concurrent tests against two databases while the fill is still in flight.