Fix the ASGI pool concurrency test racing on WAL conversion - #850
Merged
Merged
Conversation
test_pool_bounds_concurrency failed intermittently with "database is locked", raised from the connection's init_command rather than from any query. The pragmas include journal_mode=wal, and the fixture handed the pool a database file that had never been opened. Converting a file into WAL takes an exclusive lock, and SQLite answers that with SQLITE_BUSY immediately instead of waiting out the 120 second busy timeout, so the two pool workers opening that new file at the same instant raced and one lost. Reproduced at 22 failures in 60 attempts in isolation, and 3 in 8 runs of the test itself. The database codex serves is never in that state: startup migrates it on one connection well before the worker pool or the librarian exists, and asserting WAL on a file already in WAL needs no exclusive lock. So put the fixture's file in WAL when it is created, which is the state it was always meant to mirror. 30 runs of the test, clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the intermittent
test_pool_bounds_concurrencyfailure that broke thev2.3.2 CI run on
develop.Not a product bug
The traceback points at Django's
get_new_connection, inside the loop thatruns
init_command, not at any query:The first pragma is
journal_mode=wal. Converting a file into WAL takes anexclusive lock, and SQLite answers with
SQLITE_BUSYstraight away ratherthan waiting out the configured 120 s busy timeout. The
aliasfixture handsthe pool a database file nothing has ever opened, so the two workers convert
it concurrently and one loses.
Isolated repro, two threads opening one fresh file with the codex pragmas and
a 120 s timeout:
The test itself failed 3 times in 8 local runs before this change.
The database codex serves is never in that state.
codex_startuprunsmigrateon a single connection beforerun()starts the librarian or theserver, so the file is in WAL before a second connection exists, and asserting
WAL on a WAL file needs no exclusive lock.
The fix
Create the fixture's file in WAL, which is the state it was always meant to
mirror. One
sqlite3.connectplus one pragma, before the alias is yielded.Verified: 30 consecutive runs of
test_pool_bounds_concurrencyclean, 5consecutive runs of the whole file clean, 1123 pytest and 500 vitest passing,
make lintandmake tyclean.🤖 Generated with Claude Code