test(vault): separate the client timeout from sqlx acquire - #370
Merged
Merged
Conversation
LKSNDRTMLKV
force-pushed
the
fix/publish-serve-cycle-does-not-hang
branch
from
September 17, 2026 13:57
0015164 to
f331730
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to #336. Does not close it — this is a narrowing, not a fix, and it should not be merged as though the flake were solved.
What I could not do
Reproduce it. Eight consecutive runs of
publish_serve_cycleagainst Docker, all green, ~4.7 s each against a 10-second slow-test budget and a 120-second harness ceiling. That matches the issue's own observation that a re-run on the same commit passed, and it means the stall needs CI load to appear.No root cause, so no fix. A guess at a flake is worse than none, because it looks solved and the next occurrence gets attributed to something else.
What the investigation did establish
The instrumentation this issue asks for already exists, and its author said exactly what it was for. #340 (
6796e87) gaveTestClienta request timeout and named-URL reporting, closing with "This does not fix a stall. It makes the next one diagnosable, which is the prerequisite for fixing it."So the issue's title is stale: post-#340 a stall does not reach the 120-second ceiling, it fails at thirty with a named URL.
And it has now paid out. The first real data point this issue has ever had arrived today, on an unrelated dependency bump:
Two things follow that were not known before: it is the public read that stalls, not the publish, and the server had already accepted the connection — so this is not startup, port binding or container readiness.
🚨 The flaw this change fixes
PgDal::connectbuilds its pool withmax_connections(10)and no explicitacquire_timeout, so sqlx applies its default — which is also thirty seconds.The two candidate explanations for "accepted and never answered" are:
acquire;At thirty-all they expire together. Whichever fires first is a race, and the failure line is identical either way — so every occurrence costs a CI cycle and distinguishes nothing. That is the outcome #340 existed to prevent, defeated by a coincidence of defaults.
Raising the client to 45 s separates them. If the cause is pool starvation, the request now returns a 500 naming the acquire failure while this client is still waiting. If it is anything else, the named-URL timeout fires exactly as before. Still far below the 120-second ceiling, so a stall remains a failed request rather than a killed test.
One observation will now say which of the two it is.
Why the timeout moved rather than the pool
Setting an explicit, shorter
acquire_timeoutonPgDal::connectwould separate them too, and more directly. It is not done here because that is production wiring, and changing how long a live node waits for a database connection to make a test suite easier to diagnose is the wrong reason to touch it. If a shorter acquire timeout is right, it is right on its own merits and belongs in its own change.This is one constant in a test helper.
just checkgreen.