Uh oh!
There was an error while loading. Please reload this page.
test(fleet): wait on conditions instead of fixed sleeps in TUI poll tests - #478
Merged
Merged
Conversation
…ests The Fleet Manager TUI pilot tests assert on state that only becomes true once a real `set_interval` poll tick lands, but waited a fixed `asyncio.sleep(0.3)` or sampled a transient flag. CI runs pytest under `coverage` tracing on a small runner, where a tick-plus-scan is several times slower than on a developer box, so those waits asserted against pre-tick state and failed intermittently on main: * `test_a_scan_failure_does_not_wedge_the_screen` sampled the transient `_refreshing` flag. Measured, `pilot.pause()` costs ~1s under the 20Hz test timer, so its 5s deadline got only ~4 samples while the flag is legitimately True 19.5% of the time idle and 34.8% under coverage -- it could see True on every sample and report a wedged screen. * `test_poll_tick_removes_completed_run` failed on Windows CI as `assert 1 == 0` because 0.3s was not enough for a tick. Add `wait_for()` to the fleet conftest: poll a predicate to a deadline, sampling with `asyncio.sleep` rather than the ~1s `pilot.pause()`, and fail with a message naming the regression. Convert the affected sites in `test_tui_runs.py` and `test_tui_run_detail.py`. Rewrite the two guard-recovery tests to assert a monotonic, user-visible consequence -- a newly written record appears in the table -- rather than sampling a transient flag. That condition cannot be missed by any sampling rate, is unreachable if the guard latched, and proves the screen recovered all the way to rendering. Both still fail deterministically when the `finally` in `_refresh_worker` is removed. Also fix two latent siblings of the same bug: an `assert call_count == 1` that could observe 0 before the mount refresh had entered the collector, and a selection-preservation test that passed vacuously when no tick landed within the fixed sleep. Verified under 2 contended CPUs with coverage (~6x slowdown, worse than CI): the reworked tests passed 8/8 where the previous versions reproduced the failure 1/8. No `src/` change -- the `finally` these tests guard was already correct. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 24, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Main's CI has been failing intermittently. The failures are flaky tests, not a product regression —
src/needs no change.Two failures on
main, both in the Fleet Manager TUI pilot tests:test_a_scan_failure_does_not_wedge_the_screen_refreshing never clearedtest_poll_tick_removes_completed_runassert 1 == 0The first also failed on 32044377141 (Aug 17), in its pre-#464 single-sample form — so this is a recurring flake a previous fix attempt did not close.
Root cause
These tests assert on state that only becomes true once a real
set_intervalpoll tick lands, but wait a fixedasyncio.sleep(0.3)or sample a transient flag. CI runs pytest undercoveragetracing on a small shared runner, where a tick-plus-scan is several times slower than on a developer box, so the fixed wait encodes a guess about machine speed and asserts against pre-tick state when the guess is wrong.The guard-recovery test is the more interesting one. Instrumenting it:
pilot.pause()costs ~1s per call under the tests' 20 Hz timer (it waits for the app to go idle, which a 0.05s poll timer keeps deferring), so the helper's 5s deadline yielded only ~4 samples._refreshingis legitimatelyTrue19.5% of the time idle and 34.8% under coverage.So the assertion was a coin flip that a slow machine loads — it can see
Trueon every sample and report a wedged screen. Thefinallyin_refresh_workerthat it guards is correct and present.Fix
wait_for(pilot, predicate, message=...)totests/test_fleet/conftest.py: polls a condition to a deadline, samples withasyncio.sleeprather than the ~1spilot.pause(), and fails with a message naming the regression instead of an opaqueassert 1 == 0.test_tui_runs.pyandtest_tui_run_detail.py.assert call_count == 1that could observe0before the mount refresh had entered the collector, and a selection-preservation test that passed vacuously when no tick landed inside the fixed sleep.Verification
finallyin_refresh_workermakes both rewritten tests fail deterministically in ~2s with the intended message (no scan ran while the failure was injected -- the screen was already wedged...). The new assertions are not vacuous.-m "not real_api and not performance", with coverage): 7426 passed, 60 skipped.make lintandmake typecheckclean.settle().