Flaky CI: RapidTrackChangesNeverDropTheOutgoingTracksAudio fails ~1 run in 3

[revtex]

Session_RapidTrackChangesNeverDropTheOutgoingTracksAudio fails intermittently on CI — roughly one run in three — while passing consistently on a developer machine. It is green and red on identical code, so it is a flake rather than a regression.

Evidence

Same commit, two runs:

• run 31757208049, first attempt — FAILURE
• same run, re-run of the failed job, no code change — SUCCESS

The failure:

Assert.Equal() Failure: Values differ
Expected: 4
Actual: 3
 at RecordingSessionTests.Session_RapidTrackChangesNeverDropTheOutgoingTracksAudio()
 tests/Offstream.Core.Tests/Recording/RecordingSessionTests.cs:line 499

Locally it passes 15/15, including runs pinned to DOTNET_PROCESSOR_COUNT=1 and =2. It does not reproduce in isolation — on CI the whole suite runs in parallel on a shared two-core runner, which is different contention from a single filtered test.

Why this is worth fixing rather than re-running

Two costs, and the second is the expensive one:

1. It blocks unrelated PRs at random. It has already done so once, on Phase 7: raise the TFM, and make SMTC the primary track source #13, which touches nothing under Recording or Audio.
2. It trains everyone to hit re-run on a red build — which is precisely how a real regression gets waved through.

The test must not be weakened to silence it. It guards a bug that already shipped once: a track change discarding the outgoing track's audio tail, fixed in 8d4dca5 by having RecordingSession.OnTrackChanged block on TrackRecorder.BufferDrained. The 20-iteration loop is there because a single pass proves nothing about which of two tasks the thread pool happens to schedule first. Relaxing it would let that regression back in unnoticed.

The specific question to answer

Which assertion failed is the clue. These run in order:

Assert.DoesNotContain(harness.Recorded, r => r.Outcome == RecordingOutcome.Silent); // passed
Assert.Equal(4, harness.Saved.Count); // failed, 3

The check guarding the actual audio-drop bug passed. Only the save count was short, so no track lost its audio — one save had simply not been observed yet.

harness.Saved is filled from the TrackSaved event. RecordingSession.StopAsync awaits _backlog.CompleteAsync(cancellationToken) before returning, so by the time the assertion runs every save is supposed to have been raised.

So: can _backlog.CompleteAsync return before every TrackSaved event has been raised?

• If yes, this is a production defect, not a test one. StopAsync would be returning while a save is still in flight, meaning a user who stops a recording can be told it finished before the file is written. The test is right and the contract needs fixing.
• If no, the gap is in the test — most likely that the event is raised on a pool thread and the enqueue is not ordered against CompleteAsync returning. Then the fix is to wait for the count with a timeout, the way the other tests in this file already use WaitFor, keeping the Silent assertion exactly as it is.

Worth resolving in that order: the production contract first, the test only if the contract turns out to be sound.

Not urgent, but not nothing

No user-visible symptom is known. The risk is to the signal quality of the build, which is why it is worth a real fix rather than a suppression.