Make the test suite tell the truth (MOB-154, MOB-119, MOB-123)

[GenericJam]

Closes MOB-154, MOB-119, MOB-123.

A 1-in-20 flake corrupted a mutation-testing verdict and, a day later, sent a bisect down the wrong path when it appeared in the same run as a real failure. The cost of a noisy suite is not the red build, it is the hours spent trusting it.

Three mechanisms

A globally-named singleton owned by whichever test started it.Mob.ComponentRegistry is named and owns a named ETS table. Two async: true modules each called start_supervised({Mob.ComponentRegistry, []}) and tolerated {:error, {:already_started, _}}. The winner owned it, and ExUnit tore the process and its table down when that test ended, while a concurrent test in the other module was still using it. It now starts in test_helper.exs, owned by the run.

Check-then-act across a process boundary. 19 on_exit(fn -> if Process.alive?(pid), do: GenServer.stop(pid) end) sites across 11 modules. Since MOB-112 the screen owner is linked to the test process, which ExUnit exits with :shutdown at test end, so the owner is dying concurrently with the callback trying to stop it. 13 further modules had each independently written the correct private helper, byte for byte identically.

Fixed durations standing in for synchronisation.

Not every sleep is a bug

All 35 were classified. 12 are Process.sleep(:infinity) — a parked stub, not a wait. Of the 23 finite ones, 15 went and 8 remain:

• 5 had nothing to wait for. A GenServer.call from the same process that sent the earlier messages is already a barrier: Erlang orders messages pairwise, so every send is ahead of the call. And Trace.broadcast/3 folds over the table in the calling process, so its cleanup is done before dispatch/4 returns.
• 7 became the real barrier — a ready-message, Logger.flush/0, a monitor.
• 3 became a bounded poll (eventually/2), for the one shape where pairwise ordering genuinely does not help: a GenServer processing a :DOWN sent by a monitor, not by the test.
• 8 stay: 3 are the subject under test in render_stats_test.exs (it measures elapsed time), 3 are the backoff inside a poll loop with its own deadline, 1 is a @doc example quoting the bad pattern, and 1 is a genuine bet (router_hot_path_test.exs:206) recorded rather than disguised.

mix mob.flake

Runs the suite repeatedly and reports which tests are non-deterministic. Its green output says outright that 20 green runs miss a 1-in-17 flake about 30% of the time, because "I ran it 20 times" is the reasoning that let this persist.

Honesty notes

• The registry race is fixed by construction, not by demonstration. It did not reproduce in 25 full-suite runs or 40 concentrated ones. The mechanism is provable by reading the code and was observed once here; that is weaker evidence than a reproduction and the record says so.
• Tightening stop_pid/2 broke 3 tests, and only under a full run. Making the timeout raise meant replacing a blanket :exit, _ -> :ok with enumerated clauses, and the enumeration was wrong: a linked owner dying during teardown exits with {{:shutdown, {:sys, :terminate, _}}, {GenServer, :stop, _}}. Every file passed alone; the full run failed 3 tests, then 4. Fixed by inverting — special-case only the timeout, treat every other exit as "already gone". Enumerating the uninteresting set is a bet on having seen every shutdown shape, the same class of mistake as betting on a duration.
• The Logger.flush/0 swaps are not demonstrably load-bearing — with the barrier deleted outright those tests still passed 8/8 here. flush/0 is kept because it is the correct primitive and free when nothing is pending, whereas the sleep cost 300ms per run. That is a reason, not a measurement.

Verification

• Full suite 5x green: 1557 passed, 38 excluded (master: 1550).
• mix credo --strict clean, mix format --check-formatted clean.
• mix mob.flake verified against a deliberate 2-failure test: reports both, exits 1. Its first version silently dropped failures near the end of the output via chunk_every(12, 1, :discard); now it splits on the failure-header regex.
• New process_helpers_test.exs (7 tests) pins the helper's own contract — including that stop_pid/2raises on timeout. That test caught a real bug in the first version of the catch clause, which matched a bare :timeout that GenServer.stop/3 never emits.

Decision record: decisions/2026-09-06-tests-wait-for-events-not-durations.md

🤖 Generated with Claude Code