🤔 Background
A random-order run prints its seed so the run can be replayed:
Randomized with seed: 12345
Replaying it means typing that seed back. On its own it selects nothing — --seed only pins a shuffle that --random-order has to turn on separately, and BASHUNIT_SEED is read nowhere else (src/main/run.sh:121-126). So:
./bashunit --seed 12345 tests/ # defined order, green
The run comes back in defined order and passes, and the shuffle that produced the failure never happened. Nothing reports that the flag did nothing, so the natural reading is that the flake is fixed.
Verified on main: --seed 42 alone produces byte-identical output to no flags at all; only --random-order --seed 42 shuffles.
docs/command-line.md makes it worse — it says a failing run "can be replayed exactly with --seed <n>", which is not true as written, while a later line says --seed on its own has no effect.
💡 Proposal
Make --seed <n> imply --random-order. A seed names an order and has no other meaning, so giving one is unambiguous about intent — unlike a nonexistent path, where refusing is right because guessing is impossible.
This is RSpec's behaviour (--seed N ≡ --order rand:N), the one major framework with a standalone order seed. jest's --seed seeds Math.random rather than the order, so it is not the analogue.
An order named explicitly should still win: --order-by defined --seed 42 runs in defined order, whichever side the seed is typed on.
BASHUNIT_SEED should keep its current meaning. A seed at the prompt replays one run; a seed in a config file is read by every run including the nested ones a suite spawns, so flipping those to random order on upgrade is a much bigger change than fixing the replay flow.
🤔 Background
A random-order run prints its seed so the run can be replayed:
Replaying it means typing that seed back. On its own it selects nothing —
--seedonly pins a shuffle that--random-orderhas to turn on separately, andBASHUNIT_SEEDis read nowhere else (src/main/run.sh:121-126). So:./bashunit --seed 12345 tests/ # defined order, greenThe run comes back in defined order and passes, and the shuffle that produced the failure never happened. Nothing reports that the flag did nothing, so the natural reading is that the flake is fixed.
Verified on
main:--seed 42alone produces byte-identical output to no flags at all; only--random-order --seed 42shuffles.docs/command-line.mdmakes it worse — it says a failing run "can be replayed exactly with--seed <n>", which is not true as written, while a later line says--seedon its own has no effect.💡 Proposal
Make
--seed <n>imply--random-order. A seed names an order and has no other meaning, so giving one is unambiguous about intent — unlike a nonexistent path, where refusing is right because guessing is impossible.This is RSpec's behaviour (
--seed N≡--order rand:N), the one major framework with a standalone order seed. jest's--seedseedsMath.randomrather than the order, so it is not the analogue.An order named explicitly should still win:
--order-by defined --seed 42runs in defined order, whichever side the seed is typed on.BASHUNIT_SEEDshould keep its current meaning. A seed at the prompt replays one run; a seed in a config file is read by every run including the nested ones a suite spawns, so flipping those to random order on upgrade is a much bigger change than fixing the replay flow.