Problem
Nothing stops a test from calling a real command. A test that means to exercise a mocked curl but has a typo in the mock name (or whose mock was cleared by a previous bashunit::unmock) will happily hit the network — and pass, slowly, non-deterministically, and differently in CI. The same applies to git, aws, docker, rm.
bashunit already fixed the adjacent hole for spies: #895 made assertions on a command that was never spied fail loudly instead of vacuously passing. The command side has no equivalent guard.
This is shellspec's headline safety feature: sandbox mode makes PATH empty and readonly except for the mock directory, so any unmocked external command fails immediately.
Proposal
--sandbox Fail any test that executes an external command that is not mocked
--sandbox-allow <cmd,…> Commands that stay available (repeatable)
- Under
--sandbox, an unmocked external command produces a clear failure naming the command and the test — not a generic 127. - A baseline allowlist of what bashunit itself needs internally must be established (
awk, sed, grep, cat, … depending on the code path) and documented; the goal is to constrain the test body, not to break the framework. - Shell builtins are unaffected.
- Off by default; opt-in per run and settable in
.bashunitrc.
Design note for whoever picks this up: emptying PATH is one implementation, a command_not_found_handle hook is another, a PATH prepended with a shim directory is a third. The shim approach composes best with the existing mock registry (src/doubles/mock.sh:45) — evaluate all three and record the choice in an ADR, since this is an architectural decision.
Acceptance criteria
Repo checklist (agent)
- TDD: RED → GREEN → REFACTOR. Write the failing test first.
- Bash 3.0+ only: no
printf -v, no += append, no declare -A, no [[ ]], no ${var,,}, no &>>, no ${arr[-1]}. Expanding a possibly-empty array under set -u needs ${arr[@]+"${arr[@]}"}. - A new CLI flag must be wired in all of these or a parity test fails:
- Gates:
make sa, make lint, ./bashunit tests/, ./bashunit --parallel tests/. Never run shfmt -w. - Docs: update
docs/command-line.md. Editing docs/assertions.md invalidates the bashunit doc acceptance snapshot — regenerate it. - CHANGELOG.md: add one line under
## Unreleased. - Fixtures under
tests/acceptance/fixtures/ must not end in *test.sh. - One issue = one PR.
Problem
Nothing stops a test from calling a real command. A test that means to exercise a mocked
curlbut has a typo in the mock name (or whose mock was cleared by a previousbashunit::unmock) will happily hit the network — and pass, slowly, non-deterministically, and differently in CI. The same applies togit,aws,docker,rm.bashunit already fixed the adjacent hole for spies: #895 made assertions on a command that was never spied fail loudly instead of vacuously passing. The command side has no equivalent guard.
This is shellspec's headline safety feature: sandbox mode makes
PATHempty and readonly except for the mock directory, so any unmocked external command fails immediately.Proposal
--sandbox, an unmocked external command produces a clear failure naming the command and the test — not a generic 127.awk,sed,grep,cat, … depending on the code path) and documented; the goal is to constrain the test body, not to break the framework..bashunitrc.Design note for whoever picks this up: emptying
PATHis one implementation, acommand_not_found_handlehook is another, a PATH prepended with a shim directory is a third. The shim approach composes best with the existing mock registry (src/doubles/mock.sh:45) — evaluate all three and record the choice in an ADR, since this is an architectural decision.Acceptance criteria
curlunder--sandboxfails with a message namingcurlcurlis mocked withbashunit::mock--sandbox-allow curllets it throughecho,printf,[,test) are unaffected--sandbox(the framework is not sandboxed against itself)--sandbox, behaviour is byte-identical to today--paralleland withset_up/tear_downhooksbashunit::unmockmid-test is defined and testedadrs/recording the mechanism chosen and whydocs/test-doubles.mdRepo checklist (agent)
printf -v, no+=append, nodeclare -A, no[[ ]], no${var,,}, no&>>, no${arr[-1]}. Expanding a possibly-empty array underset -uneeds${arr[@]+"${arr[@]}"}.src/main/test.sh(report-style flags needexport -n, seesrc/main/test.sh:188-196for why)bashunit::main::validate_config_or_exit(src/main/validate.sh:60) — unvalidated input used to run the wrong thing and exit 0 (Unknown options are silently ignored: a typo'd flag runs a different suite and exits 0 #871, --jobs with a non-integer value hangs on Bash 3.x and is silently ignored on Bash 4.3+ #873)src/config/env.shand a documented line in.env.example--helptext in the same block it belongs tocompletions/bashunit.bashandcompletions/_bashunit(anti-drift test feat(cli): bash and zsh completion scripts with an anti-drift test #778 fails otherwise)make sa,make lint,./bashunit tests/,./bashunit --parallel tests/. Never runshfmt -w.docs/command-line.md. Editingdocs/assertions.mdinvalidates thebashunit docacceptance snapshot — regenerate it.## Unreleased.tests/acceptance/fixtures/must not end in*test.sh.