docs: bring the reference pages back to parity with the code by Chemaclass · Pull Request #1063 · TypedDevs/bashunit

Chemaclass · 2026-08-11T13:00:48Z

🤔 Background

An audit of the docs against src/ found the reference pages several releases behind the code, and two defects in the code the docs described. Every finding was reproduced by running the real binary, not by reading source.

💡 Changes

Corrections where the docs were wrong: the assert_equals examples were inverted (the documented test_success failed), assert_true/assert_false summaries said the opposite of the code, BASHUNIT_DEFAULT_PATH was documented as empty, an empty BASHUNIT_COVERAGE_REPORT was documented as disabling the file, and custom-asserts told readers to call a function that does not exist.
Gaps closed: 17 settings missing from configuration.md, 19 from .env.example, the assert subcommand had no section, and four provider gotchas were undocumented.
Two code fixes the audit surfaced: --coverage-report with no value aborted the run with $2: unbound variable, and --list-format json split the tag list on whitespace where every other consumer splits on commas.
Six parity tests so this cannot rot silently again, comparing sets of names rather than prose. Mutation-tested.
Deduplication: the coverage settings and the diff-coverage narrative existed twice and had already drifted; each now has one owner.

…cumented semantics The assert_equals code-group was inverted: the documented test_success failed and test_failure passed. Verified by running both. Also: assert_true/assert_false summaries stated the opposite of the code, assert_files_not_equals reused the assert_files_equals sentence, the array assertions glob-match against the space-joined array rather than per element, assert_file_contains matches literally while assert_file_not_contains matches as a regex, JSON key assertions treat null/false as absent, the duration assertions error instead of skipping without awk, assert_between reports a usage error, assert_exec silently drops unrecognised flags, and assert_have_been_called_with eats a trailing numeric argument as nth. Directory examples used /home/user, which does not exist on macOS or CI; they now use bashunit::temp_dir. Four cross-references used an underscore anchor VitePress never generates.

Adds the 17 registered settings the page never documented: ORDER_BY, REPEAT, FAIL_ON_FLAKY, EXCLUDE_FILTER, CHANGED, CHANGED_REF, SHARD_INDEX/TOTAL, LIST_TESTS, LIST_FORMAT, REPORT_MD, GHA_ANNOTATIONS, SNAPSHOT_REPORT_UNUSED, COVERAGE_REPORT_HTML, COVERAGE_DIFF, the three coverage detail blocks, and BENCH_MODE. Corrections: DEFAULT_PATH defaults to tests and not empty; an empty BASHUNIT_COVERAGE_REPORT does not disable the file, --no-coverage-report does; parallel does not randomize; -vvv is not the counterpart of --simple; NO_COLOR is the external standard while BASHUNIT_NO_COLOR is the setting; coverage colour thresholds are inclusive; only 43 of 66 settings have an unprefixed alias, the rest are prefix-only by design. The precedence list gains the --env/--boot file, which overrides .env and the ambient environment, and the section now says .env is sourced as shell while .bashunitrc is parsed as KEY=value. The coverage env-var list existed twice and had drifted; configuration.md is now the single owner and docs/coverage.md links to it. coverage.md also stated that path matching decides what is tracked, when only executed files are ever reported (#1053).

Both flags are accepted by the parser (src/main/test.sh:280,284) and documented in docs/configuration.md, but 'bashunit test --help' never listed them. Also registers BASHUNIT_COVERAGE_SHOW_FUNCTIONS and BASHUNIT_COVERAGE_SHOW_UNCOVERED in src/config/env.sh. They shipped read-only from src/coverage/report_text.sh with a :-false guard and no default, which is why they were absent from .env.example and from every settings list. .env.example now covers all 66 registered settings; 19 were missing, including RETRY, SEED, TEST_TIMEOUT, the shard pair, the snapshot trio, REPORT_TAP, REPORT_JSON, PARALLEL_JOBS and WATCH_INTERVAL.

… gaps Every example output in this commit was reproduced against the real binary. Corrections: the JSON schema was missing the flaky summary key and the per-test retries field; the page claimed per-test rows only come from a sequential run, which #1004 fixed; the --retry example predated the flaky counter; --profile prints Time taken before the slowest list, not after; the GitHub Actions annotation sample was not percent-encoded although the prose next to it says it is; the Markdown failure fence repeats the test name and location; init also writes .github/workflows/tests.yml and .env; two output blocks still showed 0.34.1; --coverage-exclude was missing *Test.sh. Additions: a ## assert section for the subcommand that had none, -h/--help in the test table, --no-color and -h in the bench table, --repeat and the three enum checks in the invalid-values list, and the watch constraint that only -f/--filter forwards its value. Deduplication: the eight coverage flags were tabulated twice with disagreeing defaults, so the Test Options table now points at the Coverage section, and the Diff coverage explainer moved to docs/coverage.md, which the page already names as the owner of coverage detail.

Two defects the docs audit surfaced while checking documented behaviour. --coverage-report is documented with an optional value and behaves like --coverage-report-html, but it read $2 unconditionally: omitting the value aborted the run with '$2: unbound variable' before any test ran, and a following flag was consumed as the filename. It now falls back to coverage/lcov.info, and the docs state that a path must be written before the flag because an optional value cannot be told apart from a test path. --list-format json split the tag list on whitespace while every other consumer splits it on commas (src/helper/tags.sh:137). A test with two tags rendered as the single element "slow,fileTag", so the jq recipe documented on docs/command-line.md matched nothing, and a tag containing spaces was split into one element per word.

Best practice 3 told readers to call state::add_assertions_passed, which does not exist under that name and aborts the test with 'command not found'. Every bashunit::assert_that example ended the custom assertion with the call itself, so its documented 'return 1' escaped the test function and one failure was reported twice, as a failure plus a spurious Error. The examples now end with return 0 and the API entry says why. Verified by running them. Also: the 'Fails with' comment quoted the humanised test name instead of the message, bashunit::fail was used in examples but missing from the API reference, the bashunit doc output block showed one docstring line where the whole comment run is printed, the --custom filter example dropped --boot and therefore printed 'No custom assertions found', the guard box did not mention -R/--run-all, the assert_once parameter table existed twice, and two examples used [[ ]] against house style, as did the fixture they mirror. data-providers gains the four gotchas it never documented: the two-line annotation proximity rule, a provider with no rows silently making its test disappear while the suite stays green, one concurrent job per row under --parallel, and providers living in a bootstrap. Plus how a failing row is labelled and how rows behave under --repeat, --retry, --tag, --filter and --list.

…t effects The action's version input defaults to the release pinned at the ref, not latest, and its annotations input (added in #1042) was documented nowhere. The checksum tip gave the wrong reason for opting out and hid that the default is lenient: unset verifies and warns when verification is impossible, an explicit true aborts instead, false skips. The action always passes the variable, so its default is the strict mode. init also writes .github/workflows/tests.yml and creates or edits .env, commenting out an existing BASHUNIT_BOOTSTRAP line. Both quickstart and the command-line page said it only creates two files. Requirements now lists the optional tooling that silently changes behaviour, most importantly jq turning every JSON assertion into a skip. The pipeline tip pointed at tests.yml, which does not use the action at all; test-action.yml does. Adds an Updating section for bashunit upgrade, which shipped undocumented on this page. The nine-line WSL preamble was pasted four times; Requirements now carries it once. Quickstart's sample output showed a ./ the command did not pass and a duration column that only appears when the clock is cheap, and it never showed how to control a run. index.md's feature cards predated parallel runs, coverage, tags, sharding, --changed and every report format. README claimed 77 assertions; there are 84.

The audit found 17 settings missing from docs/configuration.md, 19 from .env.example and two flags missing from --help. Reviews do not catch that reliably, so it is a test now. The checks compare sets of names and never prose, so a rewording cannot break them. Mutation-tested: adding an unregistered setting or an undocumented flag fails three of the six checks.

bashunit::assert_that returns 1 on failure so it can be chained, which means a custom assertion ending with it makes the whole test body exit 1. The runner classified any non-zero exit as an execution error, so one failure was reported twice: the ✗ Failed the assertion printed, then a spurious ✗ Error whose Output blob repeated the same text. The Error branch now requires the exit status to be unexplained: a recorded assertion failure accounts for it. A real runtime error still wins, because it sets runtime_error regardless of the counts, and there is a test for exactly that case. This removes the trailing 'return 0' the docs had to teach for every bashunit::assert_that example.

The settings-parity contract added in #1063 requires every registered BASHUNIT_* setting to have a configuration.md section.

Chemaclass · 2026-08-11T13:00:48Z

🤔 Background

An audit of the docs against src/ found the reference pages several releases behind the code, and two defects in the code the docs described. Every finding was reproduced by running the real binary, not by reading source.

💡 Changes

Corrections where the docs were wrong: the assert_equals examples were inverted (the documented test_success failed), assert_true/assert_false summaries said the opposite of the code, BASHUNIT_DEFAULT_PATH was documented as empty, an empty BASHUNIT_COVERAGE_REPORT was documented as disabling the file, and custom-asserts told readers to call a function that does not exist.
Gaps closed: 17 settings missing from configuration.md, 19 from .env.example, the assert subcommand had no section, and four provider gotchas were undocumented.
Two code fixes the audit surfaced: --coverage-report with no value aborted the run with $2: unbound variable, and --list-format json split the tag list on whitespace where every other consumer splits on commas.
Six parity tests so this cannot rot silently again, comparing sets of names rather than prose. Mutation-tested.
Deduplication: the coverage settings and the diff-coverage narrative existed twice and had already drifted; each now has one owner.