Problem
The JUnit XML report (src/reports/junit.sh:16) is minimal enough that CI tools cannot use it well.
Reading generate_junit_xml:
- One flat
<testsuite name="bashunit"> wrapping every test in the run. Jenkins, GitLab, Azure DevOps, Buildkite and the dorny/test-reporter action all group results by suite; every bashunit run collapses into a single undifferentiated bucket. - No
classname attribute on <testcase>. This is the attribute most consumers group and de-duplicate by; without it they show blank package names or fall back to the file path inconsistently. - No
<system-out> / <system-err>. The test's own output is captured by the runner and shown on the console, but the XML carries only the failure message, so the CI failure view loses the context. <failure message="Test failed"> is a constant string — the real message is in the element body, but tools that show only @message display "Test failed" for every failure.- No
timestamp, hostname, errors breakdown, or <properties>.
Proposal
- One
<testsuite>per test file, nested in <testsuites>, each with its own tests / failures / skipped / time / timestamp / name (the file path). classname on every <testcase>, derived from the test file path (e.g. tests/unit/assert/core_test.sh → tests.unit.assert.core_test), with name staying the human-readable test name.<failure message="<first line of the real message>" type="AssertionFailed"> with the full message in the body.<system-out> with the test's captured stdout when there is any, <system-err> for stderr.- Aggregate totals on the outer
<testsuites> element.
Keep backwards compatibility in mind: state in the CHANGELOG that the XML shape changed.
Where to change
src/reports/junit.sh:16generate_junit_xml.src/reports/collect.sh:54add_test — check whether the captured output is already retained; if not, this issue includes retaining it for the report path only (the console already renders it).
Acceptance criteria
Repo checklist (agent)
- TDD: RED → GREEN → REFACTOR. Write the failing test first.
- Bash 3.0+ only: no
printf -v, no += append, no declare -A, no [[ ]], no ${var,,}, no &>>, no ${arr[-1]}. Expanding a possibly-empty array under set -u needs ${arr[@]+"${arr[@]}"}. - A new CLI flag must be wired in all of these or a parity test fails:
- Gates:
make sa, make lint, ./bashunit tests/, ./bashunit --parallel tests/. Never run shfmt -w. - Docs: update
docs/command-line.md. Editing docs/assertions.md invalidates the bashunit doc acceptance snapshot — regenerate it. - CHANGELOG.md: add one line under
## Unreleased. - Fixtures under
tests/acceptance/fixtures/ must not end in *test.sh. - One issue = one PR.
Problem
The JUnit XML report (
src/reports/junit.sh:16) is minimal enough that CI tools cannot use it well.Reading
generate_junit_xml:<testsuite name="bashunit">wrapping every test in the run. Jenkins, GitLab, Azure DevOps, Buildkite and thedorny/test-reporteraction all group results by suite; every bashunit run collapses into a single undifferentiated bucket.classnameattribute on<testcase>. This is the attribute most consumers group and de-duplicate by; without it they show blank package names or fall back to the file path inconsistently.<system-out>/<system-err>. The test's own output is captured by the runner and shown on the console, but the XML carries only the failure message, so the CI failure view loses the context.<failure message="Test failed">is a constant string — the real message is in the element body, but tools that show only@messagedisplay "Test failed" for every failure.timestamp,hostname,errorsbreakdown, or<properties>.Proposal
<testsuite>per test file, nested in<testsuites>, each with its owntests/failures/skipped/time/timestamp/name(the file path).classnameon every<testcase>, derived from the test file path (e.g.tests/unit/assert/core_test.sh→tests.unit.assert.core_test), withnamestaying the human-readable test name.<failure message="<first line of the real message>" type="AssertionFailed">with the full message in the body.<system-out>with the test's captured stdout when there is any,<system-err>for stderr.<testsuites>element.Keep backwards compatibility in mind: state in the CHANGELOG that the XML shape changed.
Where to change
src/reports/junit.sh:16generate_junit_xml.src/reports/collect.sh:54add_test— check whether the captured output is already retained; if not, this issue includes retaining it for the report path only (the console already renders it).Acceptance criteria
dorny/test-reporter(or a documented equivalent)<testsuite>per test file, with correct per-file counts and times<testcase>hasclassname,name,file,time<failure message=…>carries the real message, body carries the full text<system-out>present when the test produced output, absent when it did not__xml_escape,src/reports/junit.sh:6)<testsuites>match the summary counters--parallel(regression guard for fix(reports): every report format is empty under --parallel #1004)env LC_ALL=C awkworkaround (7 test failures on Bash 5.3 macOS (nix-shell) #912) is preservedtests/unit/reports/asserting the XML structureRepo checklist (agent)
printf -v, no+=append, nodeclare -A, no[[ ]], no${var,,}, no&>>, no${arr[-1]}. Expanding a possibly-empty array underset -uneeds${arr[@]+"${arr[@]}"}.src/main/test.sh(report-style flags needexport -n, seesrc/main/test.sh:188-196for why)bashunit::main::validate_config_or_exit(src/main/validate.sh:60) — unvalidated input used to run the wrong thing and exit 0 (Unknown options are silently ignored: a typo'd flag runs a different suite and exits 0 #871, --jobs with a non-integer value hangs on Bash 3.x and is silently ignored on Bash 4.3+ #873)src/config/env.shand a documented line in.env.example--helptext in the same block it belongs tocompletions/bashunit.bashandcompletions/_bashunit(anti-drift test feat(cli): bash and zsh completion scripts with an anti-drift test #778 fails otherwise)make sa,make lint,./bashunit tests/,./bashunit --parallel tests/. Never runshfmt -w.docs/command-line.md. Editingdocs/assertions.mdinvalidates thebashunit docacceptance snapshot — regenerate it.## Unreleased.tests/acceptance/fixtures/must not end in*test.sh.