Problem
Each check in docs_parity_test.sh is a one-directional comm:
missing="$(comm -23 <(_registered_settings)<(_documented_in_env_example))"
assert_empty "$missing"
If _registered_settings stops matching — env.sh renamed, or the registration
idiom changes and the regex rots — it yields nothing, comm -23 yields
nothing, and "nothing is missing" holds. The guard that catches an
undocumented new setting passes precisely when it has stopped working.
Proven by mutation: pointing ENV_FILE at a missing file leaves both
"is documented in …" tests green. The file does go red, but only via the
inverse-direction tests, whose message ("env example lists no setting that does
not exist") points away from the real fault.
Fix
One test asserting each extractor still finds a plausible number of names.
Floors are set far below the current counts (79 settings, 20+ flags) because
these only grow — they exist to catch an extractor returning nothing, not to
track an exact number.
With it, the same mutation fails a test that names the actual problem.
Problem
Each check in
docs_parity_test.shis a one-directionalcomm:If
_registered_settingsstops matching — env.sh renamed, or the registrationidiom changes and the regex rots — it yields nothing,
comm -23yieldsnothing, and "nothing is missing" holds. The guard that catches an
undocumented new setting passes precisely when it has stopped working.
Proven by mutation: pointing
ENV_FILEat a missing file leaves both"is documented in …" tests green. The file does go red, but only via the
inverse-direction tests, whose message ("env example lists no setting that does
not exist") points away from the real fault.
Fix
One test asserting each extractor still finds a plausible number of names.
Floors are set far below the current counts (79 settings, 20+ flags) because
these only grow — they exist to catch an extractor returning nothing, not to
track an exact number.
With it, the same mutation fails a test that names the actual problem.