Skip to content

feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901

Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related

Description

@Chemaclass
Chemaclass
opened on Jul 27, 2026

Context

When a snapshot file is missing, assert_match_snapshot writes it and counts a
snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which
a first-time recording is treated as a failure.

Evidence

$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0

docs/ai-agents.md:67 already notes the underlying problem: exit code alone does
not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.

Proposal

Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a
missing snapshot fails the test instead of being created:

✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag
is probably cleaner. Worth deciding in discussion.

Pairs naturally with #900: one flag records deliberately, the other forbids
recording by accident.

Why this matters for agentic coding

The failure mode is a green run that asserted nothing. If a snapshot is
gitignored, never committed, or lost, CI records it on the fly and passes — and an
agent reading that green run reports the behaviour as verified. Agents are
especially exposed here because they add snapshot tests in bulk and trust the exit
code as their signal that work is done; a mode that fails on a missing snapshot is
what makes that signal trustworthy.

Acceptance criteria

  • A missing snapshot fails the test under the new mode, with the resolved path
    in the message
  • Default behaviour (record and pass) unchanged
  • Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  • Env-var equivalent, documented in docs/configuration.md
  • Recommended in docs/snapshots.md as the CI setting
  • Acceptance tests in tests/acceptance/
  • Bash 3.0+ compatible, make sa + make lint green

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestpure testingPure testing related

Type

No type

Projects

bashunit
Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded · Issue #901 · TypedDevs/bashunit · GitHub
Skip to content
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded #901
Closed
Closed
feat(snapshot): add a mode where a missing snapshot fails instead of being recorded#901
Assignees
Chemaclass
Labels
enhancementNew feature or requestpure testingPure testing related
Description
@Chemaclass
Chemaclass
opened on Jul 27, 2026
Context
When a snapshot file is missing, assert_match_snapshot writes it and counts a

snapshot assertion (src/assert_snapshot.sh:24-27,
bashunit::snapshot::initialize). The run exits 0. There is no mode in which

a first-time recording is treated as a failure.
Evidence
$ cat snap_test.sh
functiontest_new_snapshot_is_silently_recorded() {
assert_match_snapshot "hello world"
}
$ rm -rf snapshots && ./bashunit snap_test.sh >/dev/null 2>&1;echo$?
0
docs/ai-agents.md:67 already notes the underlying problem: exit code alone does

not distinguish a run that only recorded new snapshots. But nothing lets a caller
make that state fail.
Proposal
Add a way to require that every snapshot already exists — a --snapshot-ci /
--no-snapshot-create flag, or fold it into the existing --strict mode, so a

missing snapshot fails the test instead of being created:
✗ Failed: New snapshot is silently recorded
Snapshot does not exist: snapshots/snap_test_sh.test_....snapshot
Run without --snapshot-ci to record it.

Folding it into --strict is the smaller surface, but --strict currently means
set -euo pipefail for test bodies, which is a different axis — a separate flag

is probably cleaner. Worth deciding in discussion.
Pairs naturally with #900: one flag records deliberately, the other forbids

recording by accident.
Why this matters for agentic coding
The failure mode is a green run that asserted nothing. If a snapshot is

gitignored, never committed, or lost, CI records it on the fly and passes — and an

agent reading that green run reports the behaviour as verified. Agents are

especially exposed here because they add snapshot tests in bulk and trust the exit

code as their signal that work is done; a mode that fails on a missing snapshot is

what makes that signal trustworthy.
Acceptance criteria
  •  A missing snapshot fails the test under the new mode, with the resolved path
    
in the message
  •  Default behaviour (record and pass) unchanged
  •  Applies to both assert_match_snapshot and
    assert_match_snapshot_ignore_colors
  •  Env-var equivalent, documented in docs/configuration.md
  •  Recommended in docs/snapshots.md as the CI setting
  •  Acceptance tests in tests/acceptance/
  •  Bash 3.0+ compatible, make sa + make lint green
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestpure testingPure testing related
Type
No type
Projects
bashunit
Status
Done
Milestone
No milestone
Relationships
None yet
Development
No branches or pull requests
Issue actions