Uh oh!
There was an error while loading. Please reload this page.
fix(ci): fail closed when the staleness dedup search errors (backend#2702) - #884
Conversation
…2702) alert-workflow-staleness.sh hid `gh search issues` / `gh issue view` failures with `2>/dev/null` and read the resulting empty output as "not yet tracked", so a transient search error filed a fresh duplicate `work-type:bug` per stale workflow on every daily cron run until search recovered. Same class as backend#2631's filer-dedup gap (a separate repo, no shared helper). Capture gh's exit status instead of swallowing it: a failed search — or a failed candidate read — now aborts loud (exit 2, temp body cleaned up) rather than proceeding to create. A genuine zero-result (gh exits 0, empty stdout) still green-lights the create, so healthy dedup is unchanged. Adds an offline `gh` stub to the bats suite covering both fail-closed paths (search error, candidate-read error), the dedup-hit skip, and the genuine-empty files-anyway case. Updates the script header and docs/WORKFLOW-STALENESS.md exit-code contract to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saqlainsyed007
left a comment
There was a problem hiding this comment.
Approving at 9f24da26. The fix does exactly what it claims and fails closed on the right path.
alert-workflow-staleness.sh dedups before filing a work-type:bug issue by searching backend for the finding's hidden marker. The old code swallowed gh failures (gh search … 2>/dev/null, and gh issue view … 2>/dev/null | grep), so a transient API error produced empty output indistinguishable from a genuine zero-result — read as "not yet tracked" and filing a fresh duplicate every daily run until search recovered (backend#2702). The fix captures gh's exit status via if ! var="$(…)" and aborts exit 2 (cleaning up the temp body) on either a search or a candidate-read error, while a genuine empty result (gh exit 0) still green-lights the create. Splitting gh issue view from the grep (the old pipe masked gh's status under pipefail) is necessary and done correctly.
I checked the fail-closed tests are mutation-proof: against the reverted code both flip to exit 0 with gh issue create reached, so they redden if the guard is deleted — not vacuous — and the happy-path + dedup-skip tests pin the create/skip boundary. It's the sole dedup call site in the repo. CI green (bats 9/9, shellcheck, Bugbot), mergeable, no open threads. LGTM.
One optional, non-blocking follow-up (pre-existing, out of this diff): the dedup-skip branch around :147continues without rm -f "$bodyfile", unlike the other loop-exit paths, so it leaks one mktemp file per already-tracked workflow per run. Worth a tidy sometime; not a reason to hold this.
Uh oh!
There was an error while loading. Please reload this page.
Closes tracebloc/backend#2702
What
scripts/alert-workflow-staleness.shdeduplicates by searchingbackendfor an open issue already carrying the finding's marker, and files a newwork-type:bugonly when none is found. The search hid its own failures:A
gh search/gh issue viewerror produced empty output, indistinguishable from a genuine zero-result — so a transient search failure was read as "not yet tracked" and filed a duplicate. On the daily cron that is one new card per stale workflow per day until search recovers (Bugbot Medium, client#876 thread).Verified the finding still stands on current
develop(#876 shipped the ticket, not a fix). Same class as backend#2631's filer-dedup gap — that one lives in therelease-trainrepo (promote-repo.sh/file-hop-findings.py), a separate codebase with no shared helper, so nothing to fix centrally here;gh search issueshas a single call site in this repo.Fix — fail closed
Capture gh's exit status instead of swallowing it. A failed search — or a failed candidate read — now aborts loud (
exit 2, temp body cleaned up) rather than proceeding to create. A genuine zero-result (gh exits0, empty stdout) still green-lights the create, so healthy dedup is unchanged.exit 2matches the script's existing create-failure path and turns the (un-continue-on-error) watcher step red — the documented "fails loud" contract.I chose abort over retry: a
gherror at this point is systemic (auth / rate-limit / outage), the daily run re-attempts tomorrow, and a red watcher is the intended signal.Tests
Adds an offline
ghstub toscripts/tests/alert-workflow-staleness.batsexercising the real (non-dry-run) path:issue createissue createEvidence
Header comment and
docs/WORKFLOW-STALENESS.mdexit-code contract updated to match.🤖 Generated with Claude Code
Note
Low Risk
CI alerting script and tests only; behavior is stricter on API errors (no duplicate issues) without changing successful dedup or filing paths.
Overview
Fixes duplicate tracking issues when
gh search issuesorgh issue viewfails during dedup: those errors used to look like “no open issue,” so the daily watcher could file a new card every day until search recovered (backend#2702).alert-workflow-staleness.shnow checks exit status on dedup search and candidate body reads, aborts with exit 2 (and skips create), and documents the fail-closed rule in the script header anddocs/WORKFLOW-STALENESS.md. A successful search with zero hits still allows filing.Tests add a configurable
ghstub and four bats cases covering search failure, view failure, happy-path dedup skip, and empty-search still creates.Reviewed by Cursor Bugbot for commit 9f24da2. Bugbot is set up for automated code reviews on this repo. Configure here.