fix(canary): distinguish a goldens harness error from real drift

[LukasWodka]

What

Fixes the Cursor Bugbot "Canary treats script failures as drift" (Medium) finding, surfaced on the #323 develop→main promotion.

head-drift-canary.yml set drift=true on any non-zero exit from scripts/sync-validator-goldens.sh --check. But that exit also fires on harness failures — missing deps, a bad DATA_INGESTORS_DIR, a generator crash — so the tracking issue reported "DRIFT — verdicts/values changed" even when parity was never evaluated, sending someone down the pin-bump playbook to chase an infra bug.

Fix

sync-validator-goldens.sh --check now uses reserved exit codes so consumers can distinguish the cases:

code	meaning
0	in sync
2	harness error — could not regenerate (deps / DATA_INGESTORS_DIR / generator crash); parity not evaluated
3	drift — the ingestor's verdicts or read-path values changed

The required goldens-drift check is unaffected — it reds on any non-zero, and both 2 and 3 are non-zero.

The canary maps them:

• exit 3 → drift=true → advisory tracking issue (run stays green, as before).
• exit 2 / unexpected → evaluated=false → reported as "could not evaluate — harness error" with a note to fix the harness rather than bump the pin.
• "All clear" now also requires evaluated=true, so a harness error can't be silently reported as all-clear.

Verification

• --check with no data-ingestors → exits 2, prints HARNESS ERROR, and does not mutate goldens.json
• actionlint clean, shellcheck clean, YAML valid

Once merged to develop, #323 inherits it and Bugbot re-clears.

---

Note

Low Risk
Advisory CI reporting and shell script exit semantics only; blocking goldens-drift still fails on any non-zero exit.

Overview
The HEAD-drift canary no longer treats every non-zero exit from sync-validator-goldens.sh --check as goldens drift. --check now documents and uses reserved exit codes: 0 in sync, 2 harness failure (generator/deps/DATA_INGESTORS_DIR — parity never ran), 3 real verdict/value drift. Generator failures exit 2 after restoring goldens.json instead of being lumped with drift.

head-drift-canary.yml maps those codes: exit 3 → drift=true; exit 2 or other failures → evaluated=false with messaging that parity was not checked. Tracking issues distinguish harness vs drift in the table and body, use pin_actionable so harness-only failures do not show the pin-bump playbook, refresh issue title on update, and require evaluated=true for the "All clear" step.

The blocking goldens-drift job still fails on any non-zero exit (2 or 3); behavior there is unchanged.

^{Reviewed by Cursor Bugbot for commit 69ed892. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

@BugBot run

[LukasWodka]

👋 Heads-up — Code review queue is at 47 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• averaging-service#182 — feat(weights): averaging writes SafeTensors (TF + PyTorch) — SEC-03 §8 step 4 · author: @shujaatTracebloc · reviewer: @divyasinghds
• averaging-service#183 — docs: SafeTensors deployment-order & resume-safety runbook (epic feat(auth): logout revokes the token server-side (cli#112) #116) · author: @shujaatTracebloc · reviewer: @saadqbal
• averaging-service#184 — test(weights): shared SafeTensors golden-corpus drift guard + declare safetensors dep (SEC-03) · author: @LukasWodka · no reviewer assigned
• averaging-service#185 — test(weights): average a real SafeTensors edge + a legacy edge in one cycle — SEC-03 §8 · author: @LukasWodka · no reviewer assigned
• backend#1093 — chore(deps): bump django from 5.2.14 to 5.2.15 · author: @dependabot · no reviewer assigned
• backend#1108 — test(api): CLI response-shape contract fixtures for the 9 CLI endpoints (WS-D.2) · author: @LukasWodka · reviewer: @saadqbal
• backend#1111 — feat(download): edge weights fetch tolerates .safetensors (SEC-03 step 5c hardening) · author: @shujaatTracebloc · reviewer: @saadqbal
• backend#1112 — fix(runners): migrate deprecated trainingClasses/getTrainingPlan to snake_case (#1102) · author: @LukasWodka · reviewer: @saqlainsyed007
• backend#1114 — perf(leaderboard): serializer prefetch reads + TeamSerializer is_* in Python (#978 pt2) · author: @aptracebloc · reviewer: @saqlainsyed007
• cli#323 — main -> enhance CI workflows · author: @saadqbal · reviewer: @LukasWodka

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

[LukasWodka]

@BugBot run

[LukasWodka]

@BugBot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 69ed892. Configure here.}