docs(rag): document the eval-canary pair protocol's dispatch-only trigger and two canary-reading lessons

docs/branch-review-records/65462d1d993e393eb03e0fc2cefc1b6b50174027a14f1831edb0d266829567aa.record.md

@@ -0,0 +1 @@
+| 2026-08-18 | claude/eval-canary-protocol-docs-01bb49 | 3e9905d0ac9d88f22dc88d41639399ae69237cf0 | docs/rag-behaviour/safeguards.md, docs/rag-improvement/README.md — eval-canary pair protocol trigger mechanics and bisection lessons (ledger #TYJ0XP) | approved — docs-only, no code/behaviour change | verify:pr-local (docs-scoped: format:changed, sitemap:check, docs:check-index, docs:check-inventory, docs:check-scripts, docs:check-links, check:branch-review-ledger, check:outstanding-issues, check:ledger-write-discipline) all passed |

docs/outstanding-issues-inbox/5899581b-dc23-486d-a6da-e456c5c5f3bc.json

@@ -0,0 +1,11 @@
+{
+ "version": 2,
+ "id": "5899581b-dc23-486d-a6da-e456c5c5f3bc",
+ "createdOn": "2026-08-18",
+ "action": "done",
+ "payload": {
+ "id": "#TYJ0XP",
+ "outcome": "Recorded in docs/rag-behaviour/safeguards.md (eval-canary dispatch/cron-only trigger, gh api dispatch command, denominator caveat, S1b/S1d + #2065/#2088 lessons) and docs/rag-improvement/README.md §5/§6 cross-links; PR opened from claude/eval-canary-protocol-docs-01bb49 at 3e9905d0a.",
+ "baseRowFingerprint": "27208bbb65830782e093bb6677d42aa887b9bf46066d810a83eee48af67138c2"
+ }
+}

docs/rag-behaviour/safeguards.md

@@ -63,24 +63,46 @@ keys above relevance, read this folder first. **Also:** do not add `review_due`
 `unknownCurrentness` / governance-metadata ranking penalties or boosts — that shape is
 refuted (`refuted-approaches.md` § Refutation 3; ledger `#032`). The eval-canary pair protocol:
 
-1. Baseline: latest green canary on current main (or one dispatch).
-2. Change merges to main. (Post runs are post-merge only: the workflow accepts only the
- `eval-canary` repository dispatch and always loads code from the default branch — no
- `workflow_dispatch`, no `ref` input; both are asserted absent by
- `tests/eval-canary-workflow.test.ts`.)
-3. Post: one dispatch; gates = recall 1.0/1.0, zero per-case rr regressions. Compare the
- pair's `--json-out` artifacts — the command diffs per-case `rr@10`/`contentRR@10` and
- exits non-zero on any per-case regression, a non-identical case set, or an unavailable
- rank metric:
+`.github/workflows/eval-canary.yml` fires only on the `eval-canary` repository dispatch and the
+Sunday 18:00 UTC cron (`schedule: cron: "0 18 * * 0"`) — it has no `workflow_dispatch` and no
+`ref` input, both asserted absent by `tests/eval-canary-workflow.test.ts`. Every run, scheduled
+or dispatched, always loads code from the default branch, so a canary can only ever measure
+`main` — there is no way to canary an open branch or PR head directly.
+
+1. Baseline: the latest green canary on current `main` **before the merge** (a scheduled run or
+ a prior dispatch).
+2. Change merges to `main`.
+3. Post: one repository dispatch **after the merge** —
+ `gh api repos/BigSimmo/Database/dispatches -f event_type=eval-canary` — then compare the
+ pair's `--json-out` artifacts, both named `eval-canary-output`, one per run:
 
  ```bash
  npm run eval:retrieval:compare -- <baseline.json> <post.json> --fail-on-regression
  ```
 
+ The command diffs per-case `rr@10`/`contentRR@10` and exits non-zero on any per-case
+ regression, a non-identical case set, or an unavailable rank metric. Gates: document/content
+ recall 1.0/1.0, zero per-case reciprocal-rank regressions, and the answer-quality gate clean
+ — read the denominator from the run's own report rather than assuming a fixed case count; it
+ has moved as `rag-eval-cases.ts` grew.
+
 4. Regression → immediate single-commit revert + one confirmation dispatch.
 
 Provider-backed dispatches always need explicit user approval (~$1–2 each).
 
+**Reading a canary result:**
+
+- A red first post-run may be an unrelated non-golden case, not a diff regression — root-cause
+ with a live probe before blaming the diff. Precedent: the first post-S1b canary (run
+ `32038751592`) failed one answer-quality case; a re-run (`32039841070`) plus cache-bypassed
+ live probes showed the fast model's phrasing had taken a different, still-correct branch, not
+ a retrieval or code fault — retrieval stayed 36/36 identical across both runs.
+- A deterministic extractive-path flip is a real regression — bisect by probe and revert in a
+ single commit. Precedent: interim post run `32097916649` went red on
+ `agitation-im-po-route-short-terms`; live bisection isolated it to PR #2065 (S1c's
+ condition-first `for`/`in` binding regex), which was reverted by PR #2088, with confirmation
+ run `32100681177` restoring green.
+
 ## Rollback proof (2026-07-20)
 
 The full cycle was exercised live: merge (#1004) → regression detected by canary #55 within

docs/rag-improvement/README.md

@@ -373,6 +373,10 @@ work packets, the live status table, checklists, and paste-ready prompts.
 | 9 | B5/B6 Ragas/reranker | conditional | No until separately promoted |
 | 10 | B7 DSPy | ≥100 labelled cases | No until separately promoted |
 
+Cross-links: `#231` cross-links to `docs/database-remediation-plan.md` Phase 5.2 (re-test after
+the trigram-index restore) — satisfied by S1's healthy-latency probes 2026-08-17; `#316` Phase
+1.2 found the RPC divergence attribute-only.
+
 ## 6. Verification commands (per PR, smallest first)
 
 ```bash
@@ -387,7 +391,9 @@ npm run check:production-readiness # domain changes (env flags, answer path)
 ```
 
 Live canary pairs fire only via the `eval-canary` repository dispatch with explicit owner
-approval per run; regression → single-commit revert + confirmation run.
+approval per run; regression → single-commit revert + confirmation run. See
+`docs/rag-behaviour/safeguards.md` § the eval-canary pair protocol for the exact trigger
+mechanics (dispatch/cron-only, no `ref` input) and the comparison command.
 
 ## 7. Rollback map