Prevent generation fallbacks from sticking in answer cache

[BigSimmo]

Summary

• reject every provider-generation fallback from rag_response_cache, including grounded extractive recovery answers, so a transient fallback cannot remain sticky after provider recovery
• centralize answer-cache eligibility on the finalized answer and preserve the existing conservative source-backed fallback behavior
• update ledger item fix(ci): bump checkout/setup-node/cache actions to v5 #231 with the completed cache fix and the live-proven residual generation-quality investigation

RAG impact: behaviour change — canary pair local-baseline-fb54b4dc3 -> local-final-0c96538c8; retrieval remained 36/36 with document/content recall 1.0, identical MRR/content-MRR, and zero per-case reciprocal-rank regressions. Final 44-case answer quality retained grounded-supported 1.0, citation failure 0, and numeric-grounding failure 0.

Verification

• node scripts/run-vitest.mjs run tests/rag-route-budget.test.ts tests/rag-answer-fallback.test.ts — 96 passed
• npm run eval:rag:offline — 23 files, 574 passed; fixture/production contracts passed
• npm run check:rag:fixtures — 36 golden cases and 23 suites passed
• npm run typecheck — passed on the exact stacked branch
• approved live baseline: 36/36 retrieval, document/content recall 1.0, MRR 0.8921, content MRR 0.9406; 44/44 answer cases passed, citation/numeric-grounding failures 0
• approved live final: 36/36 retrieval, document/content recall 1.0, MRR 0.8921, content MRR 0.9406, zero per-case RR regressions; 44/44 answer cases passed, citation/numeric-grounding failures 0, estimated answer-eval cost $0.022094
• npm run check:production-readiness — source/config gate was run in the secret-free isolated worktree and correctly reported missing local env; the live canaries instead loaded the designated primary .env.local in process memory and succeeded

UI verification not run: no UI, routing, styling, or browser behavior changed.

Risk and rollout

• Risk: cache hit rate can be lower when generation degrades, intentionally trading transient recomputation for avoiding stale degraded clinical answers.
• Rollback: revert this PR to restore prior fallback caching.
• Provider or production effects: explicitly approved live OpenAI/Supabase evals against the public clinical corpus only; no writes or deployments.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Partial resolution of fix(ci): bump checkout/setup-node/cache actions to v5 #231: stale generation fallbacks are fixed. Live Lithium dosing? probes showed the remaining fallback is generation_quality_failed with route_deadline_exceeded=false; timeout inflation was tested, rejected, and not shipped.
• Stacked on Fix PR body sync scope and root-portable handoff test #1830; review this PR by comparing against codex/pr-workflow-safety-230-296.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 34551d48-db22-49e5-80c5-9495daf4c656

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:0c96538c84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[BigSimmo]

@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. This is the pull request's single automatic repair pass: do not perform a fresh review, create new standalone findings, or request another review. Work only the existing unresolved Codex threads on the current head. The only repository destination is BigSimmo/Database, and the only branch destination is the pull request head branch codex/rag-fallback-cache-231 at starting commit 0c96538; never publish fixes to a detached or synthetic work branch and never create a stacked pull request. Use the authenticated GitHub connector to commit each approved fix to BigSimmo/Database:codex/rag-fallback-cache-231, then verify that the pull request head contains the published commit before reporting success. Always fix P0 and P1 findings. For P2 and lower findings, fix only clear, scoped, low-risk issues; otherwise disposition them with a concise reason. For a fixed thread, reply with as the first line and as the second line. For a no-code disposition, use followed by . These result markers authorize the workflow to close that exact thread only after it verifies a fixed commit is the pull request head; a local-only commit is not a fix. If publication or verification fails, do not use either result marker, do not claim success, and leave the thread open with the blocker. If human input or new authorization is required, do the same. Finish only after every actionable thread is fixed or dispositioned and closed, or explicitly left open for a human decision. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Swish!

Reviewed commit:0c96538c84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[BigSimmo]

Babysit PR summary

• Tip: codex/rag-fallback-cache-231 at 943cff937c92a86a8e59dc069f01c83f30adb2fb
• Base: codex/pr-workflow-safety-230-296
• Sync/merge-tree: fetched remote tip clean and pushed, then git merge-tree $(git merge-base HEAD origin/main) HEAD origin/main reports clean for origin/main
• Unresolved review threads: none (both prior threads PRRT_kwDOSh5Fis6YbO6o and PRRT_kwDOSh5Fis6YbQnf remain resolved)
• Fixed: src/lib/rag/rag-route-budget.ts, src/lib/rag/rag-cache.ts, tests/rag-route-budget.test.ts, tests/rag-shared-cache.test.ts
• Fixes:
  • Treat exact generation_fallback as cache-bad as well as prefixed fallback reasons
  • Reject stale shared-cache generation-fallback hits and delete them to avoid promotion/reuse
  • Added regression coverage for exact fallback reason and shared-cache deletion
• Required checks on tip:
  • GitGuardian Security Checks: PASS
  • Supabase Preview: SKIPPING (as reported by gh pr checks)
• Local decisive proof on current head:
  • npm exec eslint src/lib/rag/rag-route-budget.ts src/lib/rag/rag-cache.ts tests/rag-route-budget.test.ts tests/rag-shared-cache.test.ts
  • npm exec vitest run tests/rag-route-budget.test.ts tests/rag-shared-cache.test.ts
  • npm run ledger:append -- --ref codex/rag-fallback-cache-231 --head 943cff937c92a86a8e59dc069f01c83f30adb2fb --scope babysit-pr --outcome fixed ...
• Residual risks:
  • No remaining actionable findings on this head; merge left to user.
  • Any additional required checks outside this PR’s scope are not locally runnable from this worktree snapshot.

[BigSimmo]

Babysit review (scope: exact babysit-pr)

Snapshot (current GitHub head): 943cff937c92a86a8e59dc069f01c83f30adb2fb on branch codex/rag-fallback-cache-231.

• Merge state snapshot: mergeStateStatus=CLEAN, mergeable=true.
• Local mergeability check: merge-tree against origin/main is clean (no conflicts).
• Threads: no unresolved actionable review threads.
• Fixes made: none (only pre-existing type issue was already corrected before this review sweep).
• Local verification:
  • npm exec eslint src/lib/rag/rag-route-budget.ts src/lib/rag/rag-cache.ts tests/rag-route-budget.test.ts tests/rag-shared-cache.test.ts ✅
  • npm exec vitest run tests/rag-route-budget.test.ts tests/rag-shared-cache.test.ts ✅
  • npm run format -- --check tests/rag-route-budget.test.ts tests/rag-shared-cache.test.ts ✅
• Wider checks: npm exec vitest run tests/ci-cache-safety.test.ts tests/pr-handoff-stop.test.ts has pr-handoff-stop failures only from Windows environment tooling (ln/rmSync permissions), matching prior local baseline; npm exec vitest run tests/ci-cache-safety.test.ts tests/pr-handoff-stop.test.ts also reports spawnSync ln ENOENT / EPERM errors.
• Required checks from remote snapshot were incomplete in this local view (only baseline security/supabase entries shown here).

Outcome: dispositioned; merge left to you.

[BigSimmo]

Codex babysit snapshot (scope babysit)

• PR number/ref: 1831 from codex/rag-fallback-cache-231
• Snapshot head SHA: 943cff937c92a86a8e59dc069f01c83f30adb2fb
• Base ref: codex/pr-workflow-safety-230-296
• Fork relation at snapshot: 4 commits ahead of base, 0 commits behind.
• Mergeability: mergeable=MERGEABLE, mergeStateStatus=CLEAN.
• Required checks observed on this PR head: GitGuardian Security Checks pass, Supabase Preview skipped.
• High-confidence defects: none identified during focused review.
• Thread state: no unresolved review threads; actionable Codex/Copilot concerns were previously dispositioned on this PR’s tip.
• Proof checks run: vitest targeted on tests/rag-route-budget.test.ts and tests/rag-shared-cache.test.ts (both pass).
• Ledger: appended babysit outcome clean for this exact head.

No merge blockers are currently surfaced on this branch review sweep.

[BigSimmo]

Babysit review summary

• Snapshot: reviewed at GitHub head 943cff9 (branch codex/rag-fallback-cache-231, base codex/pr-workflow-safety-230-296).
• Merge state: CLEAN; mergeable MERGEABLE.
• Merge-tree: checked against base chain and no hard conflict at reviewed scope; merge-tree comparison clean for changed paths.
• Threads: no unresolved actionable threads (2 total, all resolved).
• Required checks: GitGuardian Security Checks success, Supabase Preview skipped.
• Disposition: clean review; no blockers; no fix required.
• Local proof: reviewed diff and inspected changed files with git show; no local �itest/typed/gating rerun due missing dependencies in this worktree (
  ode_modules absent).

No pushes made (no local modifications for this PR).