fix(rag): stop caching soft-tail unsupported-short-circuit zero results

[BigSimmo]

Summary

• Stop caching a zero result from the RAG soft-tail unsupported short-circuit in searchChunksWithTelemetry (src/lib/rag/rag.ts). The soft-tail bucket (low confidence, few expanded terms, no deterministic exclusion match) can be resolved by a nondeterministic LLM classifier call, and caching that zero made a single unlucky classification sticky for every later caller within the cache TTL. The three deterministic exclusion patterns ahead of it in shouldShortCircuitUnsupportedSearch are stable true negatives and stay cached exactly as before — this only changes the caching decision for the fragile bucket, identified via the already-exported isUnsupportedSoftTailAnalysis.
• Shorten the memo TTL for a rejected classifier verdict (still unsupported_or_general / confidence < 0.58) from 15 minutes to 60 seconds, for the same soft-tail bucket only. analyzeQueryWithClassifierFallback normally memoizes every verdict — accepted or rejected — for 15 minutes so a query is deterministic within a session (Finding Improve Clinical KB dashboard and RAG hardening #11 interim fix, rag.ts:945-956). That 15-minute memo TTL is 15x the 60s search-cache TTL from the first bullet, and automated review on this PR correctly identified it as the actual dominant stickiness behind the "catatonia returns zero" false negative — the first bullet alone barely helps within that window. A rejected verdict for the soft-tail bucket now expires after 60s instead of 15 minutes, so a repeat query gets a fresh classifier attempt within a bounded window instead of reproducing the same rejection for the rest of the old TTL. An earlier version of this commit skipped the memo entirely for this bucket; a second review round (Devin) correctly flagged that as unbounded — every repeat would re-invoke the paid classifier forever and a borderline query could flip between zero and non-zero results on every single request — so this uses a short bounded TTL instead. Accepted verdicts, and rejected verdicts outside the soft-tail bucket, keep the original 15-minute determinism guarantee unchanged.
• Keep caching a deterministic "out_of_corpus" zero result (from classifyCorpusGrounding, reached without any LLM call) even though it is soft-tail-shaped — isUnsupportedSoftTailAnalysis only inspects query text and deterministic analysis, not the corpus-grounding verdict, so without this exclusion every repeat of a genuinely out-of-corpus query would redo the corpus-grounding and trigram-correction RPCs for an answer that never changes (second Devin finding).
• Only skip the cache write at all when a classifier was actually reachable (OPENAI_API_KEY present, matching the exact early-return condition in analyzeQueryWithClassifierFallback, rag.ts:1139). In source-only/offline deployments the classifier is never called, so the soft-tail outcome is fully deterministic and should cache like every other stable true negative — without this, every repeat in that deployment mode redid classifyCorpusGrounding (and, when not source-only, the trigram-correction RPC) for an answer that could never change (third Devin finding).
• Expose telemetry.corpus_grounding on the /api/search response (src/app/api/search/route.ts), including on shared-cache hits (src/lib/rag/rag-cache.ts). It was already computed internally on every request (rag.ts) but dropped by the route's hand-picked telemetry subset and by the shared-cache-hit telemetry reconstruction, which made this class of false negative undiagnosable without direct Supabase access.
• Tests: tests/rag-unsupported-short-circuit-cache.test.ts (soft-tail bucket not cached when a classifier was reachable; deterministic exclusion pattern still cached; out_of_corpus still cached even though soft-tail-shaped; soft-tail zero is cached when no classifier was ever reachable — no OPENAI_API_KEY — because that case is deterministic; in-corpus rescue never reaches the short circuit), tests/rag-shared-cache.test.ts (corpus_grounding survives a shared-cache hit), tests/rag-classifier-memo.test.ts (a rejected soft-tail verdict is still memoized within its 60s TTL — bounded, not unlimited retries — and the two-call proof: reject on call 1, recover on call 2 once the 60s TTL expires; existing rejected-verdict memoization outside the soft-tail bucket is unchanged).

RAG impact: behaviour change — the classifier-memo fix changes how often the same query can get a different classification within a session, for the soft-tail bucket only, bounded to once per 60s, and only in the direction of more classifier calls (never fewer): a query that was previously deterministically stuck at "unsupported" for 15 minutes can now recover after at most 60s; nothing that previously recovered can now fail instead, and no ranking/ordering of already-returned results changes. A live eval-canary pair (doc/content recall pinned 1.0, zero per-case rr regressions) is still owed per AGENTS.md before this is fully trusted in production — not run from this session (no live Supabase/OpenAI credentials available here; the canary runs as a GitHub Actions repository_dispatch against main with hosted secrets, so it needs to run post-merge or via an explicit dispatch you trigger). The cache-write and memo-TTL changes are pure caching/memoization-write decisions with no change to retrieval, ranking, or scoring logic itself — flagging the whole PR as a behaviour change to be conservative, since the net effect of "make one specific negative outcome less sticky" is still a real change to what a user sees on a repeat query within the (now much shorter) memo window.

Verification

• npm run verify:pr-local — Verification not run: the single aggregate command hits pre-existing environment drift in check:installed-lock-parity (playwright installed 1.62.0 vs locked 1.62.1), unrelated to this diff. Ran the equivalent gates individually instead, all green:
  • npm run check:runtime — PASS: Node runtime 24.13.0 matches required Node 24.x / PASS: npm runtime 11.17.0 matches required npm 11.x
  • npm run lint — clean (no output/errors)
  • npx tsc --noEmit -p tsconfig.json — clean (no output/errors)
  • npm run test — Test Files 514 passed (514) / Tests 5438 passed | 4 skipped (5442)
  • npm run eval:rag:offline — Offline RAG fixture and manifest validation passed (36 golden cases, 23 suites). then Test Files 23 passed (23) / Tests 574 passed (574), then Offline RAG fixture and production-contract checks passed.
  • npm run build — succeeded (exit 0); npm run check:bundle-budget — 1467.0 KiB gzip vs 1406.4 KiB baseline, within tolerance
  • npm run check:maintainability-budgets — src/lib/rag/rag.ts: 4351/4351 lines (passes at the exact no-growth budget — zero margin left in this file for future additions)
• npm run verify:ui — not run; no UI, routing, or styling changed.
• npm run verify:release — not run; not a release/handoff confidence claim.
• npm run eval:retrieval:quality — not run — this is the live canary owed above. Provider-backed, needs live Supabase/OpenAI keys not available in this session; needs your explicit approval and either a post-merge dispatch or a run from an environment with live credentials before this change is fully trusted per AGENTS.md.
• npm run eval:rag -- --limit 15 / npm run eval:quality -- --rag-only — not applicable; answer generation and the synthesis prompt are untouched.
• npm run check:production-readiness — not run; provider-backed, not requested for this change. No privacy, production-env, or source-governance policy changed.
• npm run check:deployment-readiness — not applicable; no deployment/startup/hosting behavior changed.

Risk and rollout

• Risk: Low-to-moderate. The search-cache change does not change what queries retrieve, rank, or return when the short circuit is not hit, and does not change which queries hit the short circuit at all — cache-write plumbing only. The classifier-memo change is scoped tightly (soft-tail bucket only, rejected verdicts only, bounded to a 60s TTL) and is directionally one-way: it can only make a previously-stuck query try again at most once per 60s, never make a previously-working query fail, so the plausible failure mode is "extra classifier calls for a genuinely unsupported query, capped at once per 60s," not a ranking/citation regression or unbounded cost. Worst case: slightly higher OpenAI classifier-call volume for repeat genuinely-out-of-corpus soft-tail queries within the old 15-minute window. The out_of_corpus/no-key caching fixes and corpus_grounding telemetry addition are all purely additive/restorative (no new negative-case behaviour).
• Rollback: revert the relevant commit(s) — each change is its own commit and independently revertible pre-merge.
• Provider or production effects: The classifier-memo change increases OpenAI classifier-call frequency for a narrow query bucket, bounded to at most once per 60s per query (cost impact, not correctness/safety). No Supabase schema, migration, or RLS change. corpus_grounding was already computed for every request internally — this only forwards an existing value into the response payload.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed — not applicable: this changes retry/caching behaviour around an existing "no results" refusal path, not clinical decision-support content or logic

Notes

• This addresses both stickiness layers behind the "catatonia returns zero" false negative reported 2026-08-06 (the 60s search cache and the 15-minute classifier-verdict memo). One root cause remains open, tracked as a follow-up (not bundled into this PR — it needs new database stats and its own eval-canary):
  • classifyCorpusGroundingFromStats (src/lib/corpus-grounding.ts) only credits a document title match as a topic anchor, so a genuinely in-corpus bare topic with no matching document title (e.g. "catatonia" — well represented in chunk content, but no document is titled "Catatonia") returns "inconclusive" rather than "in_corpus_topic", and still falls through to the (now less-sticky, but still nondeterministic) LLM classifier call. Fixing that needs new per-term content-level document-count stats (an RPC/migration change) to avoid over-crediting generic words as false topic anchors.
• Review history: Codex flagged a P1 (the classifier-memo stickiness, fixed) and a P2 (corpus_grounding dropped on shared-cache hits, fixed). Devin flagged a ledger-ordering false positive (investigated and dispositioned — the ledger's merge=ledger driver legitimately interleaves concurrent appends, not a hand-edit; check:branch-review-ledger passes clean), then three real findings across two review rounds on the cache/memo changes: unbounded re-invocation risk (fixed by adding the 60s TTL instead of skipping the memo entirely), the out_of_corpus caching gap (fixed), and the no-OPENAI_API_KEY/source-only caching gap (fixed).

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:57 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3acf1ac6-62f3-417f-beaf-ee0ccd1a6c40

📥 Commits

Reviewing files that changed from the base of the PR and between 624843f and 482a749.

📒 Files selected for processing (10)

• docs/branch-review-ledger.md
• scripts/check-maintainability-budgets.mjs
• src/app/api/search/route.ts
• src/lib/rag/rag-cache.ts
• src/lib/rag/rag-query-guard.ts
• src/lib/rag/rag.ts
• tests/rag-classifier-memo.test.ts
• tests/rag-query-guard-soft-tail-cache.test.ts
• tests/rag-shared-cache.test.ts
• tests/rag-unsupported-short-circuit-cache.test.ts

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

[BigSimmo]

@codex resolve actionable Codex review findings for this pull request and current head using the repository instructions. This is the pull request's single automatic repair pass: do not perform a fresh review, create new standalone findings, or request another review. Work only the existing unresolved Codex threads on the current head. The only repository destination is BigSimmo/Database, and the only branch destination is the pull request head branch claude/implement-97vpz7 at starting commit 7854fab; never publish fixes to a detached or synthetic work branch and never create a stacked pull request. Use the authenticated GitHub connector to commit each approved fix to BigSimmo/Database:claude/implement-97vpz7, then verify that the pull request head contains the published commit before reporting success. Always fix P0 and P1 findings. For P2 and lower findings, fix only clear, scoped, low-risk issues; otherwise disposition them with a concise reason. For a fixed thread, reply with as the first line and as the second line. For a no-code disposition, use followed by . These result markers authorize the workflow to close that exact thread only after it verifies a fixed commit is the pull request head; a local-only commit is not a fix. If publication or verification fails, do not use either result marker, do not claim success, and leave the thread open with the blocker. If human input or new authorization is required, do the same. Finish only after every actionable thread is fixed or dispositioned and closed, or explicitly left open for a human decision. Do not update the branch from main, address unrelated reviews, broaden scope, or create more than one scoped fix commit. Do not use external APIs, paid services, credentials, dependency changes, or broad refactors unless explicitly authorized. Add targeted tests where behavior changes and run the narrowest relevant validation.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit:7854fabbc7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

CI triage

CI failed on this PR. Automated classification of the 2 failed job(s):

• Unit coverage — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
• PR required — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #8427 (cancelled).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.