test(rag): #019 fallback-layer reproducer for admission source drop - #1303
test(rag): #019 fallback-layer reproducer for admission source drop#1303BigSimmo wants to merge 1 commit into
Conversation
Isolate ledger #19 ("Admission doc dropped after deterministic comparison packing") to the answer composer, not retrieval. Root cause (src/lib/rag/rag-extractive-answer.ts): for the source-bound admission/discharge comparison query, buildFactSynthesizedAnswer (:2315) calls buildAdmissionDischargeComparisonAnswer (:2234), which needs one bound requirement fact per side from sourceBoundComparisonFacts (:2180). A side only binds when the retrieved chunk sentence matches the narrow admission/dischargeRequirementBindingPatterns (:1050/:1068). When retrieval surfaces the correct admission document but the chunk prose is phrased outside those regexes, admissionFacts is empty, the distinct-document pair is null, and the builder returns null (:2253); the caller then emits the gap answer with citationChunkIds: [] (:2320), dropping the retrieved admission source. Adds: - tests/rag-admission-discharge-comparison-fallback.test.ts — drop characterization (green on current main), a control proving the binder is the discriminator (same inputs, binder-matching admission prose -> both sources cited), and an it.skip RED acceptance spec for the eventual fix. - docs/rag-019-admission-source-drop-reproducer.md — root cause + fix direction. - one-line export seam on buildAdmissionDischargeComparisonAnswer (additive). Reproducer only. No behaviour change; the fix is canary-gated. RAG impact: no retrieval behaviour change — test-only reproducer plus an additive export seam; zero logic, ranking, or ordering change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Warning Review limit reached
Next review available in:12 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
BigSimmo
commented
Jul 28, 2026
Closing as invalid. #19 is already resolved on main (2026-07-27 source-bound comparison fallback + canary, per docs/outstanding-issues.md). The traced code (buildAdmissionDischargeComparisonAnswer returning null when only one side source-binds) is that fix's intended conservative evidence-gap termination, not a bug — this reproducer used synthetic admission prose that deliberately misses the requirement binders. No behaviour change; nothing to merge. |
Isolate ledger #19 ("Admission doc dropped after deterministic comparison packing") to the answer composer, not retrieval.
Root cause (src/lib/rag/rag-extractive-answer.ts): for the source-bound admission/discharge comparison query, buildFactSynthesizedAnswer (:2315) calls buildAdmissionDischargeComparisonAnswer (:2234), which needs one bound requirement fact per side from sourceBoundComparisonFacts (:2180). A side only binds when the retrieved chunk sentence matches the narrow admission/dischargeRequirementBindingPatterns (:1050/:1068). When retrieval surfaces the correct admission document but the chunk prose is phrased outside those regexes, admissionFacts is empty, the distinct-document pair is null, and the builder returns null (:2253); the caller then emits the gap answer with citationChunkIds: [] (:2320), dropping the retrieved admission source.
Adds:
Reproducer only. No behaviour change; the fix is canary-gated.
RAG impact: no retrieval behaviour change — test-only reproducer plus an additive export seam; zero logic, ranking, or ordering change.
Summary
Verification
npm run verify:pr-localDuring development, use
npm run verify:cheapas the faster iteration gate before the final PR-local preflight.npm run verify:uiwhen UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changednpm run verify:releasebefore release or handoff confidence claimsFor retrieval, ranking, selection, chunking, source/citation rendering, or answer-contract changes,
verify:pr-localrunseval:rag:offlineautomatically. Run the offline command directly during iteration before spending a live eval.npm run eval:retrieval:quality(must stay 36/36) when retrieval, ranking, selection, chunking, or scoring behavior changed — CI cannot run it (needs live keys), so run it locally and paste the summary. A metadata/governance-weighting change once buried correct docs (recall 1.0→0.76) and only this eval caught it.npm run eval:rag -- --limit 15+npm run eval:quality -- --rag-onlywhen answer generation, the synthesis prompt, or answer post-processing changed (grounded-supported must not drop; citation-failure 0)npm run check:production-readinesswhen clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changednpm run check:deployment-readinesswhen deployment startup, hosting, or rollout behavior changedRisk and rollout
Complete this section for clinical, data, API, auth/privacy, workflow, dependency, build, or deployment changes.
Clinical Governance Preflight
Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.
Clinical KB Database(sjrfecxgysukkwxsowpy)Notes