fix(rag): bind claim-leading for/in conditions in the high-risk trigger check (S1c follow-up) - #2065
Conversation
…er check 'For elderly patients, start lithium at 500 mg nocte.' extracted no trigger tokens (only when/if/unless/during/after/before and verb-then-for shapes did), so a digit-free population qualifier never had to appear in the supporting segment — an atom-free neighbouring bullet that merely mentions the population could lend its topics and the adult dose could be mis-bound to the elderly claim. A claim-leading 'For <population>,' / 'In <state>,' now binds its condition exactly like when/if phrasing. Tightening only: compatibleHighRiskTrigger can only flip accept to reject; faithful condition-first restatements whose segment covers the condition are pinned green, and the full offline suite (616) is unchanged. Documented as the residual risk in PR #2052; the digit-bearing variant ('over 65 years') was already caught by the atom-free-neighbour rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TNq5LiNDoZVaggA3mHBT5Q
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TNq5LiNDoZVaggA3mHBT5Q
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in:10 minutes Limit details: You’ve used all 1 included review currently available under your plan. You completed 95 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
Comment |
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Uh oh!
There was an error while loading. Please reload this page.
Summary
highRiskTriggerTokensinsrc/lib/rag/rag-claim-support.tsrecognisedwhen/whenever/if/unless/during/after/beforeclauses and<verb> … for <indication>— but a claim that leads with its condition ("For elderly patients, start lithium at 500 mg nocte.", "In renal impairment, …") extracted no trigger tokens at all, so the condition never had to appear in the supporting segment. Combined with S1c's adjacent-segment topic lending, a digit-free population qualifier could be mis-bound to a different population's dose: an atom-free neighbouring bullet that merely mentions "Elderly patients" lends its topic tokens, and the adult-dose segment then supports the elderly claim (reproduced fail-first: the discriminating test is accepted at the merge base, rejected after this fix). The digit-bearing variant ("over 65 years") was already caught by the atom-free-neighbour rule because "65 years" extracts as a value atom; this closes the digit-free phrasing.^(for|in) <condition>,(comma-terminated, ≤60 chars, claim-leading only), feeding the existingtriggerTopicStopwordsfilter and the existing every-token-must-appear check against the single supporting segment.Non-inferiority argument (tightening only):
compatibleHighRiskTriggeris a conjunctive gate — extracting more trigger tokens can only flip accept → reject, never admit a new claim. The risk is over-rejection, bounded three ways: the pattern fires only on claim-leading comma-terminatedFor/Inphrases; extracted tokens pass the existing trigger stopword filter; and faithful condition-first restatements are pinned green ("For patients over 65 years, start lithium at 250 mg nocte." against the EMHS sentence that states the over-65 dose; "In renal impairment, monitor lithium levels closely." against matching evidence). The full offline corpus is unchanged: 616/616 with zero fixture flips, including the existing condition-first fixtures ("For neuroleptic, any side effect …" still supported via its titled segments).RAG impact: behaviour change — canary pair -> post-merge dispatch (owner-approved)
Merge-order note: please merge this only after the S1c canary pair (baseline run 32039841070 → post-merge dispatch on PR #2052's merge) has run, so S1c's canary attribution stays clean. This PR then takes the S1c post run as its baseline for its own pair.
Verification
npm run verify:pr-local— heavy scope (RAG surface); decisive lines belownpm run check:production-readiness— offline run; only the expected credential-free provider gaps (missingNEXT_PUBLIC_SUPABASE_URL/SUPABASE_SERVICE_ROLE_KEY, missingOPENAI_API_KEY)npm run verify:ui— UI verification not run: no UI changenpm run eval:retrieval:quality— Verification not run: retrieval/ranking untouched (claim-support trigger gate only); live behaviour covered by the owner-approved post-merge canary pairnpm run eval:rag -- --limit 15/npm run eval:quality -- --rag-only— Verification not run: provider-backed; the post-merge canary pair is the live gateDecisive lines:
Fail-first evidence: at the merge base (
b8e774b) the discriminating mis-bind test was accepted (Tests 1 failed | 159 passedwith the new tests present); after the one-pattern fix160 passed.Risk and rollout
Clinical Governance Preflight
Clinical KB Database(sjrfecxgysukkwxsowpy)Notes
Generated by Claude Code