Skip to content

fix(rag): bind claim-leading for/in conditions in the high-risk trigger check (S1c follow-up) - #2065

Merged
BigSimmo merged 4 commits into
mainfrom
claude/s1c-trigger-condition-first-4pb1at
Aug 17, 2026
Merged

fix(rag): bind claim-leading for/in conditions in the high-risk trigger check (S1c follow-up)#2065
BigSimmo merged 4 commits into
mainfrom
claude/s1c-trigger-condition-first-4pb1at

Conversation

@BigSimmo

Copy link
Copy Markdown
Owner

Summary

  • fix(rag): bind claim-leading for/in conditions in the high-risk trigger check.highRiskTriggerTokens in src/lib/rag/rag-claim-support.ts recognised when/whenever/if/unless/during/after/before clauses and <verb> … for <indication> — but a claim that leads with its condition ("For elderly patients, start lithium at 500 mg nocte.", "In renal impairment, …") extracted no trigger tokens at all, so the condition never had to appear in the supporting segment. Combined with S1c's adjacent-segment topic lending, a digit-free population qualifier could be mis-bound to a different population's dose: an atom-free neighbouring bullet that merely mentions "Elderly patients" lends its topic tokens, and the adult-dose segment then supports the elderly claim (reproduced fail-first: the discriminating test is accepted at the merge base, rejected after this fix). The digit-bearing variant ("over 65 years") was already caught by the atom-free-neighbour rule because "65 years" extracts as a value atom; this closes the digit-free phrasing.
  • One new anchored pattern in the trigger extractor: ^(for|in) <condition>, (comma-terminated, ≤60 chars, claim-leading only), feeding the existing triggerTopicStopwords filter and the existing every-token-must-appear check against the single supporting segment.

Non-inferiority argument (tightening only):compatibleHighRiskTrigger is a conjunctive gate — extracting more trigger tokens can only flip accept → reject, never admit a new claim. The risk is over-rejection, bounded three ways: the pattern fires only on claim-leading comma-terminated For/In phrases; extracted tokens pass the existing trigger stopword filter; and faithful condition-first restatements are pinned green ("For patients over 65 years, start lithium at 250 mg nocte." against the EMHS sentence that states the over-65 dose; "In renal impairment, monitor lithium levels closely." against matching evidence). The full offline corpus is unchanged: 616/616 with zero fixture flips, including the existing condition-first fixtures ("For neuroleptic, any side effect …" still supported via its titled segments).

RAG impact: behaviour change — canary pair -> post-merge dispatch (owner-approved)

Merge-order note: please merge this only after the S1c canary pair (baseline run 32039841070 → post-merge dispatch on PR #2052's merge) has run, so S1c's canary attribution stays clean. This PR then takes the S1c post run as its baseline for its own pair.

Verification

  • npm run verify:pr-local — heavy scope (RAG surface); decisive lines below
  • npm run check:production-readiness — offline run; only the expected credential-free provider gaps (missing NEXT_PUBLIC_SUPABASE_URL/SUPABASE_SERVICE_ROLE_KEY, missing OPENAI_API_KEY)
  • npm run verify:ui — UI verification not run: no UI change
  • npm run eval:retrieval:quality — Verification not run: retrieval/ranking untouched (claim-support trigger gate only); live behaviour covered by the owner-approved post-merge canary pair
  • npm run eval:rag -- --limit 15 / npm run eval:quality -- --rag-only — Verification not run: provider-backed; the post-merge canary pair is the live gate

Decisive lines:

npx vitest run tests/rag-claim-support.test.ts → Tests 160 passed (160)
npm run eval:rag:offline → Tests 616 passed (616)
Offline RAG fixture and production-contract checks passed.
npm run verify:pr-local → PR-local verification summary:
completed: check:runtime, check:installed-lock-parity, format:changed, lint, typecheck, test,
build, eval:rag:offline, eval:rag:adversarial:offline, check:medication-interactions,
check:medication-lexicon-report
failed: (none) · not reached: (none)

Fail-first evidence: at the merge base (b8e774b) the discriminating mis-bind test was accepted (Tests 1 failed | 159 passed with the new tests present); after the one-pattern fix 160 passed.

Risk and rollout

  • Risk: over-rejection of condition-first claims whose condition tokens are absent from the supporting segment — which is precisely the unsafe support this closes. No acceptance widens. Zero offline fixture flips.
  • Rollback: single-commit revert; no data, schema, config, or dependency changes.
  • Provider or production effects: None in this PR. The post-merge eval-canary dispatch is provider-backed and owner-approved separately.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes


Generated by Claude Code

…er check
'For elderly patients, start lithium at 500 mg nocte.' extracted no
trigger tokens (only when/if/unless/during/after/before and verb-then-for
shapes did), so a digit-free population qualifier never had to appear in
the supporting segment — an atom-free neighbouring bullet that merely
mentions the population could lend its topics and the adult dose could be
mis-bound to the elderly claim. A claim-leading 'For <population>,' /
'In <state>,' now binds its condition exactly like when/if phrasing.
Tightening only: compatibleHighRiskTrigger can only flip accept to
reject; faithful condition-first restatements whose segment covers the
condition are pinned green, and the full offline suite (616) is unchanged.
Documented as the residual risk in PR #2052; the digit-bearing variant
('over 65 years') was already caught by the atom-free-neighbour rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNq5LiNDoZVaggA3mHBT5Q
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TNq5LiNDoZVaggA3mHBT5Q
@coderabbitai

coderabbitaiBot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:10 minutes

Limit details: You’ve used all 1 included review currently available under your plan. You completed 95 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b42a9dde-584c-4c93-a29c-fabd2bb11ae1

📥 Commits

Reviewing files that changed from the base of the PR and between 2fee9b7 and 5b54ab3.

📒 Files selected for processing (3)
  • docs/branch-review-records/ed2853a1fe7ad33b3ef164093f54fa8df20452c2f8e144ffa1546bb7603c142e.record.md
  • src/lib/rag/rag-claim-support.ts
  • tests/rag-claim-support.test.ts

Comment @coderabbitai help to get the list of available commands.

@BigSimmo
BigSimmo enabled auto-merge August 17, 2026 17:57
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@BigSimmo@claude