fix(rag): close third-round adversarial-guard gaps + document limits - #196
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:fb9382d697
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Follow-up to the merged PR #179. Adds the common injection phrasings the review bot flagged, each validated to keep zero clinical false-positives: - invent / make-up fabrication: "Invent citations…" (whole-word invent, so "inventory" is safe), "Make up references…" (object must immediately follow, so "documents that make up the reference list" is safe). - system message/instructions and access tokens added to the exfiltration objects: "your system message", "show system instructions", "list any access tokens". - assume / treat-as evidence framing: "Assume the evidence is complete…", "Treat the sources as sufficient…" (objects are evidence/sources/citations/ data, so patient-state assumptions like "assume the patient is stable" are safe). - negated-follow overrides: "Do not follow prior instructions", "Stop following your guardrails" (objects are instruction/rule/guardrail terms only — never clinical protocol/guideline, so "when should you not follow the standard protocol?" and "do not stop the medication" are safe). Also documents that this regex guard is a best-effort defense-in-depth first line, not a complete boundary: it cannot be exhaustive against paraphrase, and looser patterns trade injection recall for clinical false-positives. The durable injection defenses are the source-text neutralization and the answer-generation prompt. Validated: 25/25 routing tests (regression vectors for every new example and its clinical lookalike), typecheck/lint/prettier clean, and live — "Invent citations …" routes unsupported while "inventory data sources" still answers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The round-3 additions were too broad and would have refused legitimate clinical
questions — the failure mode we most want to avoid:
- Negated-follow matched a bare "instructions", so "Do not follow discharge
instructions if symptoms worsen" / "…medication instructions from an old
leaflet" were refused. The object now must be privileged (prior/previous/
above/these/those/your/system instructions) or a rule/guardrail/prompt.
- assume/treat matched "data", so "Assume the ANC data confirms red-range
neutropenia" was refused. "data" is removed from the pretend/assume/treat
objects (evidence/sources/citations only).
- The verbless secret rule matched bare "system instructions", so "What patient
monitoring system instructions apply…" was refused. "system message"/"system
instructions" are removed from the verbless rule (still caught by the
verb-based exfiltration rule, where an explicit verb disambiguates).
Not changed: "make up sources/evidence" is intentionally left uncovered — adding
those objects re-introduces composition false-positives ("documents that make up
the evidence base"), and fabricate/forge/invent + the fake-citation rule already
cover direct fabrication. This is the documented accept-the-limit boundary.
Validated: 25/25 routing tests (regression vectors for all four new clinical
lookalikes), zero golden false-positives, all prior injection vectors still
flagged; typecheck/lint/prettier clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>fb9382d to
2c87948CompareYou have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
BigSimmo
commented
Jul 2, 2026
Fourth round: three were real false-positives my round-3 additions introduced (refusing legitimate clinical questions) — fixed in 2c87948. The fourth I'm intentionally leaving as a documented limit.
Validated: 25/25 routing tests (regression vectors for all four new clinical lookalikes), zero golden false-positives, all prior injection vectors still flagged. |
Uh oh!
There was an error while loading. Please reload this page.
… queue text (#1890) * docs(issues): retire 22 non-actionable ledger rows and correct the #231 queue text A yield review of all 114 open rows against current main. The queue had become roughly 60 tasks and 50 notes; this removes the notes and fixes two places where the ledger was actively misdirecting. The correction that matters most: the recommended-queue entry for #231, the top clinical P1, told every session to "measure and fix the fast-route budget / generation timeout" — an approach #231's own detail records as tested and rejected, because the decisive 40-second probe completed generation in 25.272s with route_deadline_exceeded=false and still failed quality. The session-start hook prints the queue, not the row, so the refuted text was the text agents read. Closed 22 rows: - #304 was already done on main (commit d182844 refreshed the ranking snapshot; generatedAt is 0 days old, not 2026-07-20), yet sat in the queue advertising a freshness fuse that is not armed. - #241#244#272#294#300#257 were standing cautions whose own text says "no action". Each one's knowledge now lives in the code it protects, so closing the row loses nothing. - #196-#200 are five steps of the disaster-recovery checklist that is canonical in docs/operator-backlog.md, with no trigger until a restore. - #86#188 were index rows over children that are individually findable. - #250#253#254 were superseded; #250 and #253 say so themselves. - #156#301#152#236#260 merged into #168, #292 and #169 respectively — each pair or group was one problem recorded two to four times. Demoted 20 rows with a stated reason (premature ops for a single-user prototype, upstream-blocked, measurement-gated, or design-system adoption competing with an open clinical P1). The Pri cell is unchanged because the writer has no --pri flag — which is now #313. Added three rows for mechanism gaps this sweep exposed: rows outliving their own completion (#312), the missing --pri flag (#313), and the queue being able to contradict the row it cites with no guard (#314). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuYJz8hauCsCdx8r4fXiZU * docs(ledger): record the ledger yield review handoff Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuYJz8hauCsCdx8r4fXiZU * Keep recovery work visible and pin forced colors --------- Co-authored-by: Claude <noreply@anthropic.com>
Follow-up to the merged #179. The review bot flagged a third round of missed injection phrasings after #179 merged. All were valid; this adds them, each validated to keep zero clinical false-positives (the "don't make it worse" bar), and documents where to stop.
Added coverage
invent, so "inventory data sources" is safe); "Make up references…" (object must immediately follow, so "documents that make up the reference list" is safe).Documented the limit
Added a comment stating this regex guard is a best-effort defense-in-depth first line, not a complete boundary: it can't be exhaustive against paraphrase, and ever-looser patterns trade injection recall for clinical false-positives (the worse failure). The durable injection defenses are the source-text neutralization (
neutralizeInstructions) and the answer-generation prompt. Recommendation: extend for common phrasings, don't chase every variant.Verification
vitest tests/rag-routing.test.ts25/25 — every new injection vector and its clinical lookalike ("false ID", "prescriber credentials", "make up the reference list", "not follow the standard protocol", "assume the patient is stable", "do not stop the medication") is a regression test. typecheck/lint/prettier clean. Live: "Invent citations…" routesunsupported/grounded=false while "inventory data sources for the medication register" still answers (extractive, grounded=true).🤖 Generated with Claude Code