Harden production demo guards and fallback behavior - #90
Conversation
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
💡 Codex Review
Database/src/app/api/eval-cases/route.ts
Line 148 in 81a90b4
Preserve the real question for promoted eval casesWith the default RAG_PERSIST_RAW_QUERY_TEXT=false, this stores query as a redacted-query:<hash> placeholder while line 158 still marks the row as promoted_eval_case; loadCapturedRagEvalCases later uses row.query as the eval question, so npm run eval:quality will ask the hash string instead of the clinician's prompt. For promoted eval captures, either require/retain the real question under an approved retention path or avoid promoting redacted rows.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return { | ||
| answer: boldHighYieldClinicalText(answer, args.query), | ||
| body: boldHighYieldClinicalText(answer, args.query), | ||
| citationChunkIds: Array.from(new Set(facts.flatMap((fact) => fact.citationChunkIds))), |
There was a problem hiding this comment.
When the new fact synthesizer chooses facts from a lower-ranked result, these citationChunkIds can point at chunks that are not included in the top-level citations, because buildExtractiveAnswer still returns the compact citations seeded from the result list and never merges these ids. In that scenario the answer is marked grounded but the visible citations can omit the actual supporting chunk or cite an unrelated top result, which breaks source verification for clinical answers.
Useful? React with 👍 / 👎.
| return tokens.some((token) => queryTokenMatchesText(token, text)); | ||
| return ( | ||
| answerIntentEvidencePattern(intent).test(normalized) && | ||
| (intentTokens.length === 0 || intentTokens.some((token) => queryTokenMatchesText(token, normalized))) |
There was a problem hiding this comment.
For monitoring, renal-dose, and red-result questions, queryIntentTokens injects canonical words such as monitoring, renal, or threshold, but this check then requires one of those exact words even after answerIntentEvidencePattern has already accepted equivalent source language like “check levels every 3 months”, “eGFR < 30”, or “ANC < … stop”. Those directly relevant sentences are filtered out before synthesis, causing unsupported gap answers despite matching evidence.
Useful? React with 👍 / 👎.
| .replace(extractiveLabelPattern, " ") | ||
| .replace(/^[\s\-•:]+/, "") | ||
| .replace(/^(?:monitoring|dose|dosing|source|section|table|guideline)\s*[.;:,-]\s*/i, "") | ||
| .replace(/([A-Za-z)])\s*(\d{1,2})(?=(?:[,.;]|\s|$))/g, "$1") |
There was a problem hiding this comment.
This cleanup regex removes any 1–2 digit number that follows a word, so common evidence such as “maximum dose 20 mg daily”, “reduce dose 5 mg”, or “check every 3 months” becomes “maximum dose mg daily”, “reduce dose mg”, and “check every months” before fact synthesis. For dose, threshold, and monitoring answers this drops the actual clinical values from otherwise supported source sentences, which can produce unsafe or unusable guidance.
Useful? React with 👍 / 👎.
… writer (#313) Re-prioritising is the mutation triage performs most often, and it was the one mutation the ledger tooling could not express. AGENTS.md and the ledger both require mutating rows only through the writer — precisely because hand-authored rows produced the width, escaping and wrong-table failures the writer exists to stop — so every demotion had to take the forbidden path. npm run issues:update -- '#151' --pri P3 The gap turned out to be three layers deep, and each one looked fine while the others were broken: 1. scripts/outstanding-issues.mjs — updateIssue's editable map had no `pri` entry, so the Pri cell was unreachable. 2. scripts/ledger-inbox.mjs createRequest — the update payload was built from summary/detail/source only, so even once updateIssue accepted pri, the CLI could not emit it. 3. scripts/ledger-inbox.mjs validateRequest — required one of summary/detail/source, so a pri-only request was rejected before it reached either of the above. A re-prioritisation with no prose change is a legitimate triage edit and now counts on its own. Values are validated against the same P1/P2/P3 set `add` uses, at both the request boundary and the writer, and rejected before the guarded write so a bad value reports itself rather than surfacing as a gate refusal about a malformed row. Deliberately per-row: no bulk re-prioritise mode, because a sweep that moves many rows at once should be visible row by row in review. Every assertion here was mutation-tested rather than merely observed passing. Removing `pri` from the editable map makes the writer throw on a pri-only request; dropping the createRequest passthrough makes the CLI reject it; dropping it from the validator fails the inbox self-test. The writer's own self-test additionally pins that ONLY the Pri cell moves — an off-by-one there would write a priority over the ID or Type cell and still produce a structurally valid row. Verified end to end through the real CLI: `issues:update '#90' --pri P3` queues a valid merge-safe request. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuYJz8hauCsCdx8r4fXiZU
Summary
hnsw.ef_search.Verification
npm run verify:cheapnpm run verify:uiwhen UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changednpm run verify:releasebefore release or handoff confidence claimsnpm run format:checknpm run check:production-readinesswhen clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changedAdditional checks run:
npm run buildnpm run check:supabase-projectnpx vitest run tests/supabase-schema.test.tsgit diff --checkNotes on unchecked boxes:
npm run verify:releasewas attempted afterverify:cheap,verify:ui, andbuild; the attempt timed out after 10 minutes while running the full Playwright set. Equivalent local coverage completed separately for runtime, lint, typecheck, unit tests, production build, and Chromium UI.npm run format:checkcurrently fails repo-wide on pre-existing.claude/worktreesfiles and SQL files without a configured Prettier parser. Branch-touched code/docs files were formatted directly where Prettier supports them.Clinical Governance Preflight
Clinical KB Database(sjrfecxgysukkwxsowpy)Notes
.env.localservice-role warning and no blocking failures.