Search follow-ups: privacy-safe alias capture, telemetry persistence, loud demo fallback, owner-auth e2e, live chip counts

docs/rag-hybrid-findings-and-todo.md

@@ -234,28 +234,68 @@ denied to set parameter`)** — the RC11 blocker. The only method hosted allows
 
 ## Follow-ups filed 2026-07-06 (universal-search workstream)
 
-17. ⏳ **Alias promotion pipeline is blocked by privacy redaction.** `rag_query_misses` rows store
- hashed/redacted queries with empty `candidate_aliases`, so hardcoded `synonymGroups` /
- `domainAliasGroups` / special-case rewrites in `src/lib/clinical-search.ts` cannot be replaced
- with data-driven `rag_aliases` rows until a privacy-safe candidate-alias capture is designed.
+17. 🔶 **Alias promotion pipeline is blocked by privacy redaction — PARTIALLY UNBLOCKED
+ (2026-07-06).** Weak-search misses now store `queryVocabularyAliasesForStorage(query)` as
+ `candidate_aliases` when raw retention is off: only canonical terms from the curated
+ clinical vocabulary that the query MATCHED are persisted (output text comes from the fixed
+ vocabulary table, never the raw query, so RET-H4 holds). Remaining: terms OUTSIDE the
+ curated vocabulary still cannot be captured without a privacy review; promotion tooling
+ from `candidate_aliases` → `rag_aliases` is still manual.
 18. ⏳ **`document_index_units` vector recall** — no HNSW index (dropped 2026-07-02) and hosted
  Supabase denies `ALTER FUNCTION … SET hnsw.ef_search` for the `language sql` hybrid RPCs, so
- only `match_document_memory_cards_hybrid` pins `ef_search=100`. Quantify the recall impact
- before reintroducing an index.
-19. ⏳ **Demo fallback can mask live retrieval failures in non-prod.** `/api/search` and
- `/api/answer` silently swap in demo data on Supabase errors outside production (only an
- `X-Clinical-KB-Fallback` header signals it). Proposal: surface a warning in
- `check:production-readiness` output and/or a visible dev-mode banner rather than changing
- the fallback behaviour.
-20. ⏳ **Automated guard for governance-weighting regressions.** The 23/23 → 16/23 golden-set
- regression class (governance metadata weighting selection ordering) is only guarded by the
- manual PR checklist because `eval:retrieval:quality` needs live keys. Investigate a
- keys-free structural test (e.g. assert selection sort inputs exclude governance fields).
-21. ⏳ **Recalibrate gates for synthetic text-only similarity (RC9 residual).** Text-fast-path
- results now carry `similarity_origin: "synthetic_text"` telemetry; once enough data exists,
- recalibrate `evaluateEvidenceCoverageGate` / text-fast-path thresholds against real cosine
- distributions instead of the `least(0.95, 0.56 + text_rank*0.39)` proxy.
+ only `match_document_memory_cards_hybrid` pins `ef_search=100`. Concrete measurement plan
+ (needs live keys, ~1 hour): run `eval:retrieval:quality` twice with `--force-embedding`
+ (bypasses lexical fast paths, exercising vectors directly) — once as-is and once after
+ `create index concurrently` on `document_index_units.embedding` in a Supabase branch — and
+ compare doc-recall@5 + p90 latency. If recall gain < 1 case, close as not-worth-4.4GB. The
+ ef_search half can be retested via the plpgsql-wrapper trick that memory_cards already uses
+ (wrap the `language sql` RPC in a plpgsql shim that SETs it).
+19. ✅ **Demo fallback can mask live retrieval failures in non-prod — DONE (2026-07-06).**
+ `nonProductionSupabaseDemoFallbackReason` (the shared choke point for /api/search,
+ /api/answer, and /api/answer/stream) now emits a loud `console.warn` naming the env vars to
+ check whenever the non-prod demo fallback fires; behaviour and the
+ `X-Clinical-KB-Fallback` header are unchanged. A visible dev-mode banner remains optional.
+20. ✅ **Automated guard for governance-weighting regressions — ALREADY COVERED.** A keys-free
+ structural test exists: `tests/retrieval-selection.test.ts` ("keeps relevance ordering and
+ does not let source-governance metadata reorder selection") asserts a higher-relevance
+ `review_due`/`unverified` source outranks a lower-relevance `current`/`reviewed` one. The
+ manual golden-eval checklist remains the live backstop; no further action.
+21. 🔶 **Recalibrate gates for synthetic text-only similarity (RC9 residual) — DATA NOW
+ FLOWING (2026-07-06).** `synthetic_similarity_count` and `text_or_relaxation_used` are now
+ persisted into `rag_retrieval_logs.metadata` (they were computed but dropped by the
+ telemetry whitelist in /api/search). Once ~2 weeks of live rows exist, recalibrate
+ `evaluateEvidenceCoverageGate` / text-fast-path thresholds against real cosine
+ distributions: query `metadata->>'synthetic_similarity_count'` joined to `is_miss` to see
+ how often synthetic scores cross the 0.58/0.62 gates on misses vs hits.
 22. ⏳ **Registry-to-corpus embedding (universal search Phase 5).** Medications/services/forms/
  differentials are federated into `/api/search/universal` but are not retrieval-corpus
- entities, so Answer mode cannot cite them. If product wants that: env-flagged ingestion,
- golden-eval + invented-term controls first (depends on 17 for alias hygiene).
+ entities, so Answer mode cannot cite them. Concrete implementation spec (in order):
+ 1. Flag `RAG_REGISTRY_CORPUS_EMBEDDING` (default off) in `src/lib/env.ts`.
+ 2. Ingestion script `scripts/embed-registry-records.ts`: map each registry record to a
+ synthetic "document" (`metadata.source_kind = 'registry_record'`, title = record title,
+ one chunk per record from the record's search text, embedded with the standard
+ `text-embedding-3-small` path) so the existing chunk pipeline/RPCs need no schema change.
+ 3. Re-embed on registry edit: hook `ensureRegistrySeeded` / record-update routes to enqueue
+ re-embedding for the changed slug only.
+ 4. Answer-surface labelling: `sourceGovernanceWarnings` must label registry-backed
+ citations distinctly (registry records are curated summaries, not source documents).
+ 5. Gates before enabling anywhere real: `eval:retrieval:quality` 23/23 with the flag ON,
+ plus invented-term controls ("florbizone syndrome management") still refusing — registry
+ rows must not become a fabrication surface for unsupported topics.
+23. ⏳ **Finding #11 full fix (RAG optimisation Phase 2)** — the classifier-verdict memo (shipped
+ 2026-07-06) makes zero-result behaviour deterministic per query but does not close the gap:
+ the deterministic analyzer still cannot tell in-corpus topics from out-of-corpus ones.
+ Phase-2 spec stands (corpus-grounded relevance: IDF/corpus-frequency weighting of query
+ terms + data-driven vocabulary), with the added prerequisite that item 17's vocabulary
+ capture now supplies real miss data to seed the vocabulary from.
+24. ⏳ **OCR dropped-letter corruption in table index units** — no reliable detector exists (82%
+ false positives; guard reverted). Next viable angle: dictionary-based repair at INGESTION
+ (compare table-cell tokens against the document's own clean chunk text — "p ycho ocial"
+ aligns to "psychosocial" within the same page's raw text) rather than heuristic detection at
+ query time. Scope to `worker/` table extraction; requires the Python OCR stack to test.
+25. ⏳ **Retrieval latency p90 ~8.6s (local)** — remaining sequential layers after the 2026-07-01
+ parallelisation. Cheapest next step (measure first): overlap `embedTextWithTelemetry` with
+ the text fast path unconditionally (today preload only fires when `shouldPreloadEmbedding`),
+ and collapse the repeated `attachDocumentRankingMetadata` calls to one batched fetch per
+ request. Both are perf-only; gate with the golden eval unchanged + p90 from
+ `rag_retrieval_logs` before/after.

src/app/api/search/route.ts

@@ -30,6 +30,7 @@ import {
  queryDerivedTokensForStorage,
  queryPrivacyMetadata,
  queryTextForStorage,
+ queryVocabularyAliasesForStorage,
 } from "@/lib/query-privacy";
 import { safeErrorLogDetails } from "@/lib/privacy";
 import { nonProductionSupabaseDemoFallbackReason } from "@/lib/supabase/errors";
@@ -414,8 +415,12 @@ function candidatePromotions(query: string, results: SearchResult[]) {
  document_id: label.document_id,
  confidence: label.confidence,
  }));
+ const rawTokens = queryDerivedTokensForStorage(Array.from(new Set(queryTerms)).slice(0, 10));
  return {
- aliases: queryDerivedTokensForStorage(Array.from(new Set(queryTerms)).slice(0, 10)),
+ // With raw retention off, fall back to curated clinical-vocabulary matches — output text
+ // comes from the fixed vocabulary table, never the query, so it is RET-H4 safe and keeps
+ // the alias-promotion pipeline fed (rag-hybrid-findings item 17).
+ aliases: rawTokens.length ? rawTokens : queryVocabularyAliasesForStorage(query),
  labels: topLabels,
  };
 }
@@ -528,6 +533,11 @@ function retrievalDecisionTelemetry(telemetry: Record<string, unknown>) {
  second_stage_rerank_used: telemetryBoolean(telemetry, "second_stage_rerank_used"),
  second_stage_rerank_latency_ms: telemetryNumber(telemetry, "second_stage_rerank_latency_ms"),
  visual_direct_image_count: telemetryNumber(telemetry, "visual_direct_image_count"),
+ // RC9/P8b observability: these feed the synthetic-similarity gate recalibration and the
+ // weak-match OR-augmentation review (rag-hybrid-findings items 21 and the P8b extension) —
+ // without persisting them the recalibration has no data to work from.
+ text_or_relaxation_used: telemetryString(telemetry, "text_or_relaxation_used"),
+ synthetic_similarity_count: telemetryNumber(telemetry, "synthetic_similarity_count"),
  };
 }
 

src/components/clinical-dashboard/universal-search-command-surface.tsx

@@ -34,6 +34,17 @@ const excludedDomainByMode: Partial<Record<AppModeId, UniversalSearchDomain>> =
  tools: "tools",
 };
 
+// Reverse of modeIdByDomain for chip counts: the domain whose live result total a
+// cross-mode chip should show. Answer/favourites chips have no countable domain.
+const domainByTargetMode: Partial<Record<AppModeId, UniversalSearchDomain>> = {
+ documents: "documents",
+ prescribing: "medications",
+ services: "services",
+ forms: "forms",
+ differentials: "differentials",
+ tools: "tools",
+};
+
 const modeIdByDomain: Record<UniversalSearchDomain, AppModeId> = {
  documents: "documents",
  medications: "prescribing",
@@ -357,6 +368,7 @@ export function UniversalSearchCommandSurface({
  const showSafetyBanner =
  modeId === "differentials" && differentialRedFlagTerms.some((term) => trimmedQuery.toLowerCase().includes(term));
  const showFormCodeHint = modeId === "forms" && isFormCodeQuery(trimmedQuery);
+ const { groups: universalGroups, query: universalQuery } = universal;
 
  const sections = useMemo(() => {
  if (!config) return [];
@@ -453,8 +465,8 @@ export function UniversalSearchCommandSurface({
  // search endpoint, excluding this mode's own domain. Selecting an item navigates straight
  // to the record; each group ends with a cross-mode "view all" that re-runs the query in
  // the owning mode. Enter with nothing highlighted still runs the mode-scoped search.
- if (trimmedQuery && universal.query === trimmedQuery && universal.groups.length) {
- for (const group of universal.groups) {
+ if (trimmedQuery && universalQuery === trimmedQuery && universalGroups.length) {
+ for (const group of universalGroups) {
  const targetModeId = modeIdByDomain[group.kind];
  const targetMode = appModeDefinition(targetModeId);
  const GroupIcon = appModeIcons[targetModeId];
@@ -564,6 +576,13 @@ export function UniversalSearchCommandSurface({
  items: config.crossModes.map((target) => {
  const targetMode = appModeDefinition(target);
  const TargetIcon = appModeIcons[target];
+ // Live count from the universal typeahead response ("Forms (2)") — only shown when
+ // fresh results for this exact query exist, so the chip never shows a stale number.
+ const targetDomain = domainByTargetMode[target];
+ const targetCount =
+ targetDomain && universalQuery === trimmedQuery
+ ? universalGroups.find((group) => group.kind === targetDomain)?.total
+ : undefined;
  return {
  id: nextId(),
  label: targetMode.label,
@@ -582,6 +601,7 @@ export function UniversalSearchCommandSurface({
  >
  <TargetIcon className="h-3.5 w-3.5 text-[color:var(--clinical-accent)]" aria-hidden />
  {targetMode.label}
+ {typeof targetCount === "number" ? ` (${targetCount})` : ""}
  </span>
  ),
  };
@@ -605,8 +625,8 @@ export function UniversalSearchCommandSurface({
  router,
  showFormCodeHint,
  trimmedQuery,
-universal.groups,
-universal.query,
+universalGroups,
+universalQuery,
  ]);
 
  const flatItems = useMemo(() => sections.flatMap((section) => section.items), [sections]);

src/components/clinical-dashboard/use-universal-search.ts

@@ -32,18 +32,26 @@ export function useUniversalSearch(args: {
  const { authorizationHeader } = useAuthSession();
  const [result, setResult] = useState<{ groups: UniversalSearchGroup[]; query: string }>({ groups: [], query: "" });
  const requestSeqRef = useRef(0);
+ const prevAuthRef = useRef(authorizationHeader);
  const trimmedQuery = args.query.trim();
  const active = args.enabled && trimmedQuery.length >= minQueryLength;
  const limitPerDomain = args.limitPerDomain ?? 3;
  const excludeDomain = args.excludeDomain;
 
  useEffect(() => {
+ const authChanged = prevAuthRef.current !== authorizationHeader;
+ prevAuthRef.current = authorizationHeader;
+
  if (!active) {
  // Invalidate any in-flight request; visible state is derived, so no reset needed.
  requestSeqRef.current += 1;
  return undefined;
  }
 
+ if (authChanged) {
+ setResult({ groups: [], query: "" });
+ }
+
  const requestId = ++requestSeqRef.current;
  const timer = window.setTimeout(() => {
  const domains = (["documents", "medications", "services", "forms", "differentials", "tools"] as const).filter(

src/lib/query-privacy.ts

@@ -1,4 +1,5 @@
 import { createHash, createHmac } from "node:crypto";
+import { clinicalVocabularyMatches } from "@/lib/clinical-vocabulary";
 import { env } from "@/lib/env";
 
 export function normalizeQueryText(query: string) {
@@ -46,6 +47,16 @@ export function queryDerivedTokensForStorage(tokens: string[]): string[] {
  return env.RAG_PERSIST_RAW_QUERY_TEXT ? tokens : [];
 }
 
+// RET-H4-safe candidate aliases for the alias-promotion pipeline (rag-hybrid-findings
+// item 17): every returned string is a canonical term from the curated clinical
+// vocabulary that the query MATCHED — the output text comes from the fixed vocabulary
+// table, never from the raw query — so patient-identifying text cannot leak even with
+// raw retention off. This unblocks rag_query_misses.candidate_aliases, which was always
+// empty under redaction and starved data-driven promotion into rag_aliases.
+export function queryVocabularyAliasesForStorage(query: string, limit = 10): string[] {
+ return Array.from(new Set(clinicalVocabularyMatches(query, limit).map((entry) => entry.canonical))).slice(0, limit);
+}
+
 // Privacy metadata to fold into a logged row's `metadata` jsonb: a stable hash
 // for joins/dedup and a flag recording whether raw text was retained.
 export function queryPrivacyMetadata(query: string) {

src/lib/supabase/errors.ts

@@ -23,5 +23,14 @@ export function isSupabaseApiKeyConfigurationError(error: unknown) {
 export function nonProductionSupabaseDemoFallbackReason(error: unknown) {
  if (process.env.NODE_ENV === "production") return null;
  if (!isSupabaseApiKeyConfigurationError(error)) return null;
+ // Item 19 (rag-hybrid-findings): this fallback silently swaps demo data in for live search
+ // and answer responses outside production, which can make a broken live path look healthy
+ // during local/dev testing. Keep the behaviour, but make it loud in the server log — the
+ // only other signal is the easy-to-miss X-Clinical-KB-Fallback response header.
+ console.warn(
+ "[clinical-kb] Supabase unavailable — serving DEMO data as a non-production fallback. " +
+ "Live search/answer paths are NOT being exercised. Check NEXT_PUBLIC_SUPABASE_URL / " +
+ "SUPABASE_SERVICE_ROLE_KEY if this is unexpected.",
+ );
  return "supabase_api_key_configuration_unavailable";
 }

tests/privacy.test.ts

@@ -146,3 +146,31 @@ describe("query privacy storage helpers", () => {
  expect(queryDerivedTokensForStorage(["clozapine"])).toEqual(["clozapine"]);
  });
 });
+
+describe("queryVocabularyAliasesForStorage (RET-H4-safe candidate aliases)", () => {
+ it("returns only curated vocabulary canonicals matched by the query, never query text", async () => {
+ vi.doMock("@/lib/env", () => ({ env: { RAG_PERSIST_RAW_QUERY_TEXT: false } }));
+ const { queryVocabularyAliasesForStorage } = await import("../src/lib/query-privacy");
+ const { clinicalVocabularyEntries } = await import("../src/lib/clinical-vocabulary");
+ const canonicals = new Set(clinicalVocabularyEntries().map((entry) => entry.canonical));
+
+ // A query mixing a patient-identifying name with clinical vocabulary must only ever emit
+ // the curated canonical — the name cannot appear because output strings come from the
+ // fixed vocabulary table, not from the query.
+ const aliases = queryVocabularyAliasesForStorage("John Citizen ANC threshold for depot");
+ expect(aliases.length).toBeGreaterThan(0);
+ for (const alias of aliases) {
+ expect(canonicals.has(alias)).toBe(true);
+ expect(alias.toLowerCase()).not.toContain("john");
+ expect(alias.toLowerCase()).not.toContain("citizen");
+ }
+ expect(aliases).toContain("absolute neutrophil count");
+ expect(aliases).toContain("long acting injectable");
+ });
+
+ it("returns nothing for queries with no vocabulary match", async () => {
+ vi.doMock("@/lib/env", () => ({ env: { RAG_PERSIST_RAW_QUERY_TEXT: false } }));
+ const { queryVocabularyAliasesForStorage } = await import("../src/lib/query-privacy");
+ expect(queryVocabularyAliasesForStorage("John Citizen follow up appointment")).toEqual([]);
+ });
+});