refactor(rag): decompose rag.ts part 2 (extractive-answer, retrieval-variants, cache) — move-only

docs/process-hardening.md

@@ -205,6 +205,15 @@ All approved render-surface modules are extracted. `ClinicalDashboard.tsx` went
 - **Measured coupling for the remaining moves (do not trust the pre-drift map):** the extractive-answer family (L, ~1,085 lines) and the answer-quality/reasoning-effort family (M, ~465 lines) are **mutually entangled** on main (M calls `classifyAnswerIntent`/`boldHighYieldClinicalText` from L; L calls `finalQualityGapAnswer`/`isFragmentLikeClinicalAnswer`/`hasBadFinalAnswerQuality` from M) and both reach into the coverage-gate helpers (K) — extract L+M together in a dedicated pass, or accept module→rag.ts back-edges. Retrieval variants (H) additionally depend on owner-scope helpers (`assertGlobalSearchAllowed`, `ownerScopeForDocumentFilteredRetrieval`) and live alias fetching; context packing (J) depends on `stableHash` + committed-generation helpers and belongs with the cache region (F) move.
 - **Standing gate:** any continuation that touches H/F (retrieval-side) must pass `npm run eval:retrieval:quality` (23/23) before merge, per the golden-eval rule above.
 
+## rag.ts decomposition — part 2 (2026-07-07)
+
+- **Base:** branched `claude/rag-decomp-part2` off `origin/main` (8878151aa, includes part-1 #341 + retrieval-correctness #343 → golden set is now **36**, not the 23 the part-1 note recorded). Approved 3-move sequence: (1) L+M together, (2) retrieval variants H, (3) context-packing J → cache F. Method identical to part 1 (verbatim byte-identical block moves, one module per commit, per-move gate). The remeasured region map (do NOT trust part-1's line numbers post-drift): L = extractive-answer at 3954–5039 (1,085 lines) and M = answer-quality/reasoning-effort at 5040–5503 (464 lines), confirming part-1's family sizes.
+- **Move 1 (shipped): L+M → `rag-extractive-answer.ts`.** Lines 3954–5503 moved verbatim (empty block diff modulo the `export` keyword added to 9 functions that now cross the boundary). Extracting L+M **together** keeps their mutual entanglement internal → no cross-module cycle between them. rag.ts imports the 12 names it still calls, re-exports the 6 tests import. Per the sanctioned "accept back-edges" path, the module imports **9 call-time helpers back from rag.ts** (`evidenceTextForGate`, `rankMemoryCardsForAnswer`, `deriveConfidence`, `scoreValue`, `machineReadableFallbackAnswer`, `buildAnswerScoreExplanations`, `buildIndexingQuality`, `collectMemoryCards`, `fallbackReasonFromRouting`) — a rag.ts↔module cycle that is runtime-safe (all uses are call-time; no `import/no-cycle` lint rule). 6 of those 9 gained an `export` keyword; the aliased `citationFromResult as resultCitation` import travels with the module; 5 imports orphaned by the move were removed from rag.ts. rag.ts 7,550 → 6,025 lines. Gate: tsc clean (56 pre-existing `fast-check` property-test errors only — `fast-check` is not installed in the shared node_modules; no new errors from the move), focused vitest 75/75, lint (only the 7 pre-existing rag.ts unused-import warnings; module 0), prettier clean.
+- **Trap logged:** a backgrounded `tsc … | tail; echo EXIT` reports the pipeline's (echo's) exit, not tsc's — always read the captured EXIT marker, never trust the background "exit 0" notification. This masked both the true baseline (56 fast-check errors) and move 1's first-cut breakage (9 unexported functions + a missed aliased import) until vitest failed.
+- **Move 2 (shipped): retrieval variants H → `rag-retrieval-variants.ts`.** Lines 1999–2283 moved verbatim (empty block diff modulo the `export` keyword added to 5 declarations): `buildRetrievalQueryVariants`, alias normalize/expand + live `fetchEnabledRagAliases`, `assertGlobalSearchAllowed`/`ownerScopeForDocumentFilteredRetrieval`, `shouldRelaxWeakTextMatches`, `textCandidateBudgetForQueryClass`, budget consts. rag.ts imports the 11 names it calls, re-exports the 6 `retrieval-query-variants.test.ts` imports. Back-edges: `SearchChunksArgs` (type-only, no runtime cycle) + `shouldShortCircuitUnsupportedSearch` (one call-time value fn, gained an `export`; belongs to the soft-tail family staying in rag.ts). 4 orphaned imports removed from rag.ts. rag.ts 6,025 → 5,750 lines. Gate: tsc clean, focused vitest 102/102 (incl. `retrieval-query-variants.test.ts`), lint (7 pre-existing warnings only), prettier clean.
+- **Move 3 (shipped, final): cache region F + context-packing J → `rag-cache.ts`.** Cache is interleaved in rag.ts with the query-classifier, unsupported-soft-tail, and query-expansion regions, so this was a **multi-region verbatim move** (4 contiguous regions assembled into one module, each byte-identical to source; diff shows only the `export` keyword on 11 boundary-crossing declarations): cache maps/consts + cache-key helpers + main cache block (get/set answer+search, shared-cache, `stableHash`, `retrievalPlanCacheQuery`, invalidation) + context-packing. rag.ts imports the 12 names it calls (incl. the `answerInflight` map) and re-exports 4 (`invalidateRagCachesForOwner`/`…ForDocumentMutation` for API routes; `packedContextCacheKey`/`retrievalPlanCacheQuery` for tests). Back-edges: `SearchChunksArgs` + `SearchTelemetry` (type-only) and one call-time value fn `retrievalPlanForQueryClass` (gained `export`). Notably the interleaved classifier/soft-tail/expansion helpers are cleanly _not_ referenced by the cache union — only those 3 back-edges. 3 orphaned imports removed. rag.ts 5,750 → 5,108 (**7,550 → 5,108 across part 2, −2,442 / −32%**; the file now holds the orchestrator + retrieval-diagnostics + classifier + coverage-gate + memory/enrichment + logging). Gate: tsc clean, focused vitest 118/118, lint (7 pre-existing warnings only; module 0), prettier clean.
+- **Golden retrieval eval environment-blocked — root cause is a data/owner mismatch, not code or OpenAI:** `npm run eval:retrieval:quality` FAILS ~36/36 with every retrieval layer at 0 (`text_candidates:0`, `hybrid_vector:0`, …). Proven not a regression: pristine `origin/main` rag.ts (moved modules inert) reproduces the **byte-identical** failure output (24/24 printed cases, latency aside). Direct Supabase probe with the configured service key: **2,065 documents / 69,334 chunks present, but `owner_id` is `null` (global) on the sampled docs while `RAG_EVAL_OWNER_ID` (2bac05f1-…) owns 0**, and owner-scoped retrieval now **fail-closes** on a null owner (privacy hardening) — so no owner value retrieves the null-owner corpus and the eval returns 0 candidates. Restoring OpenAI credits fixed embeddings (`embedding:1` succeeds) but not this. To run a real 36/36, the corpus docs must be assigned to the eval owner (or the eval must target the actual owner / allow global search). Behavior preservation for moves 1–3 is evidenced by byte-identical moves + tsc + focused vitest + **pristine-eval equivalence**; the mandated clean 36/36 + `eval:quality --rag-only` must run once the eval-owner/corpus config is fixed (or in CI), before merge.
+
 ## Answer-thread Back button: URL and visible answer can disagree (2026-07-06)
 
 - **Behaviour:** inside an Answer thread, browser Back changes the URL (`/?mode=answer&q=A&run=1` ← `...q=B&run=1`) but the rendered answer/thread does not change. Two guards produce this: the auto-run effect skips when `run=1` is already present, and the answer view early-returns when an answer is already on screen (`ClinicalDashboard`). This is thread persistence by design, not an accident — clearing the thread on Back would destroy in-progress clinical context.