diff --git a/.github/workflows/codex-autofix-review-comments.yml b/.github/workflows/codex-autofix-review-comments.yml index 317d2a0d56..89602d6f50 100644 --- a/.github/workflows/codex-autofix-review-comments.yml +++ b/.github/workflows/codex-autofix-review-comments.yml @@ -7,6 +7,7 @@ on: permissions: contents: read issues: write + models: read pull-requests: read jobs: diff --git a/docs/forward-codify-retrieval-rpcs-workorder.md b/docs/forward-codify-retrieval-rpcs-workorder.md new file mode 100644 index 0000000000..336e414928 --- /dev/null +++ b/docs/forward-codify-retrieval-rpcs-workorder.md @@ -0,0 +1,93 @@ +# Work-order — forward-codify live-ahead retrieval RPC bodies (drift backlog #0) + +**Status: staged, not executed.** This is plan task 1.2. It is deliberately **not** shipped as a +migration yet because the two safety preconditions were unavailable at authoring time (2026-07-12): + +1. **Byte-faithful Docker-replay validation is required and Docker was down.** The established method + (see [supabase-migration-reconciliation.md](supabase-migration-reconciliation.md) and the + `20260701140631_codify_live_retrieval_rpcs` precedent) validates each codified body byte-equivalent + to live via a whitespace-stripped `pg_get_functiondef` md5 against a container replay **before** any + apply — this is what makes the apply a proven no-op on live. Docker Desktop would not start here. +2. **Live is under active concurrent multi-session editing.** `supabase/drift-allowlist.json` warns the + snapshot "needs regeneration once churn settles." A capture taken now can be stale by apply time, so + the codification must **re-capture and re-compare at apply time**. + +Shipping a large multi-function migration + `schema.sql` reconciliation without (1), against a moving +target, would risk introducing the exact silent-retrieval-regression class this fixes. So this file +locks the capture fingerprints and the exact procedure; execute it in an environment with Docker and a +quiescent live DB. + +## Live fingerprints captured 2026-07-12 (project `sjrfecxgysukkwxsowpy`) + +Read-only capture via `pg_get_functiondef`. Re-run the capture at execution time; if any `body_md5` +below differs, live moved — codify the **new** body and note the change. + +| Function (identity args) | body_md5 | len | secdef | In allowlist as | +| ---------------------------------------------------------------------------- | ---------------------------------- | ---- | ------- | ---------------------------- | +| `match_document_chunks(vector,int,float8,uuid,uuid)` | `45cc06effe9a753604eba4af5ae43c7e` | 1553 | no | live-ahead | +| `match_document_chunks_hybrid(vector,text,int,float8,uuid[],uuid)` | `90a027977c84847730a0e48481060502` | 5638 | no | live-ahead | +| `match_document_chunks_text(text,int,uuid[],uuid)` | `9d2f8aa374f01bd149a17af56e075171` | 5596 | no | live-ahead | +| `match_document_table_facts_text(text,int,uuid[],uuid)` | `904049c7635b996a6653e91c49d86ec2` | 4609 | no | live-ahead | +| `match_documents_for_query(text,int,uuid)` | `53234990b88dba5c0466f9dccb512455` | 1448 | no | live-ahead | +| `match_document_index_units_hybrid(vector,text,int,float8,uuid[],uuid)` | `7b144ff6fdd93b753bf67f7317be0cc6` | 3309 | no | (verify) | +| `match_document_memory_cards_hybrid(vector,text,int,float8,uuid[],uuid)` | `274ec5832d85a95880ec26daf5b79f23` | 1014 | no | (verify) | +| `match_document_memory_cards_hybrid_v2(vector,text,int,float8,uuid[],uuid)` | `977ba52a9a239962d3105f2e5b071f82` | 3714 | no | (verify) | +| `match_document_embedding_fields_hybrid(vector,text,int,float8,uuid[],uuid)` | `9d6f2b7bcd009d23739aabaf93234deb` | 2476 | no | not listed (already matches) | +| `get_related_document_metadata(uuid[],uuid)` | `ada68dd136a9878de2ce45e522fc0208` | 1363 | no | live-ahead (non-retrieval) | +| `get_visual_evidence_cards(uuid,int)` | `f4aac704317472dfc06c361389793cf9` | 1370 | **yes** | unexpected_live (live-only) | + +Capture query: + +```sql +select p.oid::regprocedure::text, pg_get_functiondef(p.oid), + md5(pg_get_functiondef(p.oid)) as body_md5 +from pg_proc p join pg_namespace n on n.oid = p.pronamespace +where n.nspname='public' and p.proname = ANY (ARRAY[ + 'match_document_chunks','match_document_chunks_hybrid','match_document_chunks_text', + 'match_document_table_facts_text','match_documents_for_query', + 'match_document_index_units_hybrid','match_document_memory_cards_hybrid', + 'match_document_memory_cards_hybrid_v2','get_related_document_metadata','get_visual_evidence_cards', + 'run_visual_eval_case','run_all_visual_eval_cases','repair_enrichment_quality_batch']) +order by 1; +``` + +## Procedure (execute with Docker up + quiet live queue) + +1. **Re-capture** the full `pg_get_functiondef` text for each function above (read-only). Apply + identical whitespace normalization (strip leading/trailing whitespace, normalize internal + whitespace) to both live-captured definitions and the migration text before computing md5 hashes. + Confirm normalized md5s vs this table; codify whatever live currently is. +2. **Author** `supabase/migrations/_codify_live_retrieval_rpcs_forward.sql` — one + `CREATE OR REPLACE FUNCTION` per function, body = the verbatim captured definition. Apply the + same whitespace normalization before hashing to ensure normalized comparison (not byte-identical + raw text). Preserve `security definer`/`invoker`, `set search_path`, and grants exactly. + **Capture and replay ACLs:** alongside `pg_get_functiondef`, capture each function's ACLs using + `proacl` or routine privileges query. Include the captured grants in the migration (apply after + each `CREATE OR REPLACE`) and verify the resulting ACLs match live during validation. + `get_visual_evidence_cards`, `run_visual_eval_case`, `run_all_visual_eval_cases`, + `repair_enrichment_quality_batch` are **live-only** (`unexpected_live`) — codify as `CREATE OR +REPLACE` too so schema.sql declares them. These security-definer RPCs require particular ACL + coverage. +3. **Reconcile** `supabase/schema.sql`: replace each function's old body with the captured one (same + text as the migration). Update `tests/supabase-schema.test.ts` if it pins any of these bodies. +4. **Validate (the load-bearing gate):** `npm run drift:manifest` (replays schema.sql into Docker from + scratch — proves replayability) then confirm each codified body's whitespace-normalized + `pg_get_functiondef` md5 equals live. Compare the resulting ACLs against live using the same + query from step 2. `npm run verify:cheap` includes `tests/drift-detection.test.ts` + (migration↔schema.sql parity, allowlist hygiene). +5. **Golden eval:** `npm run eval:retrieval:quality` must stay **36/36** (retrieval bodies changed — + though a faithful capture is behavior-neutral by construction). +6. **⏸ Apply** via the approved live migration workflow. Because it is a verbatim capture, apply is an + idempotent no-op on live at capture time. +7. **Remove** the now-resolved `functions` entries from `supabase/drift-allowlist.json` (the + `match_document_*` "live-ahead" set + the codified live-only functions) and regenerate the manifest. + +## Not in this work-order (separate allowlist backlog) + +The allowlist also carries, with the same boilerplate reason, non-retrieval drift that should be triaged +separately: the **Data-API grant** posture on ~18 tables (live revoked `authenticated` grants that +schema.sql still declares — decide codify-revokes vs restore-grants), legacy/duplicate **indexes** +(`unexpected_live` drop-candidates + `missing_live` recreate-or-remove), and the PUBLIC-execute +`security-invoker` functions (`detect_legacy_ivfflat_indexes`, `document_summary_text`, +`set_document_embedding_field_content_hash`). See +[database-drift-detection.md](database-drift-detection.md#reconciliation-backlog). diff --git a/docs/launch-operator-runbook.md b/docs/launch-operator-runbook.md new file mode 100644 index 0000000000..c89119193a --- /dev/null +++ b/docs/launch-operator-runbook.md @@ -0,0 +1,164 @@ +# Launch operator runbook + +**Single sequenced runbook for the operator-gated launch steps.** It ties together the detailed docs +(linked per step) into one ordered flow with exact commands and explicit approval gates. Nothing here +runs automatically — every **⏸ PAUSE** is a provider-touching action (Supabase / Railway / OpenAI / +GitHub) that needs your explicit go-ahead, per the AGENTS.md provider boundary. + +Host note: production app + worker run on **Railway** (user directive 2026-07-12), not Fly. The image is +host-agnostic. Railway has no Sydney region (closest Singapore); data at rest stays in Supabase Sydney, +so this is a latency/SLO tradeoff only — confirm answer-p95 in the staging soak (step 4). + +Legend: **⏸ PAUSE** = provider action, needs your approval · **✅ verify** = check to run after. + +--- + +## Order at a glance + +```text +0. Pre-flight identity check +1. Apply pending live migrations (July-8 batch + PIA-4 + drift-codify) [Supabase] +2. Run the full release gate [live keys] +3. Provision staging + seed [Supabase + Railway] +4. Staging soak + rollback rehearsal [Railway] +5. Production deploy [Railway] +6. Post-deploy: worker, registry seed, auth conn cap, observability wiring +``` + +--- + +## 0. Pre-flight (read-only) + +```bash +npm run check:supabase-project # must report Clinical KB Database / sjrfecxgysukkwxsowpy +npx supabase migration list --linked +npm run reindex:health # note jobs_pending / jobs_processing (needed for step 1 R17) +``` + +## 1. Apply pending live migrations 🧑 Supabase + +Detailed runbook: [operator-apply-july8-batch.md](operator-apply-july8-batch.md). Apply **in this order** +when the ingestion queue is quiet. **Do not redeploy the worker until step `20260708130000` is live.** + +| # | Migration | Note | +| --- | ----------------------------------------------------- | ------------------------------------------- | +| a | `20260708140000_drop_ingestion_job_stages_job_id_fk` | no-op on live | +| b | `20260708130000_ingestion_concurrency_rpc_hardening` | **worker-redeploy blocker** | +| c | `20260708150000_ensure_retrieval_owner_matches` | helper before fail-closed | +| d | `20260708160001_retrieval_owner_matches_fail_closed` | tenancy fail-closed (#409) | +| e | `20260708310000_r5_document_metadata_merge` | R5 deep-merge (#408) | +| f | `20260708170000_ingestion_jobs_one_open_per_document` | R17 — approved manual `CONCURRENTLY` path | +| g | `20260708120000_rag_query_misses_retention` | **PIA-4** purge cron | +| h | `` | **only after task 1.2 lands** — see step 1b | + +**⏸ PAUSE:** apply via `supabase db push` (queue quiet) or the R17 manual `CREATE UNIQUE INDEX CONCURRENTLY` +path in the July-8 doc. R17 manual path is an approved exception to the live-change guardrail; record +the migration history entry and reconcile schema.sql after manual execution to prevent untracked drift. +R17 uses its own version so history/repair can't collide with `20260708160001`. + +**✅ verify:** + +```bash +SUPABASE_ENVIRONMENT=production npm run check:july8-live-batch +npm run check:drift +npm run check:indexing # search_schema_health() ok +npm run eval:retrieval:quality # must stay 36/36 (retrieval-affecting: step d + drift-codify) +``` + +### 1b. Drift-codify apply (task 1.2) + +The forward-codify migration (live-diverged `match_document_chunks` `hnsw.ef_search=100` wrapper + `*_text` +multi-strategy bodies) is authored + validated with normalized fingerprint comparison vs a Docker replay +before it reaches you, so its apply is an **idempotent no-op on live**. **This step is blocked until the +migration artifact is committed and execution-time live fingerprint recapture is completed.** Before +applying, recapture live fingerprints using the exact committed capture query and compare normalized md5s +against the committed table. Abort on mismatch. Apply as step 1h only after verification, then re-run +`check:drift` + `eval:retrieval:quality` (36/36). Background: +[database-drift-detection.md](database-drift-detection.md). + +## 2. Full release gate 🧑 live keys + +Clears the accumulated verification debt (universal search, cross-mode links, rag.ts decomp). + +**⏸ PAUSE** (bounded OpenAI spend): + +```bash +npm run verify:release # full Playwright matrix + check:production-readiness + # + governance:release + eval:quality:release +npm run eval:retrieval:quality # 36/36 +npm run eval:quality -- --rag-only # grounded-supported must not drop; citation-failure 0 +``` + +Record outcomes in release notes / [process-hardening.md](process-hardening.md). + +## 3. Provision staging + seed 🧑 Supabase + Railway (billable) + +Detailed: [staging-setup.md](staging-setup.md). No code change — the identity guard activates on env. + +1. **⏸ PAUSE** create Supabase project `Clinical KB Staging`, same org, **ap-southeast-2**, generate DB + password (Supabase MCP `create_project` after `confirm_cost`, or dashboard). Record ``. +2. `supabase link --project-ref ` → `supabase db push` → `npm run check:indexing`. +3. Seed synthetic (~50 docs, **never** production clinical docs): + ```bash + npm run samples && npm run import:docs + npm run registry:seed -- --owner-id --write --confirm + npm run differentials:seed && npm run medications:seed + ``` +4. Capture staging keys (`sb_publishable_…`, `sb_secret_…`). + +## 4. Staging soak + rollback rehearsal 🧑 Railway + +1. Build image (real staging publishable key inlines into the client bundle): + ```bash + docker build --build-arg NEXT_PUBLIC_SUPABASE_URL=https://.supabase.co \ + --build-arg NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY= \ + -t clinical-kb-app:staging . + ``` + (Local Docker can OOM on the 8 GiB Next heap — prefer the CI image-build workflow if it wedges.) +2. **⏸ PAUSE** deploy to Railway staging + set runtime secrets (staging values, distinct from prod): + `SUPABASE_SERVICE_ROLE_KEY`, `OPENAI_API_KEY`, `SUPABASE_PROJECT_REF=`, + `SUPABASE_PROJECT_NAME=Clinical KB Staging`, `SUPABASE_STAGING_PROJECT_REF=`, + `SUPABASE_STAGING_PROJECT_NAME=Clinical KB Staging`, `RAG_QUERY_HASH_SECRET` (staging), + `RAG_PROVIDER_MODE=auto`. Keep one warm instance (no scale-to-zero); health `/api/health`. +3. **✅ verify** boot + soak (soak is hard-guarded against production): + ```bash + npx tsx scripts/soak-test.ts --target https:// --confirm-staging \ + --users 30 --duration-s 600 --ramp-s 120 + ``` + Targets ([capacity-review.md](capacity-review.md) §4): search p95 ≤ 3 s, **answer p95 ≤ 25 s** + (watch this given the Railway↔Sydney hop), non-429 error rate < 1 %. +4. Rehearse rollback = redeploy the previous Railway image tag; confirm health returns. + +## 5. Production deploy 🧑 Railway + +Decision record: [deployment-architecture.md](deployment-architecture.md) §2. Same image contract, prod +build-args + secrets. + +**⏸ PAUSE:** authorize the Railway account/service, build with the **production** publishable key, set +runtime secrets (**incl. `RAG_QUERY_HASH_SECRET`** — PIA-2 fail-closed guard requires it at boot; +`SUPABASE_SERVICE_ROLE_KEY`, `OPENAI_API_KEY`, `SUPABASE_PROJECT_REF/NAME` for prod). One warm instance, +no scale-to-zero, health `/api/health`. I'll prep the Railway service config via the `use-railway` skill. + +**✅ verify:** `GET /api/health` → `{"status":"ok"}`; `npm run check:deployment-readiness`. + +## 6. Post-deploy + +- **Worker** 🧑 — build `Dockerfile.worker`, run **one** always-on instance in the same region with prod + secrets. **Only after migration `20260708130000` is live.** Confirms via `npm run reindex:health`. +- **Registry seed (prod)** 🧑 — `npm run registry:seed -- --owner-id --write --confirm` + (+ `differentials:seed` for the slug-retitle prune). Until seeded, Services/Forms show empty. +- **Auth connection cap** 🧑 — before the first vertical scale-up, switch Supabase auth from the 10-absolute + cap to **percentage-based** allocation in the dashboard ([capacity-review.md](capacity-review.md) §3). + Not settable via SQL/MCP. +- **Observability wiring** 🧑 — once host metrics exist, wire the warn/page SLO thresholds + ([observability-slos.md](observability-slos.md) §2) into a real alert channel; confirm the nightly eval + canary is green from `main` (one `workflow_dispatch` run). + +--- + +## Standing guardrails + +- Never raw-SQL against live — committed migration + `schema.sql` reconciliation only. +- Worker redeploy is blocked until `20260708130000` is live. +- Any retrieval/ranking change re-runs `eval:retrieval:quality` 36/36 before it ships. +- Each environment gets separate service-role + OpenAI keys (per-env blast radius). diff --git a/docs/site-map.md b/docs/site-map.md index 6012d1938f..c4cdc4da70 100644 --- a/docs/site-map.md +++ b/docs/site-map.md @@ -16,6 +16,7 @@ This file is generated by `npm run sitemap:update`. Run `npm run sitemap:check` - `/favourites` - Saved clinical items and sets. Source: `src/app/favourites/page.tsx`. - `/forms` - Forms home and search surface. Source: `src/app/forms/page.tsx`. - `/medications` - Medication index redirect. Source: `src/app/medications/page.tsx`. +- `/privacy` - Route discovered from app directory Source: `src/app/privacy/page.tsx`. - `/reference/colour-coding` - Route discovered from app directory Source: `src/app/reference/colour-coding/page.tsx`. - `/services` - Services home and search surface. Source: `src/app/services/page.tsx`. diff --git a/package-lock.json b/package-lock.json index 01dc224f57..41cebc91bd 100644 --- a/package-lock.json +++ b/package-lock.json @@ -881,9 +881,9 @@ } }, "node_modules/@eslint/eslintrc": { - "version": "3.3.5", - "resolved": "https://registry.npmjs.org/@eslint/eslintrc/-/eslintrc-3.3.5.tgz", - "integrity": "sha512-4IlJx0X0qftVsN5E+/vGujTRIFtwuLbNsVUe7TO6zYPDR1O6nFwvwhIKEKSrl6dZchmYBITazxKoUYOjdtjlRg==", + "version": "3.3.6", + "resolved": "https://registry.npmjs.org/@eslint/eslintrc/-/eslintrc-3.3.6.tgz", + "integrity": "sha512-l2Ul9PrHsPCKcEY/ac7VgFj9D80C7S68sOKc618SyHDPK36s1XcFebXY0iTzUVn4Yq+YbwvSnDmCz9yxjX+QrA==", "dev": true, "license": "MIT", "dependencies": { @@ -893,7 +893,7 @@ "globals": "^14.0.0", "ignore": "^5.2.0", "import-fresh": "^3.2.1", - "js-yaml": "^4.1.1", + "js-yaml": "^4.3.0", "minimatch": "^3.1.5", "strip-json-comments": "^3.1.1" }, @@ -905,9 +905,9 @@ } }, "node_modules/@eslint/js": { - "version": "9.39.4", - "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.4.tgz", - "integrity": "sha512-nE7DEIchvtiFTwBw4Lfbu59PG+kCofhjsKaCWzxTpt4lfRjRMqG6uMBzKXuEcyXhOHoUp9riAm7/aWYGhXZ9cw==", + "version": "9.39.5", + "resolved": "https://registry.npmjs.org/@eslint/js/-/js-9.39.5.tgz", + "integrity": "sha512-QywQuszQh77pIXCsq998c8hbhSTI/azTty1Z6N53dmAudKHhy573j3yvRLsX2BSp8YpLtoCEG8E9DJe+8zUh4A==", "dev": true, "license": "MIT", "engines": { @@ -2321,27 +2321,27 @@ "license": "MIT" }, "node_modules/@supabase/auth-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/auth-js/-/auth-js-2.108.2.tgz", - "integrity": "sha512-tNaQmBgodDZwgB40mRwVbxFy8IDYwjdpcZ0BYrWiwlULCSQoJj4QoG4zgJT7QRPXcqipefNOzvO/qAu4dF98ag==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/auth-js/-/auth-js-2.110.2.tgz", + "integrity": "sha512-Qj7a6EDP+AMMQFWqGv+qFa8r6re//dk+qQI5bA0KK+PZmnI3JPu97TDeNt6SMiQ2FkklP79hP2yDFYSnA989OA==", "license": "MIT", "dependencies": { "tslib": "2.8.1" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@supabase/functions-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/functions-js/-/functions-js-2.108.2.tgz", - "integrity": "sha512-RNUX8EiBy3iLwAX19jtRzLyePnl11/fHcgwDHLnpKcDSXt/5qBnh3LUwAtIjT21Q66QsmNUR2esrHziLCpNubw==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/functions-js/-/functions-js-2.110.2.tgz", + "integrity": "sha512-ZjjqrXpxM9/rE+eAtZxiK45EWy9EBoJQ322Q5Y75LccYQNh212neHTgXP/o4MIzmH0LNXT8UzvTZtQOfOzyoeQ==", "license": "MIT", "dependencies": { "tslib": "2.8.1" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@supabase/phoenix": { @@ -2351,28 +2351,28 @@ "license": "MIT" }, "node_modules/@supabase/postgrest-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/postgrest-js/-/postgrest-js-2.108.2.tgz", - "integrity": "sha512-GQ28/Y8hk3CFmkb3kXH1h/AQx6JIYSQfO0CJMRVBcEKZoNy6C45cXAZ4fcJvRC5Id0cs6xnkUV0+c0rIocigsw==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/postgrest-js/-/postgrest-js-2.110.2.tgz", + "integrity": "sha512-++LBmcIMwCtgO4tISQUmo9+2xkRwHQqS8ZKMCnhXLe9P8k8YQRXuMoh/RiSzQSoev8gqet0W7yOboW0cUxnt0Q==", "license": "MIT", "dependencies": { "tslib": "2.8.1" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@supabase/realtime-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/realtime-js/-/realtime-js-2.108.2.tgz", - "integrity": "sha512-aAGxCSUemZvQIibnCdvNvgaKib28I4rfrNjKbQ9cG1uBLwUsI7hVpGXgEbypCCDhLjQlDTAiJlu7rgljYUT73g==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/realtime-js/-/realtime-js-2.110.2.tgz", + "integrity": "sha512-z3jTOTPgyn6E3r6dVOOQ10He4yAMB2czjFw7xVdX3s16MHElna5rY1gVaePs0NIo6xvtMYbtmOXlFaFt/ePLpg==", "license": "MIT", "dependencies": { - "@supabase/phoenix": "^0.4.2", + "@supabase/phoenix": "0.4.4", "tslib": "2.8.1" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@supabase/ssr": { @@ -2388,32 +2388,32 @@ } }, "node_modules/@supabase/storage-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/storage-js/-/storage-js-2.108.2.tgz", - "integrity": "sha512-TVZPQxXGxY2+A6yTtm77zUHsh70lBhYUEaJL8RQC+BghcX/ygiMG/rmXrNVBce30/WAeNPa8FiG8HbqlGeV05g==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/storage-js/-/storage-js-2.110.2.tgz", + "integrity": "sha512-EhsRSwSnmQefKJsAxoRUZ0hvHr92ECM8DDGAKR5z0HdoJx4heI60PjHUTruVNZxKX6XeobLGDyLud020Bw1iwg==", "license": "MIT", "dependencies": { "iceberg-js": "^0.8.1", "tslib": "2.8.1" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@supabase/supabase-js": { - "version": "2.108.2", - "resolved": "https://registry.npmjs.org/@supabase/supabase-js/-/supabase-js-2.108.2.tgz", - "integrity": "sha512-hFhnPveb5JQg4a0QYicM0swT253YHMdfeRAl2BKHOlI5VAzuHxUGSr8RbwNLYNPauWOgQMS1H8sz8bvYlgwUfQ==", + "version": "2.110.2", + "resolved": "https://registry.npmjs.org/@supabase/supabase-js/-/supabase-js-2.110.2.tgz", + "integrity": "sha512-r9q9w4ZQ6mOjh36aqUNFSisBF611vzpO8JphBESr2Q1SWvmGFQeI7Jq7Y+PaNMZ6Zszz+S2yTlJStCpnaMSnQg==", "license": "MIT", "dependencies": { - "@supabase/auth-js": "2.108.2", - "@supabase/functions-js": "2.108.2", - "@supabase/postgrest-js": "2.108.2", - "@supabase/realtime-js": "2.108.2", - "@supabase/storage-js": "2.108.2" + "@supabase/auth-js": "2.110.2", + "@supabase/functions-js": "2.110.2", + "@supabase/postgrest-js": "2.110.2", + "@supabase/realtime-js": "2.110.2", + "@supabase/storage-js": "2.110.2" }, "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" } }, "node_modules/@swc/helpers": { @@ -2825,9 +2825,9 @@ "license": "MIT" }, "node_modules/@types/node": { - "version": "24.13.2", - "resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.2.tgz", - "integrity": "sha512-fRa09kZTgu8o71KFcDjUFuc7F+dEbZYZmkI0mg5YBTRs0yMKjYHsq/c0urDKeDb+D5qVgXOdFcuu+DZPKOITwA==", + "version": "24.13.3", + "resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.3.tgz", + "integrity": "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q==", "dev": true, "license": "MIT", "dependencies": { @@ -4998,9 +4998,9 @@ } }, "node_modules/eslint": { - "version": "9.39.4", - "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.4.tgz", - "integrity": "sha512-XoMjdBOwe/esVgEvLmNsD3IRHkm7fbKIUGvrleloJXUZgDHig2IPWNniv+GwjyJXzuNqVjlr5+4yVUZjycJwfQ==", + "version": "9.39.5", + "resolved": "https://registry.npmjs.org/eslint/-/eslint-9.39.5.tgz", + "integrity": "sha512-DgZS62aPLXKlnxILS/AYCoRvHaZeXceIzlXPkkGGzJWSow1aEk0lbTlxUSlyjC8jcaKxAdOnTDz+o1JFSBsyjw==", "dev": true, "license": "MIT", "dependencies": { @@ -5009,8 +5009,8 @@ "@eslint/config-array": "^0.21.2", "@eslint/config-helpers": "^0.4.2", "@eslint/core": "^0.17.0", - "@eslint/eslintrc": "^3.3.5", - "@eslint/js": "9.39.4", + "@eslint/eslintrc": "^3.3.6", + "@eslint/js": "9.39.5", "@eslint/plugin-kit": "^0.4.1", "@humanfs/node": "^0.16.6", "@humanwhocodes/module-importer": "^1.0.1", @@ -5484,9 +5484,9 @@ } }, "node_modules/fast-check": { - "version": "4.8.0", - "resolved": "https://registry.npmjs.org/fast-check/-/fast-check-4.8.0.tgz", - "integrity": "sha512-GOJ158CUMnN6cSahsv4+ExARvIDuzzinFjkp0E9WtiBa5zcVeLozVkWaE4IzFcc+Y48Wp1EDlUZsXRyAztQcSg==", + "version": "4.9.0", + "resolved": "https://registry.npmjs.org/fast-check/-/fast-check-4.9.0.tgz", + "integrity": "sha512-7ms6T7SybUev/PQITciI0yLM2pOSFy5zpG8Ty7tQofcVaQUvrMXp6CBwqF6fThLCLOrfBtuHAtwq6Yu4XPCllg==", "dev": true, "funding": [ { @@ -6691,9 +6691,9 @@ "license": "MIT" }, "node_modules/js-yaml": { - "version": "4.2.0", - "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.2.0.tgz", - "integrity": "sha512-ePWsvanv0DWuDRsW8dnt+R4jQ31SCRCQ7hhNcPXZPsoBZiemuZNYGf7adZdqX2D86j6rvKp3RpCxVTSb8WQlOw==", + "version": "4.3.0", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.0.tgz", + "integrity": "sha512-1td788aAnnZ5qs7V2QIRl1owjtYpbKt749Y3xauqQgwIIGF/xXWz1wMTEBx5O3LK3lXLVuqXPdPxj2BoFHaW9Q==", "dev": true, "funding": [ { @@ -7276,9 +7276,9 @@ } }, "node_modules/lucide-react": { - "version": "1.22.0", - "resolved": "https://registry.npmjs.org/lucide-react/-/lucide-react-1.22.0.tgz", - "integrity": "sha512-c9o3l0PiNcgOQDW4F31BEYHudE7kgxVt3o30qMl36ZPwTxXlGB4QnLilhERvVM4uh/pl5MDyY1/gzZSYcHDtBg==", + "version": "1.24.0", + "resolved": "https://registry.npmjs.org/lucide-react/-/lucide-react-1.24.0.tgz", + "integrity": "sha512-YT6mBD8lGKkg4nM39enlm94/sfJIiW0YKUT60fBy4YK8tai31ylg1VhGNWxkpSKHo9UagfnZqwIff3HTDQwXeA==", "license": "ISC", "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" @@ -7546,12 +7546,6 @@ } } }, - "node_modules/next/node_modules/@next/env": { - "version": "16.2.10", - "resolved": "https://registry.npmjs.org/@next/env/-/env-16.2.10.tgz", - "integrity": "sha512-zLPxg9M0MEHmygpj5OuxjQ+vHMiy/K7cSp74G8ecYolmgUWw0RwN02tF56npup/+qaI8JB97hQgS/r2Hb6QwVA==", - "license": "MIT" - }, "node_modules/node-exports-info": { "version": "1.6.0", "resolved": "https://registry.npmjs.org/node-exports-info/-/node-exports-info-1.6.0.tgz", @@ -7734,9 +7728,9 @@ } }, "node_modules/openai": { - "version": "6.45.0", - "resolved": "https://registry.npmjs.org/openai/-/openai-6.45.0.tgz", - "integrity": "sha512-5DQVNErssk0afNpTTHUm/qZPU4iKR9OYdNid8Ib4puq4gHNNvGWZht2zY4h9a8JMF949Ik6m8gQutllVPbjdnw==", + "version": "6.46.0", + "resolved": "https://registry.npmjs.org/openai/-/openai-6.46.0.tgz", + "integrity": "sha512-DFg6jEPT2RO+oAyXtddeUJU8zkGy1OQ1AjGzNIJUMQG03TTqvCpy9tBpQ+2VVVnvrl3E56F8GEin2JYtWpITtA==", "license": "Apache-2.0", "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", @@ -8295,9 +8289,9 @@ } }, "node_modules/postcss": { - "version": "8.5.15", - "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.15.tgz", - "integrity": "sha512-FfR8sjd4em2T6fb3I2MwAJU7HWVMr9zba+enmQeeWFfCbm+UOC/0X4DS8XtpUTMwWMGbjKYP7xjfNekzyGmB3A==", + "version": "8.5.16", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.16.tgz", + "integrity": "sha512-vuwillviilfKZsg0VGj5R/YwwcHx4SLsIOI/7K6mQkWx+l5cUHTjj5g0AasTBcyXsbfTgrwsUNmVUb5xVwyPwg==", "funding": [ { "type": "opencollective", @@ -8333,9 +8327,9 @@ } }, "node_modules/prettier": { - "version": "3.9.4", - "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.4.tgz", - "integrity": "sha512-yWG/o/4oJfo036EKAfK6ACAoDOfHeRHx4tuxkfBZiauURiaSmYwlpOr5LQqKtIkRD2z1PLteme2WoxEnj4tHTg==", + "version": "3.9.5", + "resolved": "https://registry.npmjs.org/prettier/-/prettier-3.9.5.tgz", + "integrity": "sha512-/FVl766LpUfB5vXgCYOYa0MeV/441Ia99AeICQIQFTY/Nw0roZwULcXpku5i1/m5kt/baz+s4Zogspd839HSMg==", "dev": true, "license": "MIT", "bin": { @@ -9455,9 +9449,9 @@ "license": "0BSD" }, "node_modules/tsx": { - "version": "4.22.4", - "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.22.4.tgz", - "integrity": "sha512-X8EX+XV4QR5xCsrgxaED954zTDfY8KqlDtskKEL0cHhyS/P8b4IFOvGDQpsC9Q1XnLq915wEfwwY/zzskCtmhg==", + "version": "4.23.0", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.0.tgz", + "integrity": "sha512-eUdUIaCr963q2h5u3+QwvYp0+eqPvn+egeqZUm0hwERCqqx1E3kK5ehbGCvqSE5MQAULr67ww0cA3jKc3YkM1w==", "dev": true, "license": "MIT", "dependencies": { diff --git a/scripts/check-codex-autofix-workflow.mjs b/scripts/check-codex-autofix-workflow.mjs index 1e68045bd2..4d59357d6e 100644 --- a/scripts/check-codex-autofix-workflow.mjs +++ b/scripts/check-codex-autofix-workflow.mjs @@ -87,6 +87,7 @@ const requiredTriggerAndPermissionChecks = [ " types: [created]", " contents: read", " issues: write", + " models: read", " pull-requests: read", `uses: actions/github-script@${githubScriptPin} # v9.0.0`, "github.event.pull_request.state == 'open'", diff --git a/src/app/api/health/route.ts b/src/app/api/health/route.ts index 0e2521f509..b53de63130 100644 --- a/src/app/api/health/route.ts +++ b/src/app/api/health/route.ts @@ -1,5 +1,6 @@ import { NextResponse } from "next/server"; import { env, isDemoMode } from "@/lib/env"; +import type { AnswerSloSnapshot } from "@/lib/observability/answer-slo"; import { allowDeepHealthProbe } from "@/lib/deep-probe-auth"; export const runtime = "nodejs"; @@ -14,17 +15,29 @@ export async function GET(request: Request) { openaiConfig: env.OPENAI_API_KEY ? "ok" : "missing", }; + let slo: AnswerSloSnapshot | null = null; if (deep) { if (!allowDeepHealthProbe(request)) { checks.supabase = "unauthorized"; } else if (supabaseConfigured && !isDemoMode()) { try { - const [{ createAdminClient }, { probeSupabaseHealth }] = await Promise.all([ + const [{ createAdminClient }, { probeSupabaseHealth }, { answerSloSnapshot }] = await Promise.all([ import("@/lib/supabase/admin"), import("@/lib/supabase/health"), + import("@/lib/observability/answer-slo"), ]); - const health = await probeSupabaseHealth(createAdminClient()); + const admin = createAdminClient(); + const health = await probeSupabaseHealth(admin); checks.supabase = health.ok ? "ok" : "error"; + if (health.ok) { + // Reliability telemetry only — a failure here must NOT flip liveness to + // 503, so it stays out of `checks` (which gates `ready`). + try { + slo = await answerSloSnapshot(admin); + } catch { + slo = null; + } + } } catch { checks.supabase = "error"; } @@ -44,6 +57,7 @@ export async function GET(request: Request) { timestamp: new Date().toISOString(), uptimeSeconds: Math.round(process.uptime()), checks, + ...(slo ? { slo } : {}), }, { status: ready ? 200 : 503, headers: { "Cache-Control": "no-store" } }, ); diff --git a/src/app/privacy/page.tsx b/src/app/privacy/page.tsx new file mode 100644 index 0000000000..b0c9e25f16 --- /dev/null +++ b/src/app/privacy/page.tsx @@ -0,0 +1,139 @@ +import type { Metadata } from "next"; +import type { ReactNode } from "react"; + +import { ClinicalBadge } from "@/components/clinical-dashboard/clinical-badge"; +import { + cn, + eyebrowText, + raisedCard, + searchPageCanvas, + searchPageContainer, + searchPageShell, +} from "@/components/ui-primitives"; +import { privacyCopy } from "@/lib/ui-copy"; + +export const metadata: Metadata = { + title: "Privacy & data handling — Clinical KB", + description: + "How Clinical KB handles your questions and documents: what is collected, where it is stored, what is sent to OpenAI, and how long it is kept.", +}; + +// Plain-language, code-accurate transparency page (APP-5 collection notice + APP-1 +// openness). This summarizes the engineering posture documented in +// docs/privacy-impact-assessment.md. It is not legal advice, and it is not a +// substitute for a formal privacy policy reviewed by a privacy officer — that +// review, plus the OpenAI cross-border agreement, is the outstanding PIA-1 step. +type Section = { heading: string; body: ReactNode }; + +const SECTIONS: Section[] = [ + { + heading: "What this tool is", + body: ( + <> + Clinical KB is a knowledge base over clinical reference material (guidelines, drug monographs, protocols). It is{" "} + not a patient-record system and does not ask you for patient data. The main privacy + consideration is therefore incidental patient information that a clinician might type into a free-text + question. + + ), + }, + { + heading: "What is collected", + body: ( + <> + Your free-text questions, the generated answers, your account identity (email / sign-in), and any documents you + upload. Questions and uploaded documents may contain identifiable information if you put it there — which is why + the notice above asks you not to. + + ), + }, + { + heading: "How your questions are handled", + body: ( + <> + Question text is not stored in raw form by default. Before anything is logged it is replaced + with a keyed one-way hash, so the log tables hold a pseudonym rather than your words. Generated answers are + stored against your account only, and both are automatically deleted on a schedule (see retention). + + ), + }, + { + heading: "Where your data is stored", + body: ( + <> + All stored data — documents, indexed text, answers, logs, and sign-in — lives in a database and file store + hosted in Sydney, Australia (AWS ap-southeast-2). Uploaded files sit in private buckets and are + reachable only through short-lived (10-minute) links minted after an ownership check. + + ), + }, + { + heading: "What is sent to OpenAI (United States)", + body: ( + <> + To understand a question and generate an answer, the question text and the matching excerpts from your library + are sent to OpenAI in the United States. This is the only point where data leaves Australia. + OpenAI is asked not to retain these requests in its dashboard, and no patient identifiers are added by the app — + but any details you type are transmitted, so do not enter identifiable patient details. + + ), + }, + { + heading: "How long it is kept", + body: ( + <> + Logged (hashed) questions and answers are purged after 30 days; retrieval telemetry after{" "} + 90 days. Uploaded documents and their index remain until you remove them. + + ), + }, + { + heading: "Security", + body: ( + <> + Data is scoped to your account, storage is private, and access is checked on every request. These are + engineering safeguards; they do not replace your own judgement about what is safe to enter. + + ), + }, +]; + +export default function PrivacyPage() { + return ( +
+
+
+
+

{privacyCopy.pageEyebrow}

+

+ {privacyCopy.pageTitle} +

+

+ A plain-language summary of how Clinical KB handles your questions and documents. It reflects how the + software behaves today; it is not legal advice, and a formal privacy policy is still under review. +

+
+ +
+
+
+ +
+

+ Do not enter identifiable patient details (names, dates of birth, record numbers) into your questions. + Your question text is sent to OpenAI in the United States to generate an answer. +

+
+
+ + {SECTIONS.map((section) => ( +
+

{section.heading}

+

{section.body}

+
+ ))} +
+
+
+ ); +} diff --git a/src/components/clinical-dashboard/answer-status.tsx b/src/components/clinical-dashboard/answer-status.tsx index c80946f4e8..e620b6de41 100644 --- a/src/components/clinical-dashboard/answer-status.tsx +++ b/src/components/clinical-dashboard/answer-status.tsx @@ -1,11 +1,12 @@ "use client"; import { Clipboard, ClipboardCheck, History, MessageSquareText, Search, ShieldCheck, Upload } from "lucide-react"; +import Link from "next/link"; import { AnswerSuggestionChips } from "@/components/clinical-dashboard/answer-suggestion-chips"; import { ModeHomeTemplate, ModeHomeVerificationFooter } from "@/components/mode-home-template"; import { cn, floatingControl, sourceCard } from "@/components/ui-primitives"; -import { answerEmptyState, answerLoading, copyButton } from "@/lib/ui-copy"; +import { answerEmptyState, answerLoading, copyButton, privacyCopy } from "@/lib/ui-copy"; export function CopyButton({ label, @@ -89,6 +90,20 @@ export function AnswerEmptyState({ + {/* APP-5 collection notice + on-query PHI reminder, kept to one calm line + at the point of collection; full detail on /privacy. See PIA-1/PIA-5. */} +

+ {privacyCopy.composerNotice}{" "} + + {privacyCopy.composerLinkLabel} + +

} /> diff --git a/src/lib/observability/answer-slo.ts b/src/lib/observability/answer-slo.ts new file mode 100644 index 0000000000..ddfa52f637 --- /dev/null +++ b/src/lib/observability/answer-slo.ts @@ -0,0 +1,85 @@ +// Answer-pipeline SLO snapshot for the deep /api/health probe. +// +// The repo's defining failure mode is *silent degradation*: hybrid retrieval RPCs +// can die while the app keeps returning 200s from fallbacks (see +// docs/observability-slos.md). This turns the two load-bearing reliability signals +// from that doc — the `hybrid_rpc_errors` rate and the degraded/source-only rate — +// into a scrapeable counter so host-native alerting can poll them instead of +// running the SQL by hand. It only aggregates `rag_queries.metadata` (never raw +// query text, which is redacted at write time), and runs behind the secret-gated +// deep probe. The cache hit-rate counter from that doc is intentionally NOT here — +// it needs in-process instrumentation of the retrieval hot path (rag-cache.ts) and +// is tracked as a separate follow-up. + +export type AnswerSloSnapshot = { + windowMinutes: number; + totalQueries: number; + hybridRpcErrorQueries: number; + degradedQueries: number; + // 0..1; 0 when there were no queries in the window (avoid divide-by-zero noise). + hybridRpcErrorRate: number; + degradedRate: number; +}; + +type CountResult = { count: number | null; error: unknown }; + +// Minimal structural view of the PostgREST count query we use. The Supabase admin +// client is assignable to this (same pattern as src/lib/supabase/health.ts), so the +// route passes it directly and tests pass a small fake. +type SloCountBuilder = PromiseLike & { + gt(column: string, value: string): SloCountBuilder; + not(column: string, operator: string, value: null): SloCountBuilder; +}; + +export type SloProbeClient = { + from(table: string): { + select(columns: string, options: { count: "exact"; head: true }): SloCountBuilder; + }; +}; + +function rate(numerator: number, denominator: number) { + return denominator > 0 ? numerator / denominator : 0; +} + +/** + * Count answered queries in the trailing window and how many carried a + * `hybrid_rpc_errors` map or a `fallback_reason` (degraded/source-only). Throws on + * a query error so the caller can mark the probe degraded rather than report a + * falsely-healthy zero. + */ +export async function answerSloSnapshot(client: SloProbeClient, windowMinutes = 60): Promise { + const sinceIso = new Date(Date.now() - windowMinutes * 60_000).toISOString(); + const base = () => client.from("rag_queries").select("*", { count: "exact", head: true }).gt("created_at", sinceIso); + + const [total, hybrid, degraded] = await Promise.all([ + base(), + base().not("metadata->hybrid_rpc_errors", "is", null), + base().not("metadata->>fallback_reason", "is", null), + ]); + + for (const result of [total, hybrid, degraded]) { + if (result.error) { + if (result.error instanceof Error) throw result.error; + // Supabase surfaces a plain PostgrestError object ({ message, code, ... }), + // not an Error — pull the message out rather than stringify to "[object Object]". + const message = + result.error && typeof result.error === "object" && "message" in result.error + ? String((result.error as { message?: unknown }).message ?? "") + : String(result.error); + throw new Error(message || "answer SLO count query failed"); + } + } + + const totalQueries = total.count ?? 0; + const hybridRpcErrorQueries = hybrid.count ?? 0; + const degradedQueries = degraded.count ?? 0; + + return { + windowMinutes, + totalQueries, + hybridRpcErrorQueries, + degradedQueries, + hybridRpcErrorRate: rate(hybridRpcErrorQueries, totalQueries), + degradedRate: rate(degradedQueries, totalQueries), + }; +} diff --git a/src/lib/rag.ts b/src/lib/rag.ts index b4d11feeb5..7890ec0cdb 100644 --- a/src/lib/rag.ts +++ b/src/lib/rag.ts @@ -3998,6 +3998,7 @@ async function answerQuestionWithScopeUncoalesced( embedding_cache_hit: search.telemetry.embedding_cache_hit, supabase_rpc_latency_ms: search.telemetry.supabase_rpc_latency_ms, rerank_latency_ms: search.telemetry.rerank_latency_ms, + hybrid_rpc_errors: search.telemetry.hybrid_rpc_errors, retrieval_strategy: search.telemetry.retrieval_strategy, weighted_top_score: search.telemetry.weighted_top_score, rrf_top_score: search.telemetry.rrf_top_score, @@ -4103,6 +4104,7 @@ async function answerQuestionWithScopeUncoalesced( embedding_cache_hit: search.telemetry.embedding_cache_hit, supabase_rpc_latency_ms: search.telemetry.supabase_rpc_latency_ms, rerank_latency_ms: search.telemetry.rerank_latency_ms, + hybrid_rpc_errors: search.telemetry.hybrid_rpc_errors, retrieval_strategy: search.telemetry.retrieval_strategy, weighted_top_score: search.telemetry.weighted_top_score, rrf_top_score: search.telemetry.rrf_top_score, @@ -4750,6 +4752,7 @@ ${qualityRetryInstruction}` embedding_cache_hit: search.telemetry.embedding_cache_hit, supabase_rpc_latency_ms: search.telemetry.supabase_rpc_latency_ms, rerank_latency_ms: search.telemetry.rerank_latency_ms, + hybrid_rpc_errors: search.telemetry.hybrid_rpc_errors, context_pack_latency_ms: contextPackLatencyMs, context_pack_cache_hits: contextPackCacheHits, answer_retry_count: answerRetryCount, @@ -4902,6 +4905,7 @@ ${qualityRetryInstruction}` embedding_cache_hit: search.telemetry.embedding_cache_hit, supabase_rpc_latency_ms: search.telemetry.supabase_rpc_latency_ms, rerank_latency_ms: search.telemetry.rerank_latency_ms, + hybrid_rpc_errors: search.telemetry.hybrid_rpc_errors, context_pack_latency_ms: contextPackLatencyMs, retrieval_strategy: "generation_fallback", weighted_top_score: search.telemetry.weighted_top_score, diff --git a/src/lib/ui-copy.ts b/src/lib/ui-copy.ts index c0950bcbcf..718f95582e 100644 --- a/src/lib/ui-copy.ts +++ b/src/lib/ui-copy.ts @@ -105,6 +105,19 @@ export const emptyStates = { }, } as const; +// Privacy / data-handling copy — the APP-5 collection notice shown at the query +// composer plus the labels for the /privacy transparency page. Wording is a +// plain-language engineering summary of docs/privacy-impact-assessment.md; it is +// not legal advice. See PIA-1 / PIA-5. The composer line is deliberately short so +// it reads as guidance, not a wall of text; the full detail lives on /privacy. +export const privacyCopy = { + composerNotice: "Answers are generated by OpenAI in the US — don't enter identifiable patient details.", + composerLinkLabel: "Privacy & data handling", + noticeAriaLabel: "Privacy and data-handling notice", + pageEyebrow: "Privacy", + pageTitle: "Privacy & data handling", +} as const; + // User-visible error / status messages. export const errorCopy = { searchSetupNotReady: "Search setup not ready.", diff --git a/tests/answer-slo.test.ts b/tests/answer-slo.test.ts new file mode 100644 index 0000000000..26c6dfc32b --- /dev/null +++ b/tests/answer-slo.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, it } from "vitest"; + +import { answerSloSnapshot, type SloProbeClient } from "@/lib/observability/answer-slo"; + +// Fake PostgREST count builder: from().select().gt() is the "total" query; adding +// .not(column,...) narrows it to the hybrid-error or degraded count based on the +// filtered column. Awaiting resolves to { count, error }. +function fakeClient(counts: { total: number; hybrid: number; degraded: number }, error?: unknown): SloProbeClient { + const build = (filter: "total" | "hybrid" | "degraded") => { + const builder = { + gt: () => builder, + not: (column: string) => build(column.includes("hybrid_rpc_errors") ? "hybrid" : "degraded"), + then: (resolve: (value: { count: number | null; error: unknown }) => unknown) => + resolve({ count: error ? null : counts[filter], error: error ?? null }), + }; + return builder; + }; + return { from: () => ({ select: () => build("total") }) } as unknown as SloProbeClient; +} + +describe("answerSloSnapshot", () => { + it("computes counts and rates over the window", async () => { + const snapshot = await answerSloSnapshot(fakeClient({ total: 20, hybrid: 3, degraded: 2 }), 60); + expect(snapshot).toMatchObject({ + windowMinutes: 60, + totalQueries: 20, + hybridRpcErrorQueries: 3, + degradedQueries: 2, + }); + expect(snapshot.hybridRpcErrorRate).toBeCloseTo(0.15, 5); + expect(snapshot.degradedRate).toBeCloseTo(0.1, 5); + }); + + it("reports zero rates (not NaN) when there are no queries in the window", async () => { + const snapshot = await answerSloSnapshot(fakeClient({ total: 0, hybrid: 0, degraded: 0 })); + expect(snapshot.totalQueries).toBe(0); + expect(snapshot.hybridRpcErrorRate).toBe(0); + expect(snapshot.degradedRate).toBe(0); + }); + + it("throws when a count query errors so the probe is not falsely healthy", async () => { + await expect( + answerSloSnapshot(fakeClient({ total: 0, hybrid: 0, degraded: 0 }, { message: "boom" })), + ).rejects.toThrow(/boom/); + }); +}); diff --git a/tests/codex-autofix-workflow.test.ts b/tests/codex-autofix-workflow.test.ts index c7c2f4f731..875e66e04d 100644 --- a/tests/codex-autofix-workflow.test.ts +++ b/tests/codex-autofix-workflow.test.ts @@ -229,6 +229,16 @@ describe("Codex auto-resolve workflow guard", () => { expect(result.output).toContain("immutable github-script pin"); }); + it("rejects workflows missing models read permission", () => { + const workflow = originalWorkflow.replace(" models: read\n", ""); + expect(workflow).not.toBe(originalWorkflow); + + const result = runGuard(workflow); + + expect(result.status).toBe(1); + expect(result.output).toContain("models: read"); + }); + it("allows findings to quote marker text without being mistaken for requests", () => { const workflow = originalWorkflow.replace( ` const sourceBody = (reviewComment.body || "").trimStart(); diff --git a/tests/health-route.test.ts b/tests/health-route.test.ts index bae0304257..cad01debb9 100644 --- a/tests/health-route.test.ts +++ b/tests/health-route.test.ts @@ -52,6 +52,18 @@ describe("GET /api/health", () => { expect(body.checks).toMatchObject({ supabaseConfig: "missing", openaiConfig: "missing" }); }); + it("gates the deep probe without a token and omits the slo snapshot", async () => { + mockEnv({ configured: true }); + const { GET } = await import("../src/app/api/health/route"); + + const response = await GET(healthRequest("?deep=1")); + const body = await payload(response); + + expect(response.status).toBe(503); + expect(body.checks).toMatchObject({ supabase: "unauthorized" }); + expect(body.slo).toBeUndefined(); + }); + it("does not leak secret values in the payload", async () => { mockEnv({ configured: true }); const { GET } = await import("../src/app/api/health/route"); diff --git a/tests/supabase-schema.test.ts b/tests/supabase-schema.test.ts index a9d11ee863..28cca68d08 100644 --- a/tests/supabase-schema.test.ts +++ b/tests/supabase-schema.test.ts @@ -116,6 +116,10 @@ const ragRetrievalLogsRetentionMigration = readFileSync( new URL("../supabase/migrations/20260702120000_rag_retrieval_logs_retention.sql", import.meta.url), "utf8", ).replace(/\s+/g, " "); +const ragQueryMissesRetentionMigration = readFileSync( + new URL("../supabase/migrations/20260708120000_rag_query_misses_retention.sql", import.meta.url), + "utf8", +).replace(/\s+/g, " "); const liveDatabaseDriftMigration = readFileSync( new URL("../supabase/migrations/20260705230000_reconcile_live_database_drift.sql", import.meta.url), "utf8", @@ -891,7 +895,11 @@ describe("Supabase Preview replay guards", () => { }); it("guards pg_cron retention schedules for preview branches without cron.job", () => { - for (const sql of [ragQueriesRetentionMigration, ragRetrievalLogsRetentionMigration]) { + for (const sql of [ + ragQueriesRetentionMigration, + ragRetrievalLogsRetentionMigration, + ragQueryMissesRetentionMigration, + ]) { expect(sql).toContain("to_regnamespace('cron')"); expect(sql).not.toMatch(/select cron\.unschedule\(jobid\) from cron\.job/); expect(sql).not.toMatch(/select cron\.schedule\(/);