diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ad4aa2ba01..5e76a606ee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -188,6 +188,14 @@ jobs: if: needs.changes.outputs.static_heavy_changed == 'true' run: npm run check:upload-limit-parity + # The generated lexicon review is a clinical-facing artefact. It was in the + # local verify:pr-local chain but in no CI job, so it rotted on main and + # failed every local preflight while every required check stayed green + # (#333). Same static-heavy condition verify-pr-local.mjs already uses. + - name: Medication lexicon report freshness + if: needs.changes.outputs.static_heavy_changed == 'true' + run: npm run check:medication-lexicon-report + - name: GitHub Actions pin check if: needs.changes.outputs.workflow_changed == 'true' run: npm run check:github-actions diff --git a/docs/branch-review-records/d50a877214261c376d150342864090de1d5f90e53c3187b03f4165df702e9b5d.record.md b/docs/branch-review-records/d50a877214261c376d150342864090de1d5f90e53c3187b03f4165df702e9b5d.record.md new file mode 100644 index 0000000000..a4f7f63491 --- /dev/null +++ b/docs/branch-review-records/d50a877214261c376d150342864090de1d5f90e53c3187b03f4165df702e9b5d.record.md @@ -0,0 +1 @@ +| 2026-08-15 | claude/ledger-guard-ci-followups | 8fe2a50b9bce29e24710fdf72ce8b5f187c0569e | Lexicon-report CI freshness and database remediation sequencing | Fixed P1 prerequisite order and P2 generated-report scope gap; merged current base with no conflicts | git diff --check; ci-change-scope direct-report classification and self-test; GitHub Actions pin check; remediation-order/workflow-contract assertions; ledger-inbox; outstanding-issues; branch-review-ledger; ledger-write-discipline; focused Vitest unavailable (node_modules absent) | diff --git a/docs/database-remediation-plan.md b/docs/database-remediation-plan.md index 9bb9a18561..277b07e09c 100644 --- a/docs/database-remediation-plan.md +++ b/docs/database-remediation-plan.md @@ -3,14 +3,36 @@ Owner: operator (Josh) + specialist session. Source findings: open ledger `#248` (whose causal conclusion awaits a read-only history and audit check) and the queued P1 live-drift follow-up. Companion evidence: live-drift Actions runs -`30763871562` (2026-08-02) and -`31330856982` (2026-08-09), PR #1614, `supabase/migrations/20260804110240_restore_rag_search_health_indexes.sql`. - -**Scope.** Fixes, in dependency order: the live-vs-repo schema gap (21 missing indexes, 2 -unexpected indexes, 10 diverged `match_*` RPC bodies), the unresolved cause of the affected -migration history, drift-detection routing, and the surrounding database debt (`#102`, `#011`, `#036`, -`#022`, `#025`, `#056`/`#057`, `#183`, `#188`/`#196`–`#200`, `#191`, `#098`/`#099`). Ends with -standing protections so unverified history repairs remain visible and cannot silently mask drift. +`30763871562` (2026-08-02), `31330856982` (2026-08-09) and +`31813064485` (2026-08-14, the current measurement), PR #1614, +`supabase/migrations/20260804110240_restore_rag_search_health_indexes.sql`. + +> **Status as of 2026-08-14.** Phase 0 is **complete** (routing, post-migration trigger, evidence +> file, and the forced-dispatch proof — which auto-created issue **#1963**). Phase 1 is **partial**: +> 1.1 and 1.3 were run in an owner-authorised incident window, but **1.2, the RPC divergence +> dossier, is outstanding and is the gate for Phase 3**. Phase 4 is **partial**: the two +> retrieval-critical trigram indexes were restored, ~20 remain. +> +> **Next step is Phase 1.2, not Phase 3.** Complete and review the RPC divergence dossier first, +> then run the Phase 2 staging-parity rehearsal. Only after both prerequisites are complete may an +> appropriately approved Phase 3 reconciliation proceed. Before starting any phase, check the +> open-PR list for the surface first (`#292`): Phase 0 was independently built twice on 2026-08-14, +> and Phases 3 and 4 spend approved production windows and eval-canary budget rather than just tokens. + +**Scope.** Fixes, in dependency order: the live-vs-repo schema gap, the unresolved cause of the +affected migration history, drift-detection routing, and the surrounding database debt (`#102`, +`#011`, `#036`, `#022`, `#025`, `#056`/`#057`, `#183`, `#188`/`#196`–`#200`, `#191`, `#098`/`#099`). +Ends with standing protections so unverified history repairs remain visible and cannot silently mask +drift. + +The schema gap, measured 2026-08-14 (superseding the 2026-08-09 figures this plan was written +against): + +| Category | 2026-08-09 | 2026-08-14 | +| ----------------------------- | ---------- | ------------------ | +| diverged `match_*` RPC bodies | 10 | **10 — unchanged** | +| `missing_live` indexes | 21 | **20** | +| `unexpected_live` indexes | 2 | **2 — unchanged** | **Standing rules for every phase.** No hosted mutation without explicit approval for that phase. Never raw-SQL a drift fix — every live change is codified (migration + `schema.sql` mirror + @@ -63,7 +85,7 @@ hypothesis into a named, evidence-backed conclusion. noise** (whitespace/qualifier only). No edits. This is the highest-stakes unknown in the whole plan — live retrieval behaviour may currently depend on bodies the repo does not contain. -1.3 **Index inventory & sizing.** For the 21 missing and 2 unexpected indexes: owning-table +1.3 **Index inventory & sizing.** For the 20 missing and 2 unexpected indexes: owning-table `pg_relation_size`, and `EXPLAIN (ANALYZE, BUFFERS)` for the known hot queries (`documents` title ILIKE, `document_chunks` content search, `rag_retrieval_logs` miss scan). These are the before-measurements for Phase 4, including the `#231` latency hypothesis. diff --git a/docs/database-remediation-playbook.md b/docs/database-remediation-playbook.md index d9d089aac1..0c8deceb92 100644 --- a/docs/database-remediation-playbook.md +++ b/docs/database-remediation-playbook.md @@ -19,19 +19,32 @@ migrations show that history advanced, but neither point distinguishes skipped D history from indexes that were created and later dropped. The Phase 1 read-only history and audit check must establish that cause before the row is closed or remediation is attributed to it. -**Current live state** (from scheduled `live-drift.yml` Actions run `31330856982`, 2026-08-09): +**Current live state** (from `live-drift.yml` Actions run `31813064485`, 2026-08-14 — this +supersedes the 2026-08-09 run `31330856982` the plan was originally written against): -- 21 `missing_live` indexes across many migrations — tables: `audit_logs` (2), `api_rate_limits`, - `document_chunks` (2, incl. `document_chunks_content_trgm_idx`), `document_images` (3), - `document_index_quality`, `document_index_units`, `document_publication_approvals`, - `document_summaries`, `documents` (2, incl. `documents_title_trgm_idx`), +- 20 `missing_live` indexes across many migrations — tables: `audit_logs` (2), `api_rate_limits`, + `document_chunks`, `document_images` (3), `document_index_quality`, `document_index_units`, + `document_publication_approvals`, `document_summaries`, `documents`, `image_caption_cache`, `indexing_v3_agent_jobs`, `ingestion_job_stages`, `medication_records`, `rag_aliases`, `rag_queries`, `rag_query_misses`, `storage_cleanup_jobs`. + **`documents_title_trgm_idx` and `document_chunks_content_trgm_idx` are no longer among them** — + both were restored in the 2026-08-14 incident window and re-verified `indisvalid`/`indisready`. - 2 `unexpected_live` indexes: `document_table_facts_document_id_idx`, `storage_cleanup_jobs_owner_id_idx`. -- `def_hash` mismatches on 10 `match_*` retrieval RPCs (protected RAG surface; live bodies vs - repo — direction unknown until diffed). -- The weekly `live-drift` run has been red since 2026-07-26 with no notification routing. +- `def_hash` mismatches on 10 `match_*` retrieval RPCs — **unchanged, and entirely outstanding** + (protected RAG surface; live bodies vs repo — direction unknown until diffed). This is now the + highest-stakes remaining unknown and the reason Phase 1.2 comes next. +- Drift-failure routing is **live**: a failed run creates or updates the pinned issue + "Live drift check failing" (currently **#1963**) and a green run closes it. + +**Phase status.** Phase 0 complete. Phase 1 partial — 1.1 and 1.3 done in the owner-authorised +incident window, **1.2 (RPC dossier) outstanding and gating Phase 3**. Phase 4 partial — two +indexes restored, ~20 remain. **Next: complete Phase 1.2, then the Phase 2 staging-parity +rehearsal; only then may an appropriately approved Phase 3 proceed.** + +**Before starting any phase, check the open-PR list for the surface** (`#292`). Phase 0 was built +twice independently on 2026-08-14; a duplicate in Phase 3 or 4 wastes an approved production window +and eval-canary budget, not just tokens. **Prior repair to imitate.** PR #1614 / migration `20260804110240_restore_rag_search_health_indexes.sql` is the approved pattern: operator prebuilds indexes with `CREATE INDEX CONCURRENTLY` outside any @@ -70,7 +83,10 @@ mistakes. Opus is sufficient for Phases 0, 2, 4, 5, 7 — execution against this --- -## Phase 0 — Enablement (repo-only; no approval window needed) · Opus · 2–4 h +## Phase 0 — Enablement (repo-only; no approval window needed) · Opus · 2–4 h — **COMPLETE 2026-08-14** + +Delivered in PRs #1938, #1939 and #1951; the forced-dispatch proof is Actions run `31813064485`, +which auto-created issue #1963. The prompt below is retained as history. Deliverables: drift-failure routing, post-migration drift trigger, evidence file scaffold. Definition of done: PR merged; `check:github-actions` and `verify:pr-local` green; a forced @@ -119,7 +135,7 @@ conclusion for `#248`; no writes performed. > classify each as live-ahead, repo-ahead, or normalization noise, quoting the decisive diff hunks > — this is a protected RAG surface, so classification accuracy matters more than speed, and any > ambiguous diff is recorded as UNCLASSIFIED with the ambiguity explained, never guessed; (3) for -> the 21 missing and 2 unexpected indexes, record owning-table pg_relation_size and run EXPLAIN +> the 20 missing and 2 unexpected indexes, record owning-table pg_relation_size and run EXPLAIN > (ANALYZE, BUFFERS) for the documents title ILIKE query, the document_chunks content search, and > the rag_retrieval_logs miss scan as before-baselines. Write all evidence with dates and run IDs > into docs/audit/live-drift-forensics-2026-08.md, update the live-drift tracking item, commit, push, PR (docs-only; @@ -169,7 +185,8 @@ allowlist entries); eval evidence attached for any behaviour-changing deploy. ## Phase 4 — Index restoration (approved off-peak production window) · Opus · 2–3 h active -Prerequisites: Phases 1–3. Deliverables: 21 indexes restored + validated, 2 unexpected indexes +Prerequisites: Phases 1–3. Deliverables: the ~20 still-missing indexes restored + validated (the two +trigram indexes were already restored on 2026-08-14), 2 unexpected indexes dispositioned, guard migrations landed, live-drift green. Definition of done: green live-drift dispatch output pasted; `search_schema_health()` still `ok: true`. diff --git a/scripts/ci-change-scope.mjs b/scripts/ci-change-scope.mjs index 431749391f..03307a9e6d 100644 --- a/scripts/ci-change-scope.mjs +++ b/scripts/ci-change-scope.mjs @@ -121,6 +121,11 @@ const docPatterns = [ /^LICENSE(?:\..*)?$/i, ]; +// This Markdown file is generated from the medication interaction lexicon. A +// direct edit must run its freshness check; otherwise the ordinary docs-only +// classification would let a stale clinical-facing report through CI. +const generatedMedicationLexiconReport = "docs/medication-interaction-lexicon-review.md"; + const workflowPatterns = [ ".github/workflows", ".github/actions", @@ -371,6 +376,7 @@ function isExecutableWorkflowSurfacePath(filePath) { * YAML/policy under workflow surfaces stay light; executable files there do not. */ function isRecognisedLightPath(filePath) { + if (filePath === generatedMedicationLexiconReport) return false; if (pathMatches(filePath, docPatterns)) return true; if (!pathMatches(filePath, workflowPatterns)) return false; return !isExecutableWorkflowSurfacePath(filePath); @@ -413,6 +419,7 @@ function classify(files, { readLedger = readFlakeLedger } = {}) { const docsOnly = normalized.length > 0 && normalized.every((file) => pathMatches(file, docPatterns)) && + !normalized.includes(generatedMedicationLexiconReport) && !sourceChanged && !workflowChanged; const workflowOnly = workflowChanged && !staticHeavyChanged; @@ -767,6 +774,12 @@ function selfTest() { build_changed: false, lockfile_changed: false, }); + assertScope("generated-medication-lexicon-report-stays-heavy", [generatedMedicationLexiconReport], { + docs_only: false, + docs_changed: true, + static_heavy_changed: true, + coverage_changed: true, + }); assertScope("tests-only", ["tests/rag-routing.test.ts"], { source_changed: true, coverage_changed: true, diff --git a/tests/ci-cache-safety.test.ts b/tests/ci-cache-safety.test.ts index ffd387b525..480df51044 100644 --- a/tests/ci-cache-safety.test.ts +++ b/tests/ci-cache-safety.test.ts @@ -99,6 +99,12 @@ describe("CI cache safety", () => { expect(workflow).toContain("run: npm run check:verification-plan"); }); + it("runs the generated medication lexicon freshness check through static-heavy scope", () => { + expect(workflow).toMatch( + /name: Medication lexicon report freshness\n\s+if: needs\.changes\.outputs\.static_heavy_changed == 'true'\n\s+run: npm run check:medication-lexicon-report/, + ); + }); + it("does not repeat focused workflow contracts inside the full coverage run", () => { expect(workflow).toContain( "if: needs.changes.outputs.workflow_changed == 'true' && needs.changes.outputs.coverage_changed != 'true'",