diff --git a/docs/branch-review-records/de0121cd3d30b30a3db40226b655a866957e65e04a043f71cb75b91ae612cdad.record.md b/docs/branch-review-records/de0121cd3d30b30a3db40226b655a866957e65e04a043f71cb75b91ae612cdad.record.md new file mode 100644 index 000000000..b243ec314 --- /dev/null +++ b/docs/branch-review-records/de0121cd3d30b30a3db40226b655a866957e65e04a043f71cb75b91ae612cdad.record.md @@ -0,0 +1 @@ +| 2026-08-17 | claude/database-coordination-chat-9c0730 | cef640f25aeee9b1fd8fc6c95a351ff1a6c348d3 | docs/database-remediation-coordination.md board re-baseline (#316), PR #2044 | coordinator self-review: docs-only, anchor corrected #312→#316, board rebuilt from main record | prettier --check pass; docs:check-links 1799 refs resolve | diff --git a/docs/database-remediation-coordination.md b/docs/database-remediation-coordination.md new file mode 100644 index 000000000..68ee928f7 --- /dev/null +++ b/docs/database-remediation-coordination.md @@ -0,0 +1,138 @@ +# Database remediation — coordination handover + +Written 2026-08-17 at the end of the originating investigation session (Claude Code, Fable). +Purpose: seed a **coordination chat** that oversees the whole remediation across many worker +chats. This file is the complete record of what that session established, decided, and produced — +a future coordinator needs nothing from the original chat transcript beyond this file and the two +documents it points to. + +## The document set (all committed via PR #1906) + +| Document | Role | +| -------------------------------------------- | --------------------------------------------------------------------------------------------------- | +| `docs/database-remediation-plan.md` | **Plan of record** — Phases 0–7, ordering rationale, approval map | +| `docs/database-remediation-playbook.md` | **Worker guide** — full context briefing + a copy-paste prompt per phase, with model recommendation | +| this file | **Coordinator handover** — history, decisions, status board, coordination rules | +| `docs/audit/live-drift-forensics-2026-08.md` | **Evidence file** (created in Phase 0) — all live findings land here | +| `docs/outstanding-issues.md` row `#316` (P1) | **Tracking anchor** — updated after every phase via `npm run issues:update` | + +> **Anchor correction (2026-08-17 coordinator review).** The originating session cited the anchor +> as `#312`; after inbox reconciliation the live-drift row is **`#316`** ("Live DB has 20 currently +> missing repo-defined indexes and 10 retrieval RPC bodies diverge …"). `#312` is an unrelated +> Playwright-revision row. Resolve by exact title, never by remembered number (playbook trap list). + +## Complete history of what the originating session did + +1. **Investigated ledger `#248`** ("why were 20260705180000's search-health indexes missing on + live despite applied history?"). Method: repo forensics + PR #1614's record + the + `live-drift.yml` GitHub Actions run history and failure logs. No hosted database access was + used or needed. +2. **Verdict (now recorded in `#248`'s archived row):** the migration's DDL **never executed on + live**; its history row was recorded out-of-band (mark-applied / history repair, or hand-apply + of only the function half). Proof: Supabase runs each migration in one transaction (persisted + partial apply impossible); the later 2026-07-06 migrations are behaviorally live, so the + history row had to pre-exist them; and two of its ten indexes are _still_ missing on live. + Excluded: "indexes manually dropped later" and "partial transaction". + > **Superseded by Phase 1.1 (2026-08-14, forensics file §1.1):** the live fingerprint shows + > `20260705180000` with `stmt_count 14` — its DDL **was** recorded as executed, so it does not + > carry the mark-applied signal. The "never executed" verdict above is history, not the current + > finding. Current position: created-then-dropped between 2026-07-05 and 2026-08-02, cause + > unattributed (dashboard audit-history pairing is owner action); `#248` stays open. +3. **Discovered the wider gap** from the 2026-08-09 scheduled drift run (Actions run + `31330856982`): **21 `missing_live` indexes** across many unrelated migrations, **2 + `unexpected_live` indexes** (`document_table_facts_document_id_idx`, + `storage_cleanup_jobs_owner_id_idx`), and **`def_hash` mismatches on 10 `match_*` retrieval + RPCs** (protected RAG surface; divergence direction unknown until diffed). The weekly drift + check had been red since 2026-07-26 with no notification routing; the 2026-07-19 run was green + only because the manifest was stale until its 2026-07-31 regeneration. +4. **Closed `#248`** with the verdict and **opened the P1 residual row (renumbered to `#316` after inbox reconciliation)**, via the repo's + `outstanding-issues.mjs` writer. +5. **Authored the plan and playbook** (documents above), designed the 8-phase remediation with + future-proofing, and pushed everything to branch `claude/fable-tasks-issues-49hnvp` / + **PR #1906**. + +Key evidence pointers a coordinator may need to re-verify claims: Actions runs `30763871562` +(2026-08-02 failure log — the 25-missing-index list including the four later repaired) and +`31330856982` (2026-08-09 — current state); PR #1614 (the 2026-08-04 repair: operator +`CREATE INDEX CONCURRENTLY` prebuild + fail-fast marker migration `20260804110240`). + +## Decisions already made (do not re-litigate in worker chats) + +- **Order:** routing/enablement → read-only forensics → staging rehearsal → RPC reconciliation → + index restoration → measurement → future-proofing → deferred debt. RPCs before indexes because + only RPCs can change _which_ chunks are retrieved (clinical risk); indexes only change speed. +- **Repair pattern:** every live fix is prebuilt concurrently, validated + (`indisvalid`/`indisready` + normalized `pg_get_indexdef`), then codified with a fail-fast + guard migration per `20260804110240`. **Never** raw-SQL a drift fix, never transactional index + builds on production, never a history repair without a guard migration. +- **`#102` stays separate:** the two trigram indexes in Phase 4 are the canonical + `20260705180000` definitions (completing a recorded migration); `#102`'s bare-column indexes + remain their own canary-gated work. +- **Model split:** Fable for Phases 1, 3, 6 and later `#191` (judgment on protected/privacy + surfaces); Opus for Phases 0, 2, 4, 5, 7 (execution against written spec). +- **Estimates:** ~15–25 focused hours total for Phases 0–6, spread across approval windows; + Phase 3 is the schedule variable (each repo-ahead RPC adds a canary at ~$1–2). + +## Status board (coordinator: update this table as phases land) + +_Board re-baselined 2026-08-17 by the coordination chat from `main` (`f5b093291`), the forensics +file, ledger `#316`, and the merged PR list — not from any worker chat's self-report._ + +| Phase | Status | PR / evidence | Notes | +| ------------------------------------------------------------------------------------ | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Prereq: merge PR #1906 | ☑ done 2026-08-13 | #1906 merged `4bbc4c5ca`; traps + parallel-session rules added by #2027 (`a75d226b0`) | Plan + playbook on `main`. This coordination file itself was still unmerged (branch tip `dc8969676`) until carried by the coordination chat's PR. | +| Prereq: `SUPABASE_ACCESS_TOKEN` secret (`#183`) | ☐ pending | repo-level secret names checked 2026-08-17: not present (env-level unverified) | Operator dashboard work; needed for migration-history CLI reads. Phase 1.1/1.3 ran anyway via the owner-authorised Supabase connector, so this gates CLI repair paths, not reads. | +| 0 — Enablement (routing, trigger, evidence file) | ☑ COMPLETE 2026-08-14 | #1938, #1939 (built twice — `#292`), #1951 (forced-dispatch proof, run `31813064485`, auto-issue #1963), #1978 (ledger/guard fixes) | Routing live; `tests/live-drift-workflow.test.ts` covers it offline. Issue #1963 still OPEN — last "still failing" comment 2026-08-17 (run `32011535189`) — expected until Phases 3–4 land. | +| 1 — Forensics (fingerprint, RPC dossier, EXPLAIN baselines) | ◐ PARTIAL | #1960, #1970 → `docs/audit/live-drift-forensics-2026-08.md` §1.1–1.3 | 1.1 done (`20260705180000` NOT mark-applied, 14 stmts; drop window 2026-07-05→08-02; cause unattributed, `#248` open). 1.3 done for whole schema (20 absent, 2 orphaned, zero invalid). **1.2 gating:** 6/16 mismatches proven noise, **10 UNCLASSIFIED**, hunks pending. | +| 1.2 — RPC dossier finish (read-only prod window) | ☐ NEXT | hypothesis in `#316` update (PR #2017): `SET work_mem` from `20260724000000` may classify 8/10 in one query | Fable. The only executable next step. Two `_v2` outliers need real diffs (canonical bodies named in `#316`). | +| 2 — Staging parity (`#056`) | ☐ not started | `#056` corrected by #2016: gap is **26 and widening** | Opus; staging window. Can run concurrently with 1.2 (different targets; `#056` is its ledger row). | +| 3 — RPC reconciliation | ☐ BLOCKED | — | Fable. Zero executable entries until 1.2 classifies (playbook forbids touching UNCLASSIFIED). Do not request a production window yet. | +| 4 — Index restoration | ◐ PARTIAL (incident) | #1960 §Phase 4: `documents_title_trgm_idx` + `document_chunks_content_trgm_idx` restored 2026-08-14 (648 kB / 68 MB, valid+ready) | Opus; off-peak prod window; blocked on 1–3. **20** still missing, 2 unexpected undispositioned, guard migration for the repaired pair still owed (plan 4.4), no green live-drift yet. | +| 5 — Measure (`#231` re-test) | ◐ PARTIAL (incident) | #1960 §Phase 5: text fast path 31,610 → 1,535 ms; hybrid 21,757 → 8,519 ms | `#231` verdict: retrieval latency was the binding cause in that window; hybrid ~8.5 s residual owned by remaining phases. Full EXPLAIN set + `check:production-readiness` pending. | +| 6 — Future-proofing (history probe, contract, ratchet) | ☐ not started | — | Fable; one migration deploy. Can be designed repo-side any time; deploy after Phase 4. | +| 7 — Deferred (`#022` `#025` `#036` `#191` `#196`–`#200` `#057` `#098`/`#099` `#011`) | ☐ not started | — | Each its own session, per playbook. | + +**Ledger state note (2026-08-17):** the `#316` "Phase 3 next" correction (PR #2017) and the `#056` +correction (PR #2016) are inbox requests; the reconcile that applies them to the canonical table is +open PR #2032 (`BEHIND` at review time). Until it lands, `npm run issues:report` still shows the +pre-correction prose — read the inbox request, not the row. + +## Coordination rules + +- **One worker chat per phase**, launched with that phase's prompt from the playbook, in the + recommended model. The coordinator never executes phases itself — it dispatches, reviews the + returned PR + evidence, updates this status board and `#316`, and decides the next dispatch. +- **Review each phase's return against its definition of done** (in the playbook) before + dispatching the next: pasted decisive gate lines, evidence appended to the forensics file, + `#316` updated, PR opened. A phase reporting success without pasted evidence is not done. +- **Approval brokering:** Phases 1–6 need Josh to grant a window in the worker chat's own + message (the playbook prompts state exactly which). The coordinator's job is to tell Josh which + window the next phase needs and what it authorizes — never to claim an approval on his behalf. +- **Escalations that must come back to Josh, not be absorbed:** any UNCLASSIFIED RPC diff + (Phase 1/3); any repo-ahead RPC (needs per-RPC canary approval); any migration that fails on + clean staging replay (Phase 2); any index that repeatedly fails its concurrent build (Phase 4); + any Phase 6 allowlist entry. +- **If a worker chat dies mid-phase:** the phase is resumable — everything durable is in the + forensics file, `#316`, and its branch. Relaunch with the same playbook prompt plus one line: + "resume from the recorded evidence; do not redo completed steps." +- **Standing safety rules bind every chat:** no hosted mutation without an explicitly granted + window; production ref is `sjrfecxgysukkwxsowpy` only; ledger files are mutated only via the + repo CLIs; `npm run format` + commit before every push; PR bodies carry the governance + preflight / `RAG impact:` lines where their diffs require them. + +## Prompt to start the coordination chat (paste verbatim) + +> You are the coordinator for the database remediation of BigSimmo/Database. Read, in order: +> docs/database-remediation-coordination.md (handover + status board), +> docs/database-remediation-plan.md (plan of record), and +> docs/database-remediation-playbook.md (per-phase worker prompts). Also read ledger row #316 via +> npm run issues:report (#316 is the anchor; the originating session's `#312` citation was wrong). Do not execute any phase yourself and do not touch Supabase. Your job: +> (1) tell me the current state of the status board and what the next action is, including any +> prerequisite or approval window I personally need to provide; (2) when I say go, give me the +> exact worker prompt to paste (from the playbook, adjusted only if earlier phases' evidence +> requires it) and which model to run it in; (3) when I bring back a worker chat's result, verify +> it against that phase's definition of done — check the PR exists, the evidence file was +> appended, and #316 was updated — then update the status board in +> docs/database-remediation-coordination.md and commit that edit; (4) escalate to me anything the +> coordination rules say must not be absorbed. Keep every status answer short: board state, next +> action, what you need from me.