Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/outstanding-issues.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -54,6 +54,7 @@ removed after current-main verification; it is not missing recommended work.
| 2 | `#053` | A1 | Operator — legal/privacy | Start now; finish before real patient use/privacy-approved release | 4–8 hours internal; 1–6 weeks elapsed | Execute DPAs; decide ZDR/residency; obtain cache behavior in writing; review subprocessors; obtain APP 8 and APP 5/1 counsel sign-off. Do not change public copy before approval. |
| 3 | `#069` | A3 | Specialist — retrieval latency | After hosted apply of PR #1133 migrations; approval-gated live profile | 30–60 min | Operator applies `20260724120000`/`20260724130000`/`20260724130100`, then re-profiles `match_document_table_facts_text` (~70ms-class plans). Stop without mutating ranking or unpaid evals. Cloud agent blocked: no DB URL / MCP auth; live profile hit Unregistered API key. |
| 4 | `#019` | A2 | Specialist — RAG answer pipeline | Local reproducer now; behavior change after `#023` disposition | 0.5–1 day reproducer | Reproduce admission-source loss in the fallback layer using PR #1096’s source shape. Any behavior change needs protected review and an approved baseline/post canary; stop if independently non-reproducible. |
| 5 | `#024` | A2 | High — browser/Next diagnostics | Provider-free macOS Safari host available | 1–2 hours | Reproduce document-source fallbacks in Safari/STP without Playwright interception; capture `_rsc` response evidence. Treat as an app defect only if native Safari reproduces; otherwise return to the harness. Never suppress `pageerror` or change CORS without proof. |
| 6 | `#022` | A2 | Operator — clinical governance + Specialist | Decision-ready | 1–2 hours policy; 0.5–1 day first ten | Decide BMJ attestation policy and review the ten highest-impact local documents. Record reviewer/evidence/time; stop after ten and remeasure warning debt. |
| 7 | `#023` | A2 | Specialist — RAG/browser diagnostics | After hosted dependency audit is green | 1–2 hours | Capture the skipped Firefox/WebKit scheduled datapoint and disposition the human irrelevant-at-10 labels. Retrieval and answer artifacts are already compared under resolved #051; do not spend on another RAG run. |
| 8 | `#018` | A2 | Specialist — clinical RAG/retrieval | After `#023`, one mechanism at a time | 1–2 days diagnosis | Give lithium, ADHD, and metabolic residuals separate current-main reproducers and candidates. Behavior canaries require approval; stop any item without a deterministic reproducer or on regression. |
Expand DownExpand Up@@ -108,6 +109,7 @@ removed after current-main verification; it is not missing recommended work.
| #021 | P3 | rec | E-3d H2 residual: strong/comparison generation discards | approx. 6 generation attempts per full 44-case run still fail the final quality gate and fall to extractive on strong-route comparison/complex shapes (the designed-conservative outcome). PARKED: weakest cost/benefit on the queue — a wave (approx. $2-4 pair + reviewer cycle) to shave seconds off a few hard cases. Revisit only if latency/waste complaints or a cheaper lever appears. | E-3c design record; runs #59-#61 diagnostics | 2026-07-21 |
| #022 | P2 | task | Source-governance metadata refresh (operator) | **Worklist generated 2026-07-22 ($0, read-only): `docs/source-governance-refresh-worklist-2026-07-22.md`.** Reframed - this is NOT 59 clinical reviews. Of the 124 documents surfacing in canary top results, 59 are review-required, and **38 (64 pct) are the BMJ published-reference tier all sitting at `clinical_validation_status: unverified`** - one attestation-policy decision, not 38 reviews. The remaining 21 are genuine local WA health-service reviews (FSH 7, NMHS 4, CAMHS 3, AKG 2, KEMH 2, RPBG 2, RKPG 1), mostly `document_status: review_due`. Burn-down: top-10 documents clear 44 pct of flagged slots, top-20 clear 66 pct. Next: decide the BMJ attestation policy, then attest local docs by visibility (start `Clozapine Management by GP (NMHS)`, 22 slots at rank 1). | runs #61/#57 Source Governance data; `docs/source-governance-refresh-worklist-2026-07-22.md` | 2026-07-21 |
| #023 | P2 | task | Complete scheduled browser and labeling disposition | The 2026-07-26 canary artifacts are read and disposed under resolved #051. Scheduled CI run `30216361999` failed its existing production dependency audit before the dependent Firefox/WebKit job, while production Chromium passed. After the already-open dependency PRs clear the audit, capture one scheduled/manual browser-matrix datapoint; separately record the human decision for the stable irrelevant-at-10 set. Do not rerun or spend on RAG for this item. | runs `30216191889`/`30216361999`; Dependabot PRs #1268/#1269; session 2026-07-27 | 2026-07-21 |
| #024 | P2 | issue | WebKit e2e `_rsc` prefetch access-control errors | PR #1205 narrowed catch-all interception and duplicate navigation, but Next 16.2.11 still raises `_rsc` access-control `pageerror`s after document-source fallbacks: `/documents/source?id=<demo-id>&page=2&chunk=safety%20plan` → `/documents/<demo-id>?page=2&chunk=safety+plan`; `/documents/source/evidence?id=not-a-uuid&page=2` → `/documents/search`. The invalid-id failure survived removing every Playwright route; Chromium passed both. **Next:** on a provider-free macOS host, run both URLs in stable Safari and Safari Technology Preview without interception, capture console text plus `_rsc` status/access-control headers, and compare Playwright WebKit with routing on/off. Treat as an app defect only if native Safari reproduces; otherwise return to the harness. Never suppress `pageerror` or change CORS without native evidence. | PRs #1179/#1205; current-main local WebKit evidence; session 2026-07-28 | 2026-07-28 |
| #025 | P2 | task | Activate the three webhooks (operator secrets) | Merged (#968/#1100) + deployed but inert — verified live: `POST /api/webhooks/railway` returns `503 webhook_not_configured`; the Supabase document-change trigger exists but lacks both activation inputs. To turn on: (1) Railway ΓåÆ set `RAILWAY_WEBHOOK_SECRET` + add the `?token=…` webhook URL; (2) set `SLACK_WEBHOOK_URL`/`DISCORD_WEBHOOK_URL` in BOTH the Railway **app/server env** and **GitHub repo secrets**; (3) set one matching document-change secret in the Railway app env as `SUPABASE_INGESTION_WEBHOOK_SECRET` and in Supabase Vault as `ingestion_webhook_secret`, then set the per-environment database GUC `app.ingestion_webhook_base_url` to the deployed app origin. Each path fails closed until fully configured, so this is pure ops. See `docs/webhooks.md` for verification and rotation. | sessions 2026-07-22/24; PRs #968/#1100; docs/webhooks.md | 2026-07-22 |
| #027 | P3 | rec | External uptime monitor independent of GitHub/Railway | `live-domain-monitor.yml` runs on GitHub's cron, so it won't run in exactly the outage it should catch (Actions or the deploy itself down). Add an off-platform synthetic monitor (UptimeRobot / Better Stack / Checkly) hitting `/api/health` with a webhook alert. Provider setup, not code. | session 2026-07-22 webhook review | 2026-07-22 |
| #028 | P3 | rec | Runtime error tracking (Sentry or similar) | No error tracking in the repo — production exceptions on `psychiatry.tools`, including how often `RAG_PROVIDER_MODE=auto` silently degrades to source-only, are invisible. Weigh adding `@sentry/nextjs` (dependency + DSN secret + instrumentation) vs cost; alert ΓåÆ chat/issue. Provider-backed; needs explicit sign-off before adding the dependency. | session 2026-07-22 webhook review | 2026-07-22 |
Expand All@@ -134,7 +136,6 @@ Move resolved rows here with the resolution date and a one-line outcome. Keep th
| #051 | task | Stabilise the live answer-quality canary before more RAG tuning | Compared scheduled run `30216191889` with baseline `30018289898` without a dispatch or new spend: both retained 36/36 retrieval, recall 1.0, MRR@10 0.8921 and irrelevant@10 0.0917; answer grounded/unsupported rates stayed 1.0, citation failure improved to zero, and p95 improved to 17,003 ms. The repeated admission-source loss remains isolated as #019; no RAG tuning is justified by this monitoring result. | 2026-07-27 |
| #054 | task | Reconcile local and hosted secrets/config | Completed production names-only reconciliation on 2026-07-27. The correctly identified primary checkout received distinct gitignored local safety/query-hash/deep-probe values. A hardened checker now pins GitHub to `BigSimmo/Database` and Railway to the live production project/environment plus `Database`/`worker`, catches multiline schema and `.env.example` drift, and verifies GitHub secrets/variables and per-service Railway contracts without emitting provider values. All required names passed; the Ops Digest workflow is active with a successful scheduled run; both Railway services have later successful deployments; Supabase names-only proof found the expected cron/Vault configuration. Value equality remains deliberately unobservable, staging stays #056, webhook activation stays #025, and legal/ZDR work stays #053. | 2026-07-27 |
| #064 | task | Reconcile the preserved browser and contrast patch | Landed via PR #1250 squash `b91b4600171be08198e92bcf19b7d67e8207cb2f`. Opacity-free disabled Previous/Continue styling plus native-disabled/focus/axe Playwright coverage is on `main`. Historical `agent/formulation-disabled-contrast` remained unrecovered; conflicted PRs #1219/#1223/#1226/#1231/#1249 were closed without merge. | 2026-07-26 |
| #024 | issue | WebKit e2e `_rsc` prefetch access-control errors | Fixed in PR #1205 after current-main reproducers showed catch-all interception plus duplicate viewport navigation racing same-origin WebKit RSC prefetches. The harness now intercepts external HTTP(S) only, directly proves blocking/recording, navigates each route once, and leaves `pageerror` unfiltered. On `64e46036`, the local Chromium/WebKit matrix passed 24/24 and `verify:pr-local` passed 3,431 tests, build/bundle checks, and 36 offline RAG fixtures. | 2026-07-25 |
| #081 | issue | Open PR #1196 would undo the #030 alias tightening | Closed as no longer live: PR #1196 was closed 2026-07-25 as superseded by #913/current `main` (~680 commits behind, conflicting), and its successor #1198 does not touch `src/lib/eval-document-matching.ts`. The generalized alias-disjointness and single-document contracts landed in PR #1215 fail closed if any later branch re-adds the dual-listed admission aliases, so the regression route is guarded rather than watched. | 2026-07-25 |
| #077 | issue | Concurrent tasks can re-dirty the canonical primary checkout | Added cooperative primary-checkout write lease with dirty/operation fail-closed checks, stale-owner recovery, and lifecycle start/cleanup wiring; focused concurrency tests refuse a second primary writer while read-only/feature worktrees stay unblocked. | 2026-07-25 |
| #078 | task | Generate a deterministic reconciliation evidence pack | Added report-only atomic evidence pack with dispositions, markers, archive refs, bundle verify/hash, worktree counts, and local/base equality; fixture tests prove determinism/redaction and no false completion record on interrupt. | 2026-07-25 |
Expand Down
Loading