Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions docs/branch-cleanup-guide.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -121,6 +121,85 @@ credentials.
5. Record completed cleanup reviews with `npm run ledger:append -- --ref <branch> --head <full-sha> --scope branch-cleanup --outcome <o> --checks <c>`. The scope cell must be exactly `branch-cleanup` for a later sweep to treat it as complete; `branch-cleanup-deletion-pending` deliberately does not count.
6. Remove detached worktrees only when clean, unneeded, and absent from active `git worktree list` output.

## Dev Drive Stale Worktree Pruning (Inbox `ec356a7d`)

Worktrees accumulate across active multi-agent development fleets (e.g. on Dev Drive `D:` or secondary disks, where dozens of worktrees can hold ~19+ GB of duplicated `node_modules` at ~0.89 GB and ~50,000 files each). Stale worktrees compete for disk capacity, cause dependency drift, and contend on the cross-worktree test run coordinator lock (`scripts/run-heavy.mjs` / `scripts/test-run-lock.mjs`).

### Safety Rules for Worktree Pruning

- **Never pass `--force` to `git worktree remove`.** If git refuses removal due to untracked files, submodules, or uncommitted changes, that refusal is a critical safety signal, not an obstacle to override.
- **Never remove a worktree that is ahead of `origin/main` or has uncommitted work.** `git status --porcelain` in the candidate directory must be clean (`""`).
- **Never prune blind while agent sessions (Codex/Gemini/Claude) are active.** Sessions may have created new commits, modified files, or acquired test locks since the initial scan.
- **Ignore `C:` session worktrees entirely.** These paths belong to active Codex and Antigravity chat sessions.

### Step-by-Step Worktree Pruning Workflow

1. **Confirm fleet quiescence:**
Ensure no background test runner, dev server, or agent session is active or holding an execution lease.

2. **Scan landed candidates (list-only preflight):**
Run `clean-worktree.mjs` with `--merged` and `--squashed` to identify worktrees whose branch has landed on `origin/main` (either by direct ancestry or squash-merge patch-id equivalence):

```bash
node scripts/clean-worktree.mjs --merged --squashed
```

_(Note: `--merged` and `--squashed` are strictly list-only by default; they will never delete anything without `--remove`.)_

3. **Evaluate confidence ratings on each candidate:**
- `proven`: Every commit on the branch is reachable from `origin/main` (`git merge-base --is-ancestor`).
- `inferred from patch-id, corroborated`: The branch's full diff matches a squashed commit on `origin/main`, and all changed files are byte-identical to `origin/main`.
- `inferred from patch-id, NOT fully corroborated`: The branch diff matched a squashed commit, but some changed files differ from `origin/main` (frequently normal base churn on append-only docs like `docs/outstanding-issues.md`). Review differences with `git diff origin/main...<branch>` before removing. Skip any candidate where you cannot confirm all unique work is landed.

4. **Execute bounded safe removal:**
After verifying candidate confidence and confirming that trees are clean and not ahead of `origin/main`:

```bash
# Bounded batch removal
node scripts/clean-worktree.mjs --merged --squashed --remove --batch-size 5
```

Or remove an individual verified worktree directly:

```bash
git worktree remove <path>
```

5. **Prune disconnected or orphaned metadata:**

```bash
npm run clean:worktree
```

## Merge-Loss Audit Verification (#311, #324)

Merge loss occurs when merge commits or manual conflict resolutions silently revert changes from previously landed PRs without failing tests (for example, bad merge commit `acf78bf` reverting PRs #1800, #1803, #1804, #1796, and #1811 because the reverts took each PR's tests at the same time).

The merge-loss audit tool was promoted into `scripts/audit-merge-loss.mjs` (#311) and provides verification of post-merge integrity (#324):

```bash
# Run advisory merge-loss audit over the default 14-day window on origin/main
npm run audit:merge-loss

# Audit a custom window or target branch
npm run audit:merge-loss -- --since 30 --ref origin/main

# Output findings as structured JSON
npm run audit:merge-loss -- --json

# Run in strict mode (exits non-zero on any finding)
npm run audit:merge-loss -- --strict

# Run self-tests directly
node scripts/audit-merge-loss.mjs --self-test
```

### Interpreting Findings

- **Advisory by design:** The audit exits `0` by default because an intentional revert is byte-identical at blob level to an accidental merge resolution revert. A finding is a prompt for human review, not an automatic defect.
- **Classification hierarchy:** Findings sort merge-resolution reverts first (high likelihood of accidental loss) before single-parent commits (usually deliberate reverts with explanatory commit messages).
- **Remediation:** If an accidental revert is confirmed, restore the lost files on a new branch and submit a PR to re-land the changes.

## Final Verification

After each cleanup pass:
Expand Down
8 changes: 8 additions & 0 deletions docs/codex-review-protocol.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -13,6 +13,14 @@ Use this protocol for every Codex review, audit, bug hunt, PR review, release-re
- Route automatic repair only for high-risk paths, at least 10 changed non-test source files, at least 300 changed non-test source lines, or an explicit `codex-review` label. `skip-codex-review` always opts out, including when both labels are present. Small low-risk, docs-only, test-only, and generated-only changes should not receive the automatic repair request.
- Ready PRs must pass the trusted `PR policy` metadata check. It reads only the base-branch policy implementation, never executes PR code, and requires concrete verification plus risk/rollback evidence for high-risk changes.

## Review Bot Quota Conservation (Inbox `9864a5d7`)

PR churn and unthrottled reviews quickly exhaust review-bot spending caps and rate limits (e.g., CodeRabbit and Codex connector allowances; see Inbox `9864a5d7`). When quotas are exhausted, PRs land with zero automated review, increasing defect risk during churn spikes. To protect review-bot budget:

- **Skip repeat passes on unchanged SHAs**: Never re-review a commit, branch, or PR head whose HEAD SHA and scope have already been reviewed. Automated routines, babysit sweeps, and CI jobs must check prior review records and skip repeat reviews when the tree has not changed.
- **Resolve SHAs via `npm run ledger:lookup`**: Always verify review state using `npm run ledger:lookup -- <branch-or-ref> --scope "<scope>"`. It resolves the full 40-character commit SHA, matches against live rows, archives (`docs/archive/branch-review-ledger-*.md`), and immutable records (`docs/branch-review-records/*.record.md`), and prints an explicit `ALREADY REVIEWED` or `NOT REVIEWED at this HEAD` verdict. Never scan or eyeball ledger tables manually.
- **Enforce a single review pass per PR head (#328)**: Automated review is strictly limited to one pass per PR HEAD. Intermediate repair commits, formatting adjustments, or routine base syncs do not authorize automatic re-reviews without explicit human approval. Prevent review row thrashing or rows outliving completion (#328) by immediately appending an immutable record with `npm run ledger:append` upon completing a review.

## Review Output

- Lead with findings, ordered by severity: P0, P1, P2, then P3.
Expand Down
92 changes: 92 additions & 0 deletions docs/design-system-contract.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
# Design System Contract & Standards

This document specifies the blocking design system token rules, touch/tap target standards, and enforcement mechanisms for the Clinical KB application.

---

## 1. Overview & Authority

The system of record for design tokens, components, and architectural decisions is [`docs/design-system/`](./design-system/README.md) (GATES.md, SPEC.md, TOKENS.md, COMPONENTS.md).

All UI code merged into the codebase must satisfy the automated design system gates verified via:

```bash
npm run check:design-system-contract
npm run check:type-scale
npm run check:icon-scale
```

---

## 2. Blocking Token Rules

### 2.1 Colors & Semantic Palette

- **Tokens Only**: Raw CSS hex codes (e.g. `#007a78`, `#ffffff`), RGB/RGBA, HSL, and un-tokenized Tailwind color classes (e.g. `bg-white`, `text-slate-900`, `border-red-200`) are prohibited in components.
- **Variable Syntax**: All colors must use CSS custom properties defined in `src/app/globals.css` with semantic purpose:
- **Brand & Clinical Accent**: `var(--clinical-accent)`, `var(--clinical-accent-hover)`, `var(--clinical-accent-soft)`, `var(--clinical-accent-border)`
- **Surfaces & Borders**: `var(--surface)`, `var(--surface-subtle)`, `var(--surface-wash)`, `var(--surface-lux)`, `var(--border)`, `var(--border-subtle)`
- **Text Roles**: `var(--text)`, `var(--text-muted)`, `var(--text-heading)`, `var(--text-soft)`
- **Status & Safety Triads**: `--success-*`, `--warning-*`, `--danger-*`, `--info-*` (reserved exclusively for clinical/system status).
- **Focus Ring & Outlines**: `var(--focus)` for all keyboard and visible focus rings.
- **Raw Color Exemptions**: Strict and enumerated in `RAW_COLOR_EXEMPTIONS` in `scripts/design-system-contract-utils.mjs` (e.g., globals token definitions, brand mark SVG builder, diagnostic visualizations, OpenGraph art, printable patient/factsheet paper).

### 2.2 Typography Scale

- **Named Steps Only**: Font sizes must use the registered type steps in `@theme`:
- `text-3xs` (10px - absolute floor), `text-2xs` (11px), `text-xs` (12px), `text-sm-minus` (13px), `text-sm` (14px), `text-base-minus` (15px), `text-base` (16px), `text-lg-minus` (17px), `text-lg` (18px), `text-xl` (20px), `text-2xl-minus` (22px), `text-2xl` (24px).
- **Arbitrary Size Prohibited**: `text-[12px]`, `text-[13px]`, etc. are blocked by `npm run check:type-scale --strict`.
- **Declared Steps Usage**: Any type step declared in `@theme` must have production consumers (no dead or unselected type tokens).

### 2.3 Icon Scale

- Glyphs must use the dedicated `--spacing-icon-*` scale: `size-icon-xs` (12px), `size-icon-sm` (14px), `size-icon-md` (16px default), `size-icon-lg` (20px), `size-icon-xl` (24px).
- Enforced strictly by `npm run check:icon-scale --strict`.

### 2.4 Elevation, Edges, & Motion

- **Elevation**: Monotonic numeric scale `var(--e0)` through `var(--e4)`. No raw `box-shadow` values.
- **Edge Ownership**: Prohibits simultaneous `border-*` and `ring-*` styling on the same surface to prevent clipped or competing boundaries.
- **Motion Durations**: Transitions and animations must use standardized duration tokens (`var(--duration-fast)`, `var(--duration-normal)`) and respect `motion-reduce:`. Layout-property animation (e.g., width, height, padding) is disallowed except for explicitly audited phone-chrome transitions.

---

## 3. Touch & Tap Target Standards (#265, #321)

### 3.1 Minimum Touch Target Floor (48px)

- **Token**: `--spacing-tap` (48px), mapped to Tailwind classes `min-h-tap`, `min-w-tap`, and `size-tap`.
- **Target Applicability**: All primary and secondary interactive elements (buttons, links, form controls, summary disclosures, tabs, toolbar chips, filmstrip jump buttons) must guarantee a minimum 48px hit area along their primary touch axis.
- **Sub-Floor Prevention**: `interactiveTapFloorDeclarations` in `check:design-system-contract` mechanically scans intrinsic interactive tags (`a`, `button`, `input`, `select`, `summary`, `textarea`) and forbids un-prefixed sub-floor declarations (e.g., `min-h-8`, `min-h-10`).
- **No Downward Reductions**: Legacy `min-h-12` (48px) and `min-h-tap` (48px) must never be reduced to smaller arbitrary heights.

### 3.2 Secondary Navigation & Toolbar Chips (Case Study: #321)

- Secondary navigation items (such as the document figure filmstrip in `src/components/document-viewer/document-image-filmstrip.tsx`) must adhere to both token and touch targets:
- Enforce `min-h-tap` on chip buttons to allow rapid, error-free mobile and desktop interaction.
- Implement tokenized focus indicators: `focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-[color:var(--focus)]`.
- Color styling must reference semantic tokens (`var(--surface)`, `var(--surface-subtle)`, `var(--border)`, `var(--text-muted)`, `var(--clinical-accent)`).

### 3.3 Accessible Unavailable / Disabled States

- Where controls are rendered in an inert or data-unavailable state (e.g., a figure with no recorded PDF page number):
- Do **not** use the HTML `disabled` attribute if doing so breaks keyboard focusability and hides the reason for unavailability.
- Use `aria-disabled="true"` with `ignoreUnavailableActivation`.
- Provide an accessible description via `aria-describedby` linking to a screen-reader explanation (`sr-only` text), e.g., explaining why the action is unavailable.

---

## 4. Verification & Continuous Enforcement

Run the design system validation suite locally before submitting changes:

```bash
# Complete design system gate check (tokens, baselines, adoption, design-sync)
npm run check:design-system-contract

# Strict typography scale validation
npm run check:type-scale

# Strict icon scale validation
npm run check:icon-scale
```
8 changes: 6 additions & 2 deletions docs/launch-operator-runbook.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -28,6 +28,9 @@ Legend: **⏸ PAUSE** = provider action, needs your approval · **✅ verify** =
4. Staging soak + rollback rehearsal [Railway]
5. Production deploy [Railway]
6. Post-deploy: worker, registry seed, auth conn cap, observability wiring
7. Environment post-restore recovery controls (#326)
8. Operational notes & diagnostics (#248, #305, #315)
9. Ledger queue derivation & credential discipline (#327, #042)
```

---
Expand DownExpand Up@@ -170,9 +173,10 @@ Following a Supabase database restore or disaster recovery failover, verify all
4. **Cron Job Schedule Registration:** Verify pg_cron extensions and scheduled maintenance jobs (cache retention, query log sweeps) are active.
5. **Storage Bucket Policies:** Confirm private document storage buckets (`documents`, `document-images`) have active RLS policies preventing unauthenticated public reads.

## 8. Operational notes & diagnostics (#248, #305, #315)
## 8. Operational notes & diagnostics (#248, #305, #315, #102)

- **Search-Health Indexes (#248):** Ensure migration `20260705180000_search_schema_health.sql` is active on live and all 20 required indexes are present.
- **Search-Health Indexes (#248):** Ensure migration `20260705180000_reconcile_search_health_indexes.sql` is active on live and `search_schema_health()` reports `ok: true` with no `missing` entries (`npm run check:indexing`) — treat its `required_indexes` list as the recovery criterion, not a fixed count, since later migrations extend it.
- **Concurrent Document Index Recipe (#102):** When applying additive document index optimizations on a busy database (`documents_title_bare_trgm_idx`, `documents_file_name_bare_trgm_idx`, and `documents_status_id_idx`), pre-create indexes concurrently (`CREATE INDEX CONCURRENTLY IF NOT EXISTS`) before applying the committed migration and registering in `search_schema_health()` to avoid write lock contention. Validate each index with `pg_index.indisvalid`. Note that bare-column trigrams and composite `(status, id)` indexes on the RAG path are canary-gated due to unordered `LIMIT 12` selection in candidate retrieval ([operator-apply-performance-latency-remediation.md](operator-apply-performance-latency-remediation.md)).
- **Canary Latency & Cost Boundaries (#305):** Retrieval latency p90 SLO is ≤ 20s. Canary cost metrics provide lower-bound estimates without cache warmup.
- **UI Smoke Reporter Stranding (#315):** When debugging rare UI smoke test timeouts, inspect reporter stranding in Playwright hooks rather than assuming layout regressions.

Expand Down
42 changes: 42 additions & 0 deletions docs/observability-slos.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -125,6 +125,26 @@ recent pre-flag provider failures remain visible until they age out. Keep `degra
the broader source-only UI state and `fallback_reason` as diagnostic detail;
neither is narrow enough for provider health on its own.

### Sentry Production DB Span SLO (#183)

Production database query spans instrumented via Sentry PostgREST tracing
(`src/lib/observability/supabase-tracing.ts`) capture execution latency for
database RPCs and table queries without recording sensitive query parameters or
clinical text (`docs/error-tracking.md`).

- **Target / Filter:** Production database query spans (`span.op:db` where
`environment: production`).
- **SLO:** Database query span duration p95 ≤ 500 ms under normal production load.
- **Metric alert criteria:** Sentry metric alert triggers when production
database query span `p95(span.duration) > 500ms` over a **5-minute rolling window**.
- **Triage & Diagnostics:**
- Check Sentry Queries dashboard (**Dashboards → Sentry Built → Queries** and
**Explore → Traces**) to identify slow database operations, unindexed query
scans, connection pool exhaustion, or transaction lock contention.
- Cross-reference with `/api/health?deep=1` degradation counters (`slo`,
`cache`, `coalescing`) and Supabase project metrics to isolate backend query
slowdowns from application-layer bottlenecks.

## 3. Weekly production eval canary

`.github/workflows/eval-canary.yml` — scheduled weekly on Sunday at 18:00 UTC
Expand DownExpand Up@@ -234,6 +254,28 @@ regression triage does not relearn them:
downloaded artifacts — the durable trend record without any new
infrastructure.

### 3.2 Canary Latency & Cost SLOs (#305)

The weekly production canary evaluation (`.github/workflows/eval-canary.yml`)
tracks multi-stage retrieval latency against hard budgets and computes un-cached
cost lower bounds:

- **Retrieval Latency Budget (p90 ≤ 20 s):** Multi-stage retrieval evaluation
enforces an explicit latency budget of **p90 ≤ 20 s** across all evaluated cases
(with a 25 s per-case timeout ceiling). Answer generation flows nearing this
threshold risk triggering `OPENAI_ANSWER_TIMEOUT_MS` (30 s) timeouts and
falling back into degraded source-only responses.
- **Canary Cold Cost Lower Bounds:**
- The weekly canary executes 36 committed retrieval cases (plus captured
cases; embedding API calls occur only on forced-vector probes) and generates
answers for 44 golden cases.
- Cost metrics reported by the canary reflect a **cold lower bound** on
execution costs because the CI runner runs without pre-warmed in-memory or
shared response caches.
- Telemetry rows are recorded in `rag_queries` for timing and routing
analysis, but all evaluation queries are non-mutating and preserve
underlying knowledge base state.

## 4. Degradation counters on `/api/health` (shipped)

The §2 reliability SQL is now also a scrape. An **authorized deep probe** —
Expand Down
Loading
Loading