diff --git a/.github/workflows/sast.yml b/.github/workflows/sast.yml index 5bd446815e..e11d61e524 100644 --- a/.github/workflows/sast.yml +++ b/.github/workflows/sast.yml @@ -51,3 +51,48 @@ jobs: --config p/react --config p/nodejs src worker scripts supabase/functions + + # Blocking SAST gate for the untrusted-document parsing surface (maturity X4). + # These paths open attacker-supplied uploads (PDF/DOCX/XLSX parsing, OCR, + # ingestion queue and its API routes), so ERROR-severity findings here fail + # the workflow instead of merely advising. The registry-rule mutability that + # keeps the repo-wide job above advisory is accepted for this narrow surface: + # ERROR-only + ~21 files keeps rule churn rare, and a triage documented in + # docs/maturity-backlog-workorders.md (X4) started the gate at zero findings. + # p/python covers worker/python — the code that actually parses untrusted PDFs. + semgrep-ingestion-gate: + name: Semgrep ingestion gate + runs-on: ubuntu-24.04 + timeout-minutes: 15 + container: + # Digest-pinned (unlike the advisory job's tag): this job blocks merges, + # so the engine itself must be immutable. Digest of semgrep/semgrep:1.168.0, + # the exact image the X4 triage ran. check-github-action-pins.mjs rejects + # a tag-only reference here. + image: semgrep/semgrep@sha256:59fbed6127ea7c5dde3ba6a85142733bb20ea9aaa36120c953904f1539aaf66e + steps: + - name: Checkout + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + persist-credentials: false + + - name: Semgrep scan (blocking) + # Deliberately NO continue-on-error: this job must block on findings. + # scripts/check-github-action-pins.mjs enforces both halves of the + # policy (advisory repo-wide job, blocking ingestion gate). + run: > + semgrep scan + --error + --severity ERROR + --oss-only + --metrics off + --config p/typescript + --config p/javascript + --config p/react + --config p/nodejs + --config p/python + worker + src/lib/ingestion*.ts + src/lib/extractors + src/app/api/ingestion + src/app/api/upload diff --git a/docs/branch-review-ledger.md b/docs/branch-review-ledger.md index 7b51dd4589..92288bbc9e 100644 --- a/docs/branch-review-ledger.md +++ b/docs/branch-review-ledger.md @@ -661,3 +661,4 @@ Use this ledger to prevent repeated branch and PR reviews when the reviewed HEAD | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (PR: matcher + artifact follow-ups) | ab145f6 | Remaining documented improvements implemented: word-boundary textContainsClinicalTerm + top-10 canary artifact rows | Matcher: boundaries + internal separators widened to any non-alphanumeric run — PROVEN strict superset by artifact replay on canary #53 (1,126 term×alias×result comparisons, 0 lost matches, 7 gained = exactly the previously-documented punctuation-joined occurrences: treatment,/mood,/(opioid/ptsd.[35]/ciwa-ar ×3). More-tolerant measurement cannot fail a passing case → weekly scheduled canary = free live confirmation. Exported + 3 direct unit-test groups (superset preservation, audit classes incl. line-broken 'ciwa- ar' and 'full-blood-count', substring-inside-word rejections). Artifact: topResultSummary 5→10 rows so rr@10/irrelevant@10 metrics' actual inputs are captured — unblocks the offline irrelevant@10 labeling audit next artifact. docs/rag-behaviour updated to implemented state. Phase E remains gated on separate approval. | Targeted vitest 59/59; npm run test 3028 passed / 1 known container pdf-budget artifact; lint+typecheck+prettier clean; audit script run recorded above; no provider calls | | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (PR: DB easy wins) | see PR head | User-requested database + process easy-wins pass: read-only Supabase advisor sweep (user-authorized) → canary-liveness probe + advisor-disposition docs; live mutations withheld for per-item confirmation | Advisors (live, read-only): security = 1 INFO (document_title_words RLS-no-policy = the deliberate fail-closed pattern — now comment-documented at the schema block so it is never 'fixed'); performance = ~33 unused-index INFOs + auth connection-strategy note → docs/db-maintenance.md TRIAGE list with retrieval-surface trgm indexes flagged RAG-protected (dropping = full canary protocol), owner-scoped indexes retained for multi-tenant design, operational candidates deferred (negligible benefit at corpus size). Implemented: ci.yml static-pr warn-only eval-canary staleness probe (actions:read, github-script pinned, >8 days → warning; never fails) — needed because #923 (2026-07-19) moved the canary cadence from daily to weekly Sunday 18:00 UTC, where a dropped fire would go unnoticed for a week (the failure-issue step only reacts to runs that happen). CORRECTION (CodeRabbit review on this PR): the initially recorded "2026-07-20 dropped Sunday fire" incident did not occur — 2026-07-20 is a Monday; the Sunday 2026-07-19 slot fired as scheduled run #48 (19:03 UTC, success), and under the weekly cron no 2026-07-20 slot existed. Probe stands as proactive hardening, staleness now measured from updated_at with a finite-timestamp guard. Presented for confirmation (NOT implemented): scheduled telemetry retention (purge:query-logs is owner-scoped + unscheduled; needs owner/window/policy decision), auth percentage connection strategy (dashboard config), any index drops. | check:github-actions PASS; check:ci-scope PASS; check:function-grants 28/28 PASS; docs:check-links 1034 PASS; supabase-schema vitest 66/66; lint+typecheck+prettier clean; Supabase access read-only only | | 2026-07-20 | claude/clinical-kb-pwa-review-asi3wb (restarted from e6cd6cb; PR: telemetry-retention correction) | 01368ff (+0144e69 main merge-in) | User-directed "implement your recommendation for all decisions" close-out: the scheduled-retention recommendation is RETRACTED as founded on a false premise — telemetry retention is ALREADY ACTIVE inside the database via pg_cron; docs corrected, nothing built, no live mutations | Fresh read-only cron.job verification (2026-07-20) matches docs/privacy-impact-assessment.md §6 exactly: jobid 11 purge-expired-rag-queries daily 03:30 (30d), jobid 12 purge-rag-retrieval-logs daily 03:00 (90d), jobid 13 purge-rag-query-misses daily 03:45 (90d), jobid 16 purge-rag-response-cache hourly (bounded 1000); v3 worker jobs present-inactive under backlog auto-toggle (jobid 10); obsolete unbounded cache job absent; audit_logs indefinite by design. db-maintenance.md "open decision" section replaced with the resolved state; purge:query-logs clarified as the MANUAL owner-scoped tool (not the retention mechanism). A GitHub-side weekly deleter would have duplicated pg_cron with window drift (a 90d rag_queries sweep can never out-delete the live 30d job). Remaining decisions stand as documented no-action: auth percentage connection strategy deferred to next instance resize; operational index drops not recommended. | prettier + docs:check-links PASS; Supabase access read-only (single cron.job SELECT, user-authorized read-only envelope); no workflow/schema/config changes | +| 2026-07-21 | claude/x4-sast-gate (PR #1012) | 0d4985e63 | Maturity X4: blocking SAST gate on the untrusted-document parsing surface | Triage-first per workorder: CI-pinned semgrep/semgrep:1.168.0 over worker/**, src/lib/ingestion*.ts, src/lib/extractors, src/app/api/{ingestion,upload} = 0 ERROR findings (24 TS rules/17 files; 55 Python rules/3 files) — gate starts green with no suppressions. Shipped `semgrep-ingestion-gate` job (no continue-on-error; container digest-pinned to the triage-verified 1.168.0 image) with p/python added for the worker OCR stack; repo-wide advisory job untouched. check-github-action-pins.mjs now enforces both policy halves fail-closed (advisory repo-wide / blocking-and-scoped gate / digest-pinned gate container). Residuals: registry-pack mutability accepted for the narrow surface; making the workflow a branch-protection required check is an operator decision outside this PR. | Exact gate command exit 0 in pinned container; check:github-actions (new assertions verified fail-closed); check:ci-scope; yaml-contract vitest 1/1; verify:cheap 3031 tests green. No provider calls (local Docker only). | diff --git a/docs/maturity-backlog-workorders.md b/docs/maturity-backlog-workorders.md index 4c1ef4c643..a14cae8cc6 100644 --- a/docs/maturity-backlog-workorders.md +++ b/docs/maturity-backlog-workorders.md @@ -85,7 +85,7 @@ confirmation) · `SATISFIED` (already true in the repo; no work needed). - **Risk:** HIGH (behavioural surface). One file per PR. - **Verification:** `npm run typecheck` + `npm run test` (+ `npm run verify:ui` for the components). -### X4 · SAST-blocking on the parser/ingestion path — `PROVIDER-GATED (triage-first)` +### X4 · SAST-blocking on the parser/ingestion path — `DONE` - **Outcome:** Semgrep ERROR findings block on the untrusted-PDF surface, not just advise. - **Approach:** add a path-scoped Semgrep job (targets `worker/**`, `src/lib/ingestion*`, @@ -94,6 +94,13 @@ confirmation) · `SATISFIED` (already true in the repo; no work needed). - **Files:** `.github/workflows/sast.yml`. - **Risk:** MEDIUM — could block on pre-existing findings if not triaged first. - **Verification:** a Semgrep run over the scoped paths reports zero ERROR before flipping the gate. +- **Shipped 2026-07-21 (this PR):** `semgrep-ingestion-gate` job in `sast.yml` (no `continue-on-error`, + container digest-pinned to the triage-verified `semgrep/semgrep:1.168.0` image), + scoped to `worker`, `src/lib/ingestion*.ts`, `src/lib/extractors`, `src/app/api/ingestion`, + `src/app/api/upload`, with `p/python` added for the OCR stack. Triage ran the CI-pinned + `semgrep/semgrep:1.168.0` image over the scoped paths: 0 ERROR findings (24 TS rules, + 17 files; 55 Python rules, 3 files) — the gate starts green. Both policy halves + (advisory repo-wide, blocking ingestion gate) are enforced by `check-github-action-pins.mjs`. ### X5 · ACL-migration consolidation review — `PROVIDER-GATED (DB owner)` @@ -189,7 +196,7 @@ collaborators join — `AGENTS.md` + the PR template already carry that load. | X1 Import-boundary linter | Next | **DONE** (#986; service-role rule dropped) | | X2 `src/lib` rag extraction | Next | **DONE** (#994) | | X3 Monolith decomposition | Next | OPEN (first extraction landed #997) | -| X4 SAST-blocking on parser | Next | PROVIDER-GATED (triage-first) | +| X4 SAST-blocking on parser | Next | **DONE** (gate + policy check) | | X5 ACL-migration consolidation | Next | PROVIDER-GATED (DB owner) | | X6 Coverage floors | Next | OPEN | | L1 Archive one-shot scripts | Later | OPEN (index shipped) | diff --git a/scripts/check-github-action-pins.mjs b/scripts/check-github-action-pins.mjs index 52040405e2..6f167b95bf 100644 --- a/scripts/check-github-action-pins.mjs +++ b/scripts/check-github-action-pins.mjs @@ -76,6 +76,35 @@ if (!/^ src worker scripts supabase\/functions\s*$/m.test(semgrepScanSt failures.push("sast.yml: the Semgrep scan command must target src, worker, scripts, and supabase/functions."); } +// Maturity X4: the untrusted-document parsing surface has a BLOCKING Semgrep +// gate — the inverse policy of the advisory repo-wide job above. yamlBlock +// returns "" when the job is missing, so every assertion below fails closed. +const semgrepGateJob = yamlBlock(sastWorkflow, "semgrep-ingestion-gate:", 2); +const semgrepGateStep = yamlBlock(semgrepGateJob, "- name: Semgrep scan (blocking)", 6); +if (!semgrepGateJob) { + failures.push("sast.yml: the semgrep-ingestion-gate job must exist (maturity X4)."); +} +if (/^\s*continue-on-error\s*:/m.test(semgrepGateJob)) { + failures.push("sast.yml: the Semgrep ingestion gate must block — no continue-on-error anywhere in the job."); +} +for (const target of [ + "worker", + "src/lib/ingestion*.ts", + "src/lib/extractors", + "src/app/api/ingestion", + "src/app/api/upload", +]) { + if (!semgrepGateStep.includes(target)) { + failures.push(`sast.yml: the ingestion gate must keep scanning ${target}.`); + } +} +if (!semgrepGateStep.includes("--config p/python")) { + failures.push("sast.yml: the ingestion gate must include p/python for the worker OCR stack."); +} +if (!/^ image: semgrep\/semgrep@sha256:[0-9a-f]{64}\s*$/m.test(semgrepGateJob)) { + failures.push("sast.yml: the blocking ingestion gate container must be digest-pinned (semgrep/semgrep@sha256:...)."); +} + // One SHA per action across every workflow AND composite action. Dependabot bumps // one file at a time, so a laggard can sit on an old major indefinitely; because // the per-line validation above only covers workflows, a composite skew (e.g.