harden(worker): validate claimed job rows and enrichment read-backs against Zod row contracts (#212 T4)

[BigSimmo]

Summary

• Ledger #212 tranche 4 (final tranche): audit of all 12 as unknown as casts in worker/** (11 in worker/main.ts, 1 in worker/index.ts), replacing the inbound ones with Zod-backed assertion contracts. Tranches 1–3 covered src/lib/rag/rag.ts (harden(rag): validate retrieval RPC rows against a Zod shape contract #1946), rag-candidate-sources.ts (harden(rag): validate signal rows from the candidate-source RPCs #1981), and src/app/api/** (harden(api): validate inbound DB and RPC payloads against Zod row contracts #2023).
• The audit found only one genuinely inbound cast: the claim_ingestion_jobs RPC result in claimJobs (worker/main.ts:407), whose documents field is to_jsonb(d.*) — untyped jsonb asserted into JobRow with nothing checking it. Two further sites (main.ts:1805-1806) were parameter casts papering over unvalidated read-backs from document_chunks/document_images. The remaining nine are outbound insert payloads, a Json-shape cast, or CJS interop, and are deliberately untouched — the same "most sites needed nothing" finding as tranches 2 and 3.
• New worker/row-contracts.ts: worker-local counterpart of src/lib/validation/row-contracts.ts / src/lib/rag/rag-row-contracts.ts, deliberately duplicating their small validate-log-throw core (z.looseObject only; asserts helpers over safeParse; WorkerRowShapeError carrying Zod issue paths only, never row values; pins cited to supabase/schema.sql constraints). Worker-local because the esbuild worker bundle has no @/ alias; import-pure so Vitest can exercise it (worker/main.ts itself has zero executable coverage).
• claimJobs now validates per row, fail-soft via partitionClaimedJobRows: valid rows proceed by the same references in the same order; a row failing the contract is failed terminally (retry: false) through the existing fail_or_retry_ingestion_job path, and a row whose ids are themselves untrustworthy is logged and left to the bounded stale reclaim. No shape problem can throw out of claimJobs.
• loadEnrichmentRows asserts its chunk/image read-backs, which lets both upsertDocumentDeepMemory parameter casts drop with no replacement cast. A mismatch there is contained by the existing inline-enrichment try: enrichment is marked failed, the job still completes, repair is queued.
• failOrRetryJob's job parameter is structurally narrowed (Pick<JobRow, ...>) so a contract-rejected row whose documents payload cannot be trusted can still be failed; the ordinary full-JobRow caller is unchanged.
• New tests/worker-row-contract.test.ts (28 cases) mirroring the tranche-3 test shape: no-mutation/reference-identity, unknown-column preservation at both nesting levels, schema-backed rejections, no row values in errors, issue cap, and the partition semantics (sibling survival, failable identity, indexed-status preservation, unusable-id fallback).

Per-site disposition of all 12 casts:

Site	Direction	Disposition
main.ts:407 claim RPC result	inbound	contracted — per-row fail-soft partition
main.ts:1805/1806 deep-memory params	inbound read-backs	contracted in loadEnrichmentRows; casts removed
main.ts:532document_index_quality upsert	outbound	left — worker-computed payload, sanitizeJsonbRecord applied
main.ts:808 caption-cache metadata as Json	outbound	left — write failures already swallowed; read side normalizes
main.ts:1399/1496/1578document_embedding_fields inserts	outbound	left — cast exists because embedding: number[] vs pgvector string; assertEmbeddingDim validates the risky field
main.ts:1478document_chunks insert	outbound	left — satisfies IndexedChunkRow[] + assertEmbeddingDim + sanitizeJsonbRecord
main.ts:1517document_table_facts insert	outbound	left — optional write, error already downgraded to a repair warning
main.ts:1546document_index_units insert	outbound	left — optional write, try-wrapped into the repair queue
worker/index.ts:8@next/env interop	library interop	left — result is null-checked before use

Verification

• npm run verify:pr-local
• npm run verify:ui when UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changed
• npm run verify:release before release or handoff confidence claims

UI verification not run: no UI, routing, styling, or browser surface changed (worker + tests + docs only).
Live eval gates not run: no retrieval, ranking, selection, chunking-order, or answer-generation behavior changed; no file under src/lib/rag/** or any RAG-ranking-protected surface is touched.

• npm run check:production-readiness (ingestion domain change; offline capability gaps reported as expected without provider credentials)
• Focused: npx vitest run tests/worker-row-contract.test.ts — 28 passed; adjacent suites (worker-run-loop, worker-behavior, worker-safe-logging, worker-bundle, architecture-boundaries, api-row-contract, rag-retrieval-row-contract) — 72 passed.
• npm run typecheck, npm run lint (--max-warnings 0), full npm run test (640 files, 6846 passed | 4 skipped), and node scripts/build-worker.mjs (esbuild bundle proof for the new relative import) all green. verify:pr-local summary: completed lint, typecheck, test, build, RAG fixture checks, ledger checks — failed: (none).

Risk and rollout

• Risk: background-job claim path. The change is validation-only for well-formed rows (same references, same order, no transform). The failure-mode change is deliberate and conservative: today a malformed claimed row throws a TypeErrorbeforeprocessJob's try/catch, so the job is never marked failed and sits leased until the 45-minute stale reclaim while silently burning attempts; with this change it is terminally failed at claim time with the Zod issue paths in error_message. A contract mismatch is deterministic (retrying reruns the same to_jsonb over the same columns), so terminal fail cannot create a poison-pill retry loop — and per-row partition means one bad row can no longer take down claimed siblings or the --once process. Enrichment-side mismatches degrade exactly as enrichment failures already do (job completes, repair queued). Residual false-rejection risk is confined to the two documented beyond-constraint pins (jsonb record-ness, document-status enum), both of which are claims the deleted casts were already making silently.
• Rollback: single-commit revert of the code commit restores the previous cast behavior; no schema, config, or dependency change.
• Provider or production effects: None. No patient-data handling, privacy controls, or document-access behavior changes; service-role credentials remain server-side only; the configured Supabase project/target is unchanged; no provider-backed check was run.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Error messages and logs carry only Zod issue paths and codes — never row values — because claimed rows and read-backs carry clinical document text, titles, and storage paths; this preserves the worker's safe-logging discipline in error_message columns and Sentry.
• Ingestion-worker-reviewer subagent pass: approved, no defects across concurrency/recovery, schema-constraint backing, gate bypass, type ripple, and test fidelity; review record committed in this PR.
• Residual (out of scope, distinct population from the as unknown as inventory): unguarded bare as casts on extractor output (tableMetadata.tableRows, main.ts:693/1004) and on the OpenAI vision classification widening (main.ts:798-800/1100/1115). Recorded in the #212 closure note rather than bundled here — each needs its own reader-by-reader pass.
• This closes #212: the closure is queued as an immutable inbox request (npm run issues:done), not a canonical-ledger edit, and the #212 T4 row in docs/rag-improvement/HANDOVER.md §2 is updated in this PR.

---

Generated by Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:49 minutes

Limit details: You’ve used all 1 included review currently available under your plan. You completed 95 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 13f8bce0-d4c6-4a76-8ba6-ece86d774224

📥 Commits

Reviewing files that changed from the base of the PR and between 92f7618 and 81aff0c.

📒 Files selected for processing (9)

• docs/branch-review-records/da99c19c3dd97371833d588e531db66c29c2d5e5fbeb467f26082b87ad7c1c91.record.md
• docs/outstanding-issues-inbox/2bfae2cf-d91e-4617-a0cb-bb8adbbad4fc.json
• docs/outstanding-issues-inbox/38e53f36-b48f-4607-b5a0-56efde6dbb3b.json
• docs/outstanding-issues-inbox/52d0dbcc-43e0-4660-89c5-9dbd6e080910.json
• docs/outstanding-issues-inbox/6602201c-14f2-46e9-9407-36e56efe13d5.json
• docs/rag-improvement/HANDOVER.md
• tests/worker-row-contract.test.ts
• worker/main.ts
• worker/row-contracts.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

CI triage

CI failed on this PR. Automated classification of the 3 failed job(s):

• Unit coverage — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
• Static PR checks — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.
• PR required — needs investigation: inspect the failing step and uploaded diagnostics; rerun only after classifying the cause.

Compared with main CI run #11389 (cancelled).

Classification is evidence routing, not permission to ignore a failure. Exact quarantined Playwright identities remain governed by the flake ledger.