Packet S6b: Gate B PASS — Docling lab benchmark run, decision record, and harness fixes

[BigSimmo]

Summary

• Gate B verdict: PASS, recorded in docs/rag-improvement/gate-b-decision-record-2026-08-18.{md,json} from evidence run 32176604314 (artifact docling-lab-report-32176604314, commit 8a92378). All five gates pass at the owner's pre-agreed 0 pp margins, committed before any dispatch (6d05c07): parse success 36/36 both engines, numeric/unit/comparator exactness 162/162 both engines, table cell F1 parity on both table strata (table-heavy improvement leg met at the owner-agreed 0 pp parity-at-ceiling target), hostile containment 10/10 both engines with 0 crash artifacts and 0 canary echoes, and resources far inside every ceiling (docling max wall P95 12,558 ms vs the 120 s cap; max peak RSS ~1.40 GiB vs the 6 GiB cap).
• Four latent harness defects were found and fixed en route — the docling side of the S6 harness had never executed end-to-end before this session (each fix is its own revertible commit; run history is tabulated in the decision record):
  • 779af80 — regenerate eval/docling/requirements.txt with pip-tools' --allow-unsafe so torch's setuptools>=77.0.3 runtime requirement is pinned into the hashed lock (run 1 failed the --require-hashes install).
  • 958a70d — add libgl1 + libglib2.0-0 to the sandbox image; docling's rapidocr stage imports cv2 during model prefetch (run 2 failed on libGL.so.1).
  • c66faf8 — per-document progress lines now append a bounded, canary-redacted exception name/message for non-completed documents (run 3 produced a uniform all-error docling pass with the cause locked inside never-uploaded raw output; stream tails and extracted text are still never printed).
  • 6c6e80e — run the docling phase with TORCHDYNAMO_DISABLE=1: docling's models call torch.compile, whose Inductor backend needs a C++ compiler the sandbox image deliberately lacks (run 4's diagnostic named InvalidCxxCompiler on every clean document).
  • 8a92378 — make assertion scoring escape-neutral (html.unescape in score.pynormalise, alongside the existing \| handling): docling's markdown export HTML-escapes comparators (>= 1.4 mmol/L as >= 1.4 mmol/L), which run 5 scored as −25 to −62.5 pp despite the values being intact (verified locally: 8/8 assertions recover on the comparator-richest fixture).
• docs/rag-improvement/HANDOVER.md: S6b status row added; S7+ row's B4 cell set to unblocked — Gate B PASS (design-only authorisation, caveats carried in the decision record).
• Two immutable issue-inbox requests queued (never hand-edited the ledger): the docling-lab-fixtures.v2 table-hardness corpus (P3 — the v1 tables sit at legacy F1 1.0, so the improvement leg had no headroom), and the B4 shadow-mode build packet (P2 — authorised by this pass, with the parity-at-ceiling and eager-mode-latency caveats attached).
• One immutable branch-review ledger record appended at 397f40d.

RAG impact: no retrieval behaviour change — benchmark evidence and decision record only

Verification

• npm run verify:pr-local — selected plan completed: check:runtime, check:installed-lock-parity, format:changed, sitemap:check, docs:check-index/inventory/scripts/links, check:branch-review-ledger, check:outstanding-issues, check:ledger-write-discipline, lint, typecheck all green. The wrapper's test step raced this session's own concurrent commit/push (the push guard runs Prettier in a temp worktree); a clean dedicated rerun on the identical tree is green: Test Files 673 passed (673), Tests 7281 passed | 4 skipped (7285), exit 0. The three wrapper steps behind it were then run directly: check:rag:fixtures — "Offline RAG fixture and manifest validation passed (36 golden cases, 26 suites)"; check:medication-interactions — index up to date (525 rows); check:medication-lexicon-report — up to date (37 catalogue terms).
• npm run check:docling-lab — "docling-lab contract passed (36 fixtures, 10 hostile, 6 canaries; Gate B template valid)."
• node eval/docling/report/build-report.mjs --validate-record docs/rag-improvement/gate-b-decision-record-2026-08-18.json --final — "docling-lab: Gate B record valid (final mode)."
• UI verification not run: no UI, routing, styling, or browser behaviour changed (eval lab + docs only).
• verify:release not run: not a release/handoff-confidence claim, and it is provider-backed.
• eval:retrieval:quality / eval:rag / eval:quality not run: no retrieval, ranking, selection, chunking, generation, or answer-contract code changed; provider-backed and not authorised for this packet.
• check:production-readiness not run: no clinical workflow, privacy, environment, Supabase, source-governance, or deployment behaviour changed — the diff is confined to the isolated eval/docling/ lab and documentation records.

Risk and rollout

• Risk: Low. No production code path changes — everything sits under eval/docling/ (dispatch-only lab, never part of pr-required) and docs/. The worker, extractors, Dockerfile.worker, worker/python/requirements*, and the database are untouched, per the HANDOVER S6 hard boundary. The scoring change (html.unescape) alters only how the lab compares assertion strings; the leak posture is unchanged (the new log line prints exception name/message only, canary-redacted and truncated — never stream tails or extracted text).
• Rollback: git revert any individual commit; each harness fix and the decision-record commit are independent. Reverting the record commits simply withdraws the Gate B verdict; the lab remains isolated by construction.
• Provider or production effects: None from this diff. The six benchmark dispatches consumed hosted-CI minutes only (each explicitly owner-approved in the session); no OpenAI or Supabase call occurred.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• The dispatch chain (six runs) and the reason each earlier run is not the evidence run are tabulated in the decision record's "Run history" section — runs 1–2 were build failures, run 3 was workflow-green but not a valid docling measurement, run 4 was the diagnostic, run 5 fell to the scoring-encoding artifact.
• The Gate B pass authorises designing packet B4 (WORKER_DOCUMENT_EXTRACTOR_MODE=legacy|shadow) only; the ingestion-worker-reviewer subagent reviews that future PR. Two caveats travel with the pass: the table-heavy leg was satisfied at parity-on-ceiling (not a demonstrated gain), and docling ran eager at ~9–19 s/document on 2 CPUs versus legacy's ~1 s.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwXsbJpfsPmUp719Mco2Rq

---

Generated by Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in:33 minutes

Limit details: You’ve used the included review currently available. Your 101 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6fd0d94f-071b-4164-b995-67c039a5938b

📥 Commits

Reviewing files that changed from the base of the PR and between 333399f and a11609c.

📒 Files selected for processing (12)

• docs/branch-review-records/381b41511e145024c0d34d173ed186f1659871b294a41e390fa36fdf6f5bdfb9.record.md
• docs/outstanding-issues-inbox/abc21f52-0d39-490d-9676-d106b9af4302.json
• docs/outstanding-issues-inbox/fb87f710-b46c-452a-b349-e3ce71c36959.json
• docs/rag-improvement/HANDOVER.md
• docs/rag-improvement/gate-b-decision-record-2026-08-18.json
• docs/rag-improvement/gate-b-decision-record-2026-08-18.md
• eval/docling/Dockerfile
• eval/docling/generate-lock.mjs
• eval/docling/harness/entry.sh
• eval/docling/harness/run_corpus.py
• eval/docling/harness/score.py
• eval/docling/requirements.txt

---

_{Comment @coderabbitai help to get the list of available commands.}