fix(chunking): keep dose units wrapped onto their own line by PDF extraction

[BigSimmo]

Summary

PyMuPDF extraction (page.get_text("text", sort=True), the exact call in worker/python/extract_pdf_assets.py) splits a dose in a narrow table cell across lines: "12.5\nmg". looksLikeMetadataNoise treats any line of <= 2 characters as extraction debris, so removePageNoise silently deleted the unit line, indexing a unitless dose (12.5) — a clinical-output correctness risk.

removePageNoise now rejoins a unit-only line (mg, mcg, µg, g, kg, mL, L, IU, U, mmol, %) to the preceding line when that line ends in a digit, before the debris filter runs. Conservative by design:

• a lone unit token with no preceding dose number is still dropped as debris;
• standalone page footers are never merged into;
• longer unit tokens (mcg, mmol) also get rejoined, so "100\nmcg" indexes as 100 mcg instead of two lines.

Evidence

• Live corpus could not be sampled (Supabase keys rotated 2026-07-06, no valid secret key on this machine).
• Mechanism proven locally with the production extractor: a narrow-cell PDF built with PyMuPDF and extracted via get_text("text", sort=True) yields '12.5\nmg' / '600\nmcg'. Real guideline dose tables use exactly this narrow-column layout.
• The three synthetic demo PDFs (wide layout) do not produce short lines, so demo mode was unaffected.

Tests

• 4 regression tests added in tests/chunking.test.ts (rejoin decimal + integer doses, lone-unit still dropped, footer not merged). 31/31 pass.
• npm run typecheck and prettier --check clean.
• Contract comment in tests/property-chunking.test.ts (branch claude/audit-sweep-property-tests) updated to note the exception; its generators never emit unit-only lines so the property suite is unaffected.

Governance note

Touches ingestion chunking only; behavior change is additive (keeps/rejoins clinical content that was previously deleted). Existing indexed chunks are unaffected until a re-index; new ingestions pick this up immediately.

🤖 Generated with Claude Code

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.