Add first-page PDF cover thumbnails on document search cards

[BigSimmo]

Summary

• Generate compact first-page cover_page PNG thumbnails during PDF extraction (PyMuPDF) and persist them as non-searchable display artifacts.
• Surface coverImageId / cover_image_id on document search matches and render real page-1 previews in DocumentPagePreview (placeholder fallback retained).
• Add demo cover assets for all three synthetic PDFs, contract tests, and scripts/backfill-document-covers.mjs to backfill existing live PDFs without a full reindex.
• Fold cover id resolution into the existing document_images visual enrichment query (no extra round trip / offline budget stays 13).

RAG impact: no retrieval behaviour change — covers are searchable: false, excluded from caption budget / chunk attachments / isClinicalImageEvidence, and are display-only for search cards.

Verification

• Focused vitest: document-cover-thumbnails, document-enrichment-visual-counts, rag-round-trip-budget, demo-data — passed
• Python: python3 -m unittest test_extract_pdf_assets_cover — OK
• Live backfill running with --apply (operator-approved): ~513 / 2065 indexed PDFs already have covers mid-run
• UI verification not run: cover rendering covered by unit/contracts; Chromium journey deferred unless PR policy requires shared-chrome proof

Risk and rollout

• Risk: Existing live docs show placeholder cards until covers are backfilled or reindexed; cover signed-URL fetches occur for visible result cards.
• Rollback: Revert this PR; covers remain inert non-searchable rows if left in storage.
• Provider or production effects: Backfill writes cover images + metadata.cover_image_id via service role (scripts/backfill-document-covers.mjs --apply). Ingest path writes covers on future index jobs.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Covers use image_type=unclear + source_kind=cover_page (no DB CHECK migration required).
• Backfill is dry-run by default; pass --apply explicitly.
• After merge, search cards show page-1 rasters when coverImageId is present.

Summary by CodeRabbit

• New Features

  • Added first-page cover thumbnails to document search results and related-document previews.
  • Added loading placeholders and fallback previews when cover images are unavailable.
  • Automatically generates and stores PDF cover thumbnails for indexed documents.
  • Added a controlled backfill option for existing PDF documents.

• Bug Fixes

  • Cover thumbnails are excluded from clinical evidence, captioning, and search indexing.
  • Improved image associations for document previews and related search results.

• Tests

  • Added coverage for cover extraction, persistence, display, filtering, and fallback behavior.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in:45 minutes

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fa9bc51b-3d43-4b57-8738-c0fe735411a3

📥 Commits

Reviewing files that changed from the base of the PR and between 29e6003 and 0493497.

⛔ Files ignored due to path filters (1)

• worker/python/fixtures/synthetic-lithium-monitoring.pdf is excluded by !**/*.pdf

📒 Files selected for processing (11)

• docs/design-system/COMPONENTS.md
• docs/design-system/adoption-manifest.json
• docs/scripts-index.md
• scripts/backfill-document-covers.mjs
• src/components/clinical-dashboard/document-search-results.tsx
• src/lib/demo-data.ts
• src/lib/document-enrichment.ts
• tests/document-cover-thumbnails.test.ts
• tests/document-enrichment-visual-counts.test.ts
• worker/python/extract_pdf_assets.py
• worker/python/test_extract_pdf_assets_cover.py

📝 Walkthrough

Walkthrough

The PR adds first-page PDF cover extraction, stores covers as non-searchable preview images, propagates cover image IDs through search data, displays signed covers in document cards, and provides a dry-run backfill script for existing indexed PDFs.

Changes

Document cover thumbnails

Layer / File(s)	Summary
PDF cover extraction worker/python/extract_pdf_assets.py, worker/python/test_extract_pdf_assets_cover.py	The extractor renders bounded first-page PNG covers and validates their metadata, dimensions, and output files.
Cover persistence and filtering worker/main.ts, src/lib/image-filtering.ts, tests/worker-visual-capture.test.ts, tests/document-cover-thumbnails.test.ts	The worker retains covers as non-searchable previews, excludes them from clinical evidence and caption selection, and stores the first cover image ID in document metadata.
Cover ID enrichment and preview src/lib/types.ts, src/lib/document-enrichment.ts, src/app/api/search/route.ts, src/lib/demo-data.ts, src/components/clinical-dashboard/document-search-results.tsx, tests/document-enrichment-visual-counts.test.ts, tests/document-cover-thumbnails.test.ts	Search and related-document results expose cover IDs. Demo fixtures include cover records. Document previews load signed cover URLs and fall back to generated previews when needed.
Existing document backfill scripts/backfill-document-covers.mjs	The script finds eligible PDFs without covers, supports dry-run and apply modes, uploads rendered covers, updates metadata, and reports created, skipped, and failed documents.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• BigSimmo/Database#465: Both changes use the Python PDF extractor payload.
• BigSimmo/Database#912: Both changes modify PDF extraction and cover-image handling.
• BigSimmo/Database#1129: Both changes retain and persist non-searchable document-view images.

Sequence Diagram(s)

sequenceDiagram
participant PDFExtractor
participant IndexingWorker
participant SearchAPI
participant DocumentPreview
PDFExtractor->>IndexingWorker: provide first-page cover artifact
IndexingWorker->>IndexingWorker: persist non-searchable cover image
IndexingWorker-->>SearchAPI: provide cover image ID
SearchAPI-->>DocumentPreview: provide coverImageId
DocumentPreview->>DocumentPreview: request signed cover URL

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding first-page PDF cover thumbnails to document search cards.
Description check	✅ Passed	The description covers implementation, verification, risks, rollout, governance, and the known pending UI and full PR verification gates.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/document-cover-thumbnails-b74c

---

_{Comment @coderabbitai help to get the list of available commands.}

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Actionable comments posted: 7

🧹 Nitpick comments (2)

scripts/backfill-document-covers.mjs (2)

40-40: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Resolve worker/python from the script location, not the working directory.

process.cwd() requires the operator to run the command from the repository root. Any other working directory produces an import failure for extract_pdf_assets. Derive the path from import.meta.url instead.

♻️ Proposed refactor

Add the import:

 import path from "node:path";
+import { fileURLToPath } from "node:url";

Then resolve the directory relative to the script:

-sys.path.insert(0, ${JSON.stringify(path.join(process.cwd(), "worker/python"))})+sys.path.insert(0, ${JSON.stringify(+ path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "worker", "python"),+)})

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/backfill-document-covers.mjs` at line 40, Update the Python path
setup in the backfill script to resolve worker/python relative to the script’s
import.meta.url instead of process.cwd(). Use the script-location URL/path
conversion and preserve the existing sys.path insertion so extract_pdf_assets
imports work from any working directory.

---

201-206: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Exit with a non-zero code when documents fail.

The script currently exits 0 even when every document fails. Any wrapper script or scheduled job then reports success. Set the exit code from the failed counter.

♻️ Proposed refactor

 if (!apply) {
console.log("Re-run with --apply to write covers.");
}
+ if (failed > 0) {+ process.exitCode = 1;+ }
}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/backfill-document-covers.mjs` around lines 201 - 206, Update the
completion flow in the backfill script after the summary logging to set the
process exit code based on the failed counter, returning a non-zero code
whenever failed is greater than zero while preserving the existing zero-success
behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/backfill-document-covers.mjs`:
- Around line 181-189: Re-read the document’s current metadata immediately
before the update in the cover patch flow, then merge the new cover_image_id
into that refreshed object so concurrent metadata changes are preserved. In the
existing-cover skip path around the cover lookup, reuse the existing cover row’s
id when metadata.cover_image_id is absent, and patch the document metadata
instead of permanently skipping it.
- Around line 108-119: Move the document_images lookup and its coverError
handling into the per-document try block so a single lookup failure is recorded
as that document’s failure and processing continues. Preserve the existing skip
behavior for documents with an existing cover and ensure the run reaches the
final summary.
- Around line 49-70: Update the child-process promise around spawn in the
document-rendering helper to listen for the child’s error event and reject with
that error, preserving the caller’s per-document error handling. Add a render
timeout that terminates the spawned Python process and rejects when exceeded,
while clearing the timer on successful close or spawn failure so normal renders
and cleanup remain unaffected.
- Around line 37-46: Update the embedded Python script near
extractor.save_cover_page to set the extraction budget’s page count from the
opened document before rendering the cover. Keep using ExtractionBudget and
preserve the existing render and artifact-slot arguments; only ensure
set_page_count is applied so maxPages is enforced.
In `@src/components/clinical-dashboard/document-search-results.tsx`:
- Around line 802-833: Update the document page preview image flow around
showCover and coverLoaded so the skeleton remains visible until the current
signed URL has successfully loaded. Track which coverUrl has loaded, add image
error handling that invokes markFailed, and remove the useEffect reset of load
state; ensure stale URL state cannot reveal a transparent or failed image.
In `@worker/python/extract_pdf_assets.py`:
- Around line 1070-1075: Update the extraction flow around save_cover_page and
normal image extraction to process all searchable artifacts before attempting
the optional cover. Then call save_cover_page for the first page only with the
remaining budget, catch ExtractionBudgetExceeded, and skip the cover while
adding a warning; preserve successful cover insertion into images without
allowing cover budget usage to fail indexing.
In `@worker/python/test_extract_pdf_assets_cover.py`:
- Around line 32-33: Remove the no-op assertion in the test method containing
the clinical-crops comment, including the `or True` condition. If the fixture is
required to include a non-cover artifact, replace it with an explicit assertion
for that expected artifact; otherwise remove the assertion entirely.
---
Nitpick comments:
In `@scripts/backfill-document-covers.mjs`:
- Line 40: Update the Python path setup in the backfill script to resolve
worker/python relative to the script’s import.meta.url instead of process.cwd().
Use the script-location URL/path conversion and preserve the existing sys.path
insertion so extract_pdf_assets imports work from any working directory.
- Around line 201-206: Update the completion flow in the backfill script after
the summary logging to set the process exit code based on the failed counter,
returning a non-zero code whenever failed is greater than zero while preserving
the existing zero-success behavior.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ea94a577-19bd-4694-8292-8c8ffe51610b

📥 Commits

Reviewing files that changed from the base of the PR and between c839b97 and 595cdf9.

⛔ Files ignored due to path filters (3)

• public/demo-documents/synthetic-clozapine-monitoring-cover.png is excluded by !**/*.png
• public/demo-documents/synthetic-lithium-monitoring-cover.png is excluded by !**/*.png
• public/demo-documents/synthetic-risk-flow-cover.png is excluded by !**/*.png

📒 Files selected for processing (13)

• scripts/backfill-document-covers.mjs
• src/app/api/search/route.ts
• src/components/clinical-dashboard/document-search-results.tsx
• src/lib/demo-data.ts
• src/lib/document-enrichment.ts
• src/lib/image-filtering.ts
• src/lib/types.ts
• tests/document-cover-thumbnails.test.ts
• tests/document-enrichment-visual-counts.test.ts
• tests/worker-visual-capture.test.ts
• worker/main.ts
• worker/python/extract_pdf_assets.py
• worker/python/test_extract_pdf_assets_cover.py