Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) by AndresL230 · Pull Request #473 · SaplingLearn/Sapling · GitHub
Skip to content

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2) - #473

Merged
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover
Jul 30, 2026
Merged

refactor(documents): retire the legacy pipelines and delete gemini_service (#151b, 2/2)#473
AndresL230 merged 2 commits into
mainfrom
feat/b7-151b-documents-cutover

Conversation

@AndresL230

@AndresL230AndresL230 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Part 2 of 2 of the final gemini_service cutover — and the deletion itself: services/gemini_service.py is gone (zero production references; the benchmark scripts' baseline arms move to a benchmark-only scripts/_raw_gemini.py). Full detail in the commit message:

  • documents.py's legacy pipelines deleted; /upload/sync maps agent failures to a retry-friendly 502; the streaming route emits the terminal error:failed + done pair (step=fallback leaves the SSE vocabulary, frontend dead branch removed with it); /scan-concepts degrades to the empty shape.
  • The [P2] Agent platform: productionize DBOS durability + crash-safe streaming upload #154 preconditions preserved untouched (idempotency short-circuit, the three separately-threaded persistence helpers, the post-roll guard — comment strengthened).
  • concept_scan registered in the e2e function handlers (the one unregistered request-path task the scoping pass found).
  • ADR 0024 records the retirement: the canonical rung ladder (server + client — retryable, the sapling_wrote stamp, 413-vs-502), the /start-session convergence, the pre-beta rationale in place of a reachability query, and the revert path (refactor(learn): agent-only rung ladder — retire the legacy chat paths (#151a, 1/2) #472 + this PR). ADR 0001's fallback clause superseded; architecture.md / CLAUDE.md / README / SECURITY swept to the agents-only reality.

Verification

Backend 1468 passed + ruff clean; 148 passed under lock-pinned pydantic-ai 1.107; evals replay green ×6 (untouched); frontend 349 + tsc clean. 12 red-first tests; ~50 legacy tests deleted/ported per the scoping brief's disposition table. Full local e2e cycle pre-merge; results below.

Closes#151.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added deterministic concept-scan behavior for end-to-end testing.
    • Added benchmark-only tools for direct Gemini text and JSON generation.
  • Bug Fixes

    • Upload failures now provide clear retryable errors: HTTP 502 for synchronous uploads and terminal failure events for streaming uploads.
    • Concept scanning now safely returns no new concepts when agent processing fails.
    • Removed misleading fallback warnings from the document upload interface.
  • Documentation

    • Updated architecture, security, API, and decision documentation to reflect agent-based LLM processing and current failure behavior.

…rvice — the cutover completes (#151)
Part 2 of 2. services/gemini_service.py is DELETED — zero production
references remain; the benchmark scripts' baseline arms move to a
benchmark-only scripts/_raw_gemini.py helper.
- documents.py: _process_document, _extend_course_concepts,
_legacy_upload_pipeline, _stream_legacy_fallback and the three dead
coercion helpers deleted. /upload/sync maps agent failures to a
retry-friendly 502; the streaming route emits the terminal
error:failed + done pair (step=fallback leaves the SSE vocabulary, and
the frontend's dead toast branch goes with it); /scan-concepts
degrades to the empty shape (best-effort enrichment). The #154
preconditions are preserved untouched: the X-Request-ID idempotency
short-circuit, the three separately-to_thread'd persistence helpers,
and the post-roll try/except (comment strengthened — never a second
result; the fallback it guarded against no longer exists).
- concept_scan registered in the e2e function handlers (it was the one
unregistered request-path task) with the constants-sync test.
- ADR 0024 records the retirement: the canonical rung ladder (server +
client, retryable/sapling_wrote/413-vs-502), the /start-session
convergence, the pre-beta rationale (prod carries no user traffic —
catalog-only — so legacy-reachability measurement is moot; #117's
events make post-beta rates observable from day one), and the revert
path (git history, the #472 + this PR pair). ADR 0001's fallback
clause superseded; architecture.md/CLAUDE.md/README/SECURITY docs
swept to the agents-only reality.
- 12 red-first tests (502 mapping, terminal-pair, scan degrades, seam
handler); ~50 legacy tests deleted/ported per the scoping brief's
disposition table.
Gates: backend 1468 passed + ruff clean; lockvenv 148 passed; evals
replay green ×6; frontend 349 + tsc clean.
Closes#151.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pagesBot commented Jul 30, 2026

Copy link
Copy Markdown

Deploying with Cloudflare Workers Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

StatusNameLatest CommitPreview URLUpdated (UTC)
✅ Deployment successful!
View logs
frontend-staging4b61aecCommit Preview URL

Branch Preview URL
Jul 30 2026, 02:44 PM

@supabase

supabaseBot commented Jul 30, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitaiBot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:27 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 67b3cf91-7ac4-4638-92ec-4a1d3ddf7c7c

📥 Commits

Reviewing files that changed from the base of the PR and between eb7cbfd and 4b61aec.

📒 Files selected for processing (5)
  • CLAUDE.md
  • backend/tests/test_event_capture_seams.py
  • docs/architecture.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • frontend/e2e/streaming.spec.ts
📝 Walkthrough

Walkthrough

The PR retires the legacy Gemini service and document-upload fallbacks, standardizes agent failure behavior, adds deterministic concept-scan E2E handlers, introduces a benchmark-only raw Gemini helper, and updates tests, frontend handling, architecture documentation, and ADRs.

Changes

Agent migration and failure semantics

Layer / File(s)Summary
Architecture and migration contracts
CLAUDE.md, README.md, docs/architecture.md, docs/decisions/*, SECURITY.md, docs/frontend-audit/*, backend/services/cache.py, backend/agents/document.py
Documentation now identifies Pydantic AI agents as the production LLM seam, documents terminal failure behavior, and limits raw Gemini use to benchmarks.
Concept scan degradation and E2E dispatch
backend/routes/documents.py, backend/agents/function_handlers_e2e.py, backend/tests/test_concept_scan.py, backend/tests/test_e2e_function_handlers.py
Concept-scan agent failures return empty enrichment results, and deterministic function-handler registration is covered by E2E tests.
Upload orchestration and terminal failures
backend/routes/documents.py, backend/tests/test_documents_routes.py, backend/tests/test_event_capture_seams.py, frontend/src/components/DocumentUploadModal.*
Synchronous upload failures return 502, streaming failures emit error:failed followed by status:done, and legacy fallback execution and UI handling are removed.
Benchmark helper and seam cleanup
backend/scripts/_raw_gemini.py, backend/scripts/benchmark_*.py, backend/tests/*
Benchmark scripts use the new raw Gemini helper, while tests and hermetic guard documentation remove assumptions about gemini_service.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers:darkest-teddy, jose-gael-cruz-lopez

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 59.38% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly states the legacy Gemini pipeline retirement and gemini_service deletion, matching the PR's main change.
Description check✅ PassedIt covers the PR summary, linked issue, and verification details; only the exact template headings are missing.
Linked Issues check✅ PassedThe PR removes gemini_service helpers, moves benchmark calls to _raw_gemini, updates tests/docs, and adds ADR 0024 as required by #151.
Out of Scope Changes check✅ PassedThe changed files all support the Gemini cutover, legacy cleanup, tests, and docs updates; no unrelated scope stands out.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/b7-151b-documents-cutover

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/src/components/DocumentUploadModal.tsx (1)

175-185: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Terminal error:failed now double-toasts.

The backend's failure tail is error:failedstatus:done with no result, so uploadDocumentStream also rejects ("stream ended without a result event") and the catch block at Line 213 fires a second Upload failed: … toast. With step="fallback" gone this is the only failure path, so every failed upload shows two toasts.

🐛 Suggested guard
 try {
+ let toastedInBand = false;
const fd = new FormData();
@@
if (ev.step === "failed") {
toast.error(`Upload failed: ${ev.message}`);
+ toastedInBand = true;
}
@@
- if (!aborted) toast.error(`Upload failed: ${errorMsg}`);+ if (!aborted && !toastedInBand) toast.error(`Upload failed: ${errorMsg}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/components/DocumentUploadModal.tsx` around lines 175 - 185,
Update the error handling in the upload flow around uploadDocumentStream so
terminal step === "failed" records the failure state without showing a toast
there, since the rejected stream is already handled by the catch-block toast.
Preserve the existing progress update and ensure non-terminal error events
remain informational.
🧹 Nitpick comments (1)
backend/routes/documents.py (1)

895-927: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the terminal error:failed + status:done tail into a helper.

The same two-event tail is now emitted in five places in event_stream (extraction failure, unusable text, both agent-failure branches, post-roll failure). A small helper keeps the wire contract in one place.

♻️ Sketch
+def _terminal_failure_sse(message: str, request_id: str | None):+ """Terminal SSE tail for any in-stream failure: error:failed + status:done."""+ yield sapling_event_to_sse(SaplingEvent(+ type="error", step="failed", message=message,+ data={"request_id": request_id} if request_id else None,+ ))+ yield sapling_event_to_sse(SaplingEvent(+ type="status", step="done", message="Failed.",+ ))

Then each branch becomes:

- yield sapling_event_to_sse(SaplingEvent(- type="error", step="failed",- message="Document processing failed. Please try again.",- data={"request_id": request_id} if request_id else None,- ))- yield sapling_event_to_sse(SaplingEvent(- type="status", step="done",- message="Failed.",- ))- return+ for ev in _terminal_failure_sse(+ "Document processing failed. Please try again.", request_id,+ ):+ yield ev+ return
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/routes/documents.py` around lines 895 - 927, Extract the repeated
terminal error:failed and status:done SSE emission from event_stream into a
small helper, preserving the existing messages and request_id data. Replace all
five duplicated terminal branches, including the shown guardrail and
unexpected-exception handlers, with calls to that helper followed by their
existing return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/scripts/_raw_gemini.py`:
- Around line 21-24: Defer construction of the module-level _client in
_raw_gemini until after agents._providers.model_mode() confirms "real", so
importing _raw_gemini or calling _generate() in non-real modes cannot initialize
google-genai. Preserve the existing client configuration and ensure the guarded
path still provides the client for real-mode generation.
In `@backend/tests/test_hermetic_llm_guard.py`:
- Around line 69-71: Update the docstring in the hermetic LLM guard test to
qualify the “one remaining module-level client” statement, specifying that it
refers to production clients or that benchmark-only clients are outside this
test’s scope; keep the existing coverage description unchanged.
In `@docs/architecture.md`:
- Around line 17-19: Update the “LLM seam (current)” paragraph to make the
raw-client exception list consistent: explicitly identify
services/rag_service.py as the sanctioned model_mode()-gated raw embedding
client, and revise the statement about scripts/_raw_gemini.py to distinguish the
only ungated benchmark caller. Preserve the requirement that any raw client
below agents/_providers.py uses a model_mode() gate.
In `@docs/decisions/0001-adopt-pydantic-ai.md`:
- Around line 6-9: Update ADR 0001’s passages around the references to
gemini_service.py, including the sections at lines 13, 17, and 26, to remove
claims that it is currently used, remains during migration, or serves as the
legacy fallback. Rephrase them as historical context or remove them, while
preserving the framework-adoption decision and consistency with ADR 0024.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md`:
- Around line 51-52: Update the retry-guarantee statement in ADR 0020 to remove
the claim that no data is persisted on stop or failure. State instead that
transcript persistence occurs only on completion, while failures after graph or
mastery tool writes remain non-retryable, consistent with the canonical behavior
referenced by ADR 0024.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md`:
- Around line 90-94: Clarify the `/upload/sync` retry-safety statement around
`process_document` and `apply_concepts_to_graph`: do not claim that nothing was
persisted unless graph merging and document insertion are atomic or the merge is
idempotent. Update the ADR to describe the actual side-effect boundary and retry
behavior, including the conditions required for a fresh `X-Request-ID` retry to
be safe.
---
Outside diff comments:
In `@frontend/src/components/DocumentUploadModal.tsx`:
- Around line 175-185: Update the error handling in the upload flow around
uploadDocumentStream so terminal step === "failed" records the failure state
without showing a toast there, since the rejected stream is already handled by
the catch-block toast. Preserve the existing progress update and ensure
non-terminal error events remain informational.
---
Nitpick comments:
In `@backend/routes/documents.py`:
- Around line 895-927: Extract the repeated terminal error:failed and
status:done SSE emission from event_stream into a small helper, preserving the
existing messages and request_id data. Replace all five duplicated terminal
branches, including the shown guardrail and unexpected-exception handlers, with
calls to that helper followed by their existing return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c3c12d80-9acf-4cd1-9e5a-dd2fa006acb2

📥 Commits

Reviewing files that changed from the base of the PR and between edd1023 and eb7cbfd.

📒 Files selected for processing (28)
  • CLAUDE.md
  • README.md
  • SECURITY.md
  • backend/agents/document.py
  • backend/agents/function_handlers_e2e.py
  • backend/routes/documents.py
  • backend/scripts/_raw_gemini.py
  • backend/scripts/benchmark_quiz.py
  • backend/scripts/benchmark_rag.py
  • backend/services/cache.py
  • backend/services/gemini_service.py
  • backend/tests/README.md
  • backend/tests/conftest.py
  • backend/tests/test_concept_scan.py
  • backend/tests/test_documents_routes.py
  • backend/tests/test_e2e_function_handlers.py
  • backend/tests/test_event_capture_seams.py
  • backend/tests/test_gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/test_hermetic_llm_guard.py
  • backend/tests/test_usage_instrumentation_coverage.py
  • docs/architecture.md
  • docs/decisions/0001-adopt-pydantic-ai.md
  • docs/decisions/0020-streaming-tutor-interrupt-retry.md
  • docs/decisions/0024-retire-legacy-gemini-seam.md
  • docs/frontend-audit/07-integrations.md
  • frontend/src/components/DocumentUploadModal.test.tsx
  • frontend/src/components/DocumentUploadModal.tsx
💤 Files with no reviewable changes (4)
  • backend/services/gemini_service.py
  • backend/tests/test_gemini_usage_logging.py
  • backend/tests/README.md
  • backend/tests/test_gemini_service.py

Comment on lines +21 to +24
_client = genai.Client(
api_key=GEMINI_API_KEY or "dummy-key-for-import",
http_options=types.HttpOptions(timeout=180_000),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -A8 -B4 '\bdef model_mode\b|\bmodel_mode\(' backend/agents/_providers.py

Repository: SaplingLearn/Sapling

Length of output: 1173


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'--- backend/scripts/_raw_gemini.py first 90 lines ---\n'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'\n--- imports/usages of _raw_gemini and google.genai (excluding cache/tools) ---\n'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'

Repository: SaplingLearn/Sapling

Length of output: 249


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/_raw_gemini.py first 90 lines ---'
sed -n '1,90p' backend/scripts/_raw_gemini.py
printf'%s\n'''printf'%s\n''--- imports/usages of _raw_gemini and google.genai ---'
rg -n "backend/scripts/_raw_gemini|from .*_raw_gemini|import .*_raw_gemini|google\.genai|genai\.Client|from agents\._providers import model_mode|model_mode\(\) ==" backend -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8453


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/ingest_catalog.py relevant section ---'
sed -n '1,80p' backend/scripts/ingest_catalog.py
printf'%s\n'''printf'%s\n''--- backend/routes/documents.py model_mode construction guarded section ---'
sed -n '1068,1105p' backend/routes/documents.py
printf'%s\n'''printf'%s\n''--- backend/services/rag_service.py model_mode construction guarded section ---'
sed -n '20,45p' backend/services/rag_service.py

Repository: SaplingLearn/Sapling

Length of output: 6624


🏁 Script executed:

#!/bin/bashset -euo pipefail
printf'%s\n''--- backend/scripts/benchmark_quiz.py relevant imports/call site ---'
sed -n '1,80p' backend/scripts/benchmark_quiz.py
printf'%s\n'''printf'%s\n''--- backend/scripts/benchmark_rag.py relevant imports/call site ---'
sed -n '1,120p' backend/scripts/benchmark_rag.py
printf'%s\n'''printf'%s\n''--- backend/scripts/_providers-related imports/usages in backend/scripts ---'
rg -n "agents\._providers|agents/_providers|from agents\._providers import model_mode|model_mode\(\)" backend/scripts -g '*.py'||true

Repository: SaplingLearn/Sapling

Length of output: 8351


Gate the raw Gemini client before importing _raw_gemini.

This is the only sanctioned raw google-genai path, but the module-level client is still constructed at import time and _generate() can fail before model_mode() == "real" is checked. Move/defer the client construction behind the same agents._providers.model_mode() guard, or import through a gated helper.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/scripts/_raw_gemini.py` around lines 21 - 24, Defer construction of
the module-level _client in _raw_gemini until after
agents._providers.model_mode() confirms "real", so importing _raw_gemini or
calling _generate() in non-real modes cannot initialize google-genai. Preserve
the existing client configuration and ensure the guarded path still provides the
client for real-mode generation.

Source: Coding guidelines

Comment on lines +69 to +71
exact objects a forgotten `patch(...)` would leave live. (The
gemini_service client this class also covered was deleted in #151b /
ADR 0024; rag_service holds the one remaining module-level client.)"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Qualify the “one remaining module-level client” statement.

backend/scripts/_raw_gemini.py now also creates a module-level genai.Client. Either clarify this as the one remaining production client or explicitly state that benchmark-only clients are outside this test’s scope.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@backend/tests/test_hermetic_llm_guard.py` around lines 69 - 71, Update the
docstring in the hermetic LLM guard test to qualify the “one remaining
module-level client” statement, specifying that it refers to production clients
or that benchmark-only clients are outside this test’s scope; keep the existing
coverage description unchanged.

Comment threaddocs/architecture.md Outdated
Comment on lines +6 to +9
- Superseded by: ADR 0024 — the legacy-fallback clause ONLY (the
"`gemini_service.py` stays as the fallback during migration" posture).
The framework adoption itself stands; `services/gemini_service.py` was
deleted in #151.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Remove the stale migration-era claims from this ADR.

The new metadata says the fallback clause is superseded and services/gemini_service.py was deleted, but Lines 13, 17, and 26 still say current calls use that module, that it remains during migration, and that it is the legacy fallback. Rewrite those passages as historical context or remove them so ADR 0001 does not contradict ADR 0024.

Suggested direction
-Today every LLM call in the backend goes through `services/gemini_service.py`.+During the original migration, LLM calls went through `services/gemini_service.py`.-The existing `services/gemini_service.py` stays as-is during migration.+The migration-era fallback was removed by `#151`; current calls use agents.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0001-adopt-pydantic-ai.md` around lines 6 - 9, Update ADR
0001’s passages around the references to gemini_service.py, including the
sections at lines 13, 17, and 26, to remove claims that it is currently used,
remains during migration, or serves as the legacy fallback. Rephrase them as
historical context or remove them, while preserving the framework-adoption
decision and consistency with ADR 0024.

Comment on lines +51 to +52
unchanged. ADR 0024 records the full server+client rung ladder as the
canonical description post-#151.)* (`CancelledError`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Update the retry guarantee after adding the tool-write caveat.

This ADR now points to ADR 0024 as canonical, but Lines 82-84 still claim that nothing is persisted on stop/failure, contradicting Lines 46-50 where graph/mastery tool writes may already have landed. Replace the old statement with the narrower guarantee that transcript persistence is completion-only; side-effecting failures remain non-retryable.

Suggested wording
-No backend change ... nothing persisted on stop/failure ...+Transcript persistence remains completion-only, but graph/mastery tool writes+may persist before a failure; those errors are marked non-retryable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0020-streaming-tutor-interrupt-retry.md` around lines 51 - 52,
Update the retry-guarantee statement in ADR 0020 to remove the claim that no
data is persisted on stop or failure. State instead that transcript persistence
occurs only on completion, while failures after graph or mastery tool writes
remain non-retryable, consistent with the canonical behavior referenced by ADR
0024.

Comment on lines +90 to +94
- **Document upload `/upload/sync`** — both guardrail exceptions and bare
exceptions map to a retry-friendly **502** (`UPLOAD_FAILED_DETAIL`):
nothing was persisted and the client mints a fresh X-Request-ID per
attempt, so retrying re-runs the pipeline. Guardrails log WARNING; bare
exceptions log the full traceback.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== repo files =="
git ls-files | rg '(^|/)(docs/decisions/0024-retire-legacy-gemini-seam.md|backend/agents/document.py|.*upload.*|.*sync.*)'||trueechoecho"== ADR excerpt =="
sed -n '1,150p' docs/decisions/0024-retire-legacy-gemini-seam.md | cat -n
echoecho"== backend/agents/document.py outline =="
ast-grep outline backend/agents/document.py --view compact ||trueechoecho"== backend/agents/document.py relevant excerpt =="
sed -n '1,260p' backend/agents/document.py | cat -n
echoecho"== search for process_document and upload/sync references =="
rg -n "def process_document|process_document|/upload/sync|upload/sync|UPLOAD_FAILED_DETAIL|X-Request-ID|request-id" -S backend docs ||true

Repository: SaplingLearn/Sapling

Length of output: 35123


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== documents route sync handler =="
sed -n '90,125p' backend/routes/documents.py | cat -n
sed -n '521,625p' backend/routes/documents.py | cat -n
echoecho"== documents route save/persistence code =="
sed -n '250,340p' backend/routes/documents.py | cat -n
rg -n "def _save|save_assignments_to_db|insert_documents_row|upload_document|raise HTTPException|exposed_http_exception|guardrail" -S backend/routes/documents.py backend/agents -A 8 -B 4
echoecho"== graph apply implementation =="# Inspect the graph tool directly and related call sites
fd -a "graph.py|graph\.py" backend | sed 's#^\./##'
sed -n '1,300p' backend/agents/tools/graph.py 2>/dev/null | cat -n ||trueechoecho"== dbos/exposed_http_exception implementation =="
sed -n '1,240p' backend/services/durable.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 49320


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph service implementation =="
sed -n '1,220p' backend/services/graph_service.py | cat -n
echoecho"== sync rest of upload_document_sync =="
sed -n '610,650p' backend/routes/documents.py | cat -n
echoecho"== request_id idempotency tests =="
sed -n '920,970p' backend/tests/test_documents_routes.py | cat -n
sed -n '1116,1165p' backend/tests/test_documents_routes.py | cat -n
sed -n '1230,1325p' backend/tests/test_documents_routes.py | cat -n

Repository: SaplingLearn/Sapling

Length of output: 21834


🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== graph_service apply_graph_update and insert helpers =="
rg -n "def apply_graph_update|def .*graph.*|insert\(|graph_nodes|new_nodes|graph_ids|rowids" backend/services/graph_service.py -S -A 25 -B 8
echoecho"== graph service read-only static coverage =="
python3 - <<'PY'from pathlib import Pathp = Path("backend/services/graph_service.py")s = p.read_text()checks = { "contains apply_graph_update": "def apply_graph_update" in s, "contains graph_nodes table select": 'graph_nodes"' in s, "contains graph_ids table select": '"graph_ids"' in s, "contains insert": "insert(" in s, "contains idempotency/upsert on request_id": any(tok in s for tok in ('request_id"', 'X-Request-ID', 'on_conflict', 'is.null', 'is_not.null')),}for k,v in checks.items(): print(k, v)if "def apply_graph_update" in s: lines=s.splitlines() for i,l in enumerate(lines,1): if "def apply_graph_update" in l: for j in range(max(1,i-3), min(len(lines), i+65)+1): print(f"{j:04d} {lines[j-1]}") breakPY

Repository: SaplingLearn/Sapling

Length of output: 21312


Clarify the retry-safe boundary for upload failures.

process_document merges concepts via apply_concepts_to_graph before returning, and sync’s document insert happens after the same call. A fresh X-Request-ID retry is safe only if no upload side effects persist after an exception, or the graph merge is idempotent/transactional with doc persistence; otherwise the ADR’s “nothing was persisted” claim is inaccurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/decisions/0024-retire-legacy-gemini-seam.md` around lines 90 - 94,
Clarify the `/upload/sync` retry-safety statement around `process_document` and
`apply_concepts_to_graph`: do not claim that nothing was persisted unless graph
merging and document insertion are atomic or the merge is idempotent. Update the
ADR to describe the actual side-effect boundary and retry behavior, including
the conditions required for a fresh `X-Request-ID` retry to be safe.

…test/ADR gaps
- CLAUDE.md/architecture.md claimed scripts/_raw_gemini.py was the ONLY raw
google-genai caller while rag_service.py's gated embedding client exists,
and contradicted the #439 gate rule as worded — both now enumerate the two
sites and scope the rule.
- frontend/e2e/streaming.spec.ts item-3 header described the deleted
gemini_service seam in the present tense; rewritten for the post-#151
agent-based Rung-1.
- ADR 0024 now cross-references #154 (the post-roll structure it preserves).
- New events-sink test: streaming /upload agent failure emits document.upload
but never document.processed (sync twin already existed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AndresL230

Copy link
Copy Markdown
CollaboratorAuthor

Code review

Found 1 issue:

  1. The PR's own doc sweep contradicted itself on the raw-client inventory (CLAUDE.md says "Code below the agents/_providers.py seam must never construct a raw google.genai.Client without a model_mode() gate (RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439)"): the newly added conventions bullet claimed scripts/_raw_gemini.py is "the one raw-google-genai exception" — but backend/scripts/_raw_gemini.py constructs its client with no model_mode() gate while the RAG embedding path sits below the SAPLING_MODEL_MODE seam — live embed calls fire even in function mode #439 rule stood unqualified two bullets down, and services/rag_service.py's gated embedding client makes "the one exception" / architecture.md's "only raw google-genai caller left" factually wrong.

Sapling/CLAUDE.md

Lines 85 to 87 in eb7cbfd

- Display names are resolved via `services/profiles.py` (`get_display_name`/`get_display_names`), which decrypts off `user_profiles` — don't read name columns off `users`.
- All LLM calls are Pydantic AI agents in `backend/agents/` (model slots in `agents/_providers.py`); there is no other sanctioned LLM seam (ADR 0024). The one raw-`google-genai` exception is `scripts/_raw_gemini.py`, benchmark-only by contract.
- Knowledge-graph mutations go through `services/graph_service.py::apply_graph_update` — routes never write `graph_nodes`/`graph_edges` directly.

Every LLM call is a Pydantic AI agent under `backend/agents/`; there is no other sanctioned LLM seam (the legacy `services/gemini_service.py` raw-call helper was deleted in #151 — ADR 0024). `agents/_providers.py` is the single chokepoint: per-task model slots (`model_for(task)`, overridable via `SAPLING_MODEL_<TASK>` env vars), loop-safe Google providers, and the `SAPLING_MODEL_MODE` seam (ADR 0019) that swaps every agent onto deterministic FunctionModel handlers for tests and the E2E lanes — code below that seam must never construct a raw `google.genai.Client` without a `model_mode()` gate (#439; `services/rag_service.py` holds the one gated embedding client). Usage/cost capture rides `agents/usage.py::record_agent_usage` on every run (#118), and the eval harness (ADR 0021) replays six committed cassette datasets against the agents' real prompts and schemas. When an agent fails, routes degrade honestly — mapped 413/502s, terminal SSE errors, or best-effort empty results — never a second prompt stack (the rung ladder is specified in ADR 0024). The only raw `google-genai` caller left is `scripts/_raw_gemini.py`, a benchmark-only baseline helper whose docstring forbids importing it from application code.

Fixed in 4b61aec — both docs now enumerate the two raw-client sites (gated rag_service.py embedding client + out-of-request-path benchmark baseline) and scope the #439 rule. Three sub-threshold findings were fixed in the same commit: the stale frontend/e2e/streaming.spec.ts item-3 header still describing the deleted gemini_service seam in the present tense, a missing streaming-route events-sink test (document.upload emitted / document.processed never — the /upload/sync twin existed), and ADR 0024 not cross-referencing #154.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@AndresL230
AndresL230 merged commit 83f5195 into mainJul 30, 2026
7 of 8 checks passed
@AndresL230
AndresL230 deleted the feat/b7-151b-documents-cutover branch July 30, 2026 16:10
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1] Agent migration: retire call_gemini* + gemini_service.py (final cutover)

1 participant

@AndresL230