feat(observability): capture LLM token usage + cost (#118) by Darkest-Teddy · Pull Request #375 · SaplingLearn/Sapling

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Darkest-Teddy · 2026-07-21T22:14:16Z

What & why

Closes#118 — capture LLM token usage + cost for every model call and persist to llm_usage.

#118 was blocked by two unimplemented P1 prerequisites; to make it actually function (and testable) this PR delivers the full vertical slice:

[P1] Observability: create events + llm_usage tables (migration) #115 — tables (migration 0032_observability.sql): events + llm_usage. Deviations from the original issue, both deliberate: TEXT ids (not uuid) because users.id is TEXT in this schema (user_andres), and the numbered-migration convention (0032_*) per CLAUDE.md rather than the issue's older migration_*.sql name. No raw-content columns — sensitive data is a 16-hex content_fp only.
[P1] Observability: events_service async fire-and-forget write path #116 — services/events_service.py: fire-and-forget write path. A bounded queue.Queue drained by one daemon worker that batch-inserts through db/connection.py::table(). log_event / log_llm_usage return in microseconds, never raise into request handling, and never touch the DB on the calling thread. Kill switch EVENTS_LOGGING_ENABLED, overflow drop-counter, and a flush_now() test hook. Worker is started/stopped from the FastAPI lifespan.
[P2] Observability: capture LLM token usage + cost (agents + Gemini) #118 — capture + pricing:
- services/llm_pricing.py — normalizes token fields across Pydantic AI (input/output, legacy request/response) and Gemini (prompt/candidates_token_count) shapes, and computes cost_usd from an editable MODEL_PRICING map (unpriced model → NULL + one-time warning).
- agents/usage.py::record_agent_usage(result, feature=, task=, user_id=) — one line per agent run-site, reads result.usage() + the served model, returns the result unchanged, never raises.
- gemini_service.call_gemini* thread a feature arg and capture response.usage_metadata.

Coverage

Every Pydantic AI agent run (document, notes, quiz, chat_tutor, study_guide, social, graph, course_summary, flashcard, calendar syllabus, health) and every direct call_gemini* now writes one llm_usage row. A static AST guard test fails CI if a future agent run-site skips record_agent_usage.

Success criteria (#118)

Every agent run writes one llm_usage row (feature/task/model/tokens).
Every direct Gemini call writes one row from usage_metadata.
cost_usd computed for known models; NULL + logged warning for unknown.
Token normalization unit-tested against fake Pydantic AI result + fake Gemini usage_metadata.
No agent/Gemini helper left uninstrumented (AST guard test).
Capture is non-blocking and failure-isolated (via events_service).

Testing

29 new unit tests (pricing/normalization, events queue isolation + overflow + kill switch + fingerprint-only-content, agent capture, Gemini capture, instrumentation-coverage guard). Full backend suite green (999 pre-existing passing; the 1 unrelated flashcard rate-limit failure is a coarse-timer flake that reproduces on main and is addressed by open PR #351/#346).

Note: llm_usage / events are new tables — run python -m db.migrate before this deploys.

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added end-to-end AI usage tracking across document, learning, quiz, notes, graph, social, study guide, flashcards, and course workflows.
- Introduced background event/LLM-usage logging with an enable/disable switch.
- Added admin-only analytics endpoints for usage summaries, LLM cost breakdowns, and error feeds.
Bug Fixes
- Logging is now failure-isolated so AI requests remain unaffected.
- Sensitive content is stored only as a fingerprint (not raw text).
Tests
- Added/expanded tests covering instrumentation coverage, pricing/cost computation, event/usage logging, and admin analytics behavior.

coderabbitai · 2026-07-21T22:14:25Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:32 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7f876a-f736-4d45-abc7-bd23d2bf79de

📥 Commits

Reviewing files that changed from the base of the PR and between be376eb and a0bf256.

📒 Files selected for processing (28)

backend/agents/document.py
backend/agents/usage.py
backend/db/migrations/0035_observability.sql
backend/main.py
backend/routes/admin_analytics.py
backend/routes/documents.py
backend/routes/graph.py
backend/routes/learn.py
backend/routes/notes.py
backend/routes/quiz.py
backend/routes/social.py
backend/routes/study_guide.py
backend/services/calendar_service.py
backend/services/chat_stream.py
backend/services/course_context_service.py
backend/services/events_service.py
backend/services/extraction_backends/gemini_vision_backend.py
backend/services/flashcard_import_service.py
backend/services/gemini_service.py
backend/services/llm_pricing.py
backend/tests/conftest.py
backend/tests/test_admin_analytics_routes.py
backend/tests/test_agent_usage.py
backend/tests/test_chat_stream.py
backend/tests/test_events_service.py
backend/tests/test_gemini_usage_logging.py
backend/tests/test_llm_pricing.py
backend/tests/test_usage_instrumentation_coverage.py

📝 Walkthrough

Walkthrough

Adds observability tables, token normalization and pricing, asynchronous event persistence, usage instrumentation around agent and Gemini calls, and admin analytics endpoints with aggregation and access control.

Changes

Observability and Usage Tracking

Layer / File(s)	Summary
Usage contracts and normalization `backend/db/migrations/0032_observability.sql`, `backend/services/llm_pricing.py`, `backend/agents/usage.py`, `backend/tests/test_agent_usage.py`, `backend/tests/test_llm_pricing.py`	Adds observability tables, normalizes provider token formats, computes model costs, and provides failure-isolated agent usage recording with unit tests.
Asynchronous event persistence `backend/services/events_service.py`, `backend/main.py`, `backend/tests/conftest.py`, `backend/tests/test_events_service.py`	Adds bounded queueing, batching, content fingerprinting, kill-switch handling, worker lifecycle management, shutdown flushing, and persistence tests.
Instrumented AI execution paths `backend/agents/document.py`, `backend/routes/`, `backend/services/`, `backend/tests/test_gemini_usage_logging.py`, `backend/tests/test_usage_instrumentation_coverage.py`	Wraps agent and Gemini calls with feature, task, and user metadata while preserving existing outputs and validates instrumentation coverage.
Admin analytics API `backend/routes/admin_analytics.py`, `backend/main.py`, `backend/tests/test_admin_analytics_routes.py`	Adds admin-gated endpoints for usage summaries, per-user totals, grouped LLM costs, and paginated error events with date filtering and tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
participant Caller
participant AgentOrGemini
participant record_agent_usage
participant events_service
participant Postgres
Caller->>AgentOrGemini: execute AI call
AgentOrGemini-->>record_agent_usage: return result and usage metadata
record_agent_usage->>events_service: log_llm_usage(feature, task, model, usage)
events_service->>Postgres: batch insert llm_usage row
record_agent_usage-->>Caller: return original result

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Out of Scope Changes check	⚠️ Warning	The admin analytics API and related rollup endpoints are outside `#118`'s stated scope, which explicitly marks them out of scope.	Remove the admin analytics API changes from this PR or split them into a separate follow-up PR.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: observability for LLM usage and cost tracking.
Description check	✅ Passed	The PR description is substantive and covers the summary, changes, testing, and review notes, though it uses custom headings instead of the template.
Linked Issues check	✅ Passed	The PR appears to implement the `#118` observability slice: agent and Gemini usage capture, pricing, normalization, tests, and non-blocking persistence.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/118-llm-usage-observability

⚔️ Resolve merge conflicts

Resolve merge conflict in branch feat/118-llm-usage-observability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

cloudflare-workers-and-pages · 2026-07-21T22:15:32Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`a0bf256`	Commit Preview URL Branch Preview URL	Jul 29 2026, 11:26 AM

+def call_gemini(
+ prompt: str,
+ retries: int = 1,
+ json_mode: bool = False,
+ model: str = MODEL_DEFAULT,
+ feature: str = "misc",
+) -> str:




-def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT) -> str:
+def call_gemini_multiturn(system_prompt: str, history: list[dict], user_message: str, retries: int = 1, model: str = MODEL_DEFAULT, feature: str = "misc") -> str:


+ usage=getattr(response, "usage_metadata", None),
+ provider="gemini",
+ )
+ except Exception:


+ name = getattr(result.response, "model_name", None)
+ if name:
+ return name
+ except Exception:


+ name = getattr(msg, "model_name", None)
+ if name:
+ return name
+ except Exception:


coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/routes/admin_analytics.py`:
- Around line 153-158: Update the scan handling around the _SCAN_CAP check in
the admin analytics route so capped scans are not returned as complete
analytics: either return a clear range-too-large error, or include an explicit
truncation indicator and the actual scanned count in the API response. Ensure
callers can distinguish capped partial results from complete totals.
- Around line 180-205: Apply the private conditional-GET policy to all four
analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205),
the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314),
and the errors endpoint (317-349). Use make_etag, conditional, and cached_json
from services/http_cache.py, ensuring every response containing user-scoped data
sets Cache-Control to private.
- Around line 123-128: Update _resolve_range to parse both supplied and default
bounds as timezone-aware datetimes before constructing the returned ISO values.
Reject malformed or naive timestamps, and reject ranges where from exceeds to by
raising the route’s established 422 validation error so select_with_count() only
receives valid, ordered bounds.
In `@backend/tests/test_admin_analytics_routes.py`:
- Around line 208-213: Update test_default_range_is_last_30_days to freeze or
monkeypatch the clock used by analytics.datetime.now() to July 21, 2026 before
requesting the default usage summary. Keep the existing assertion that the July
fixture rows are included while the 2020 rows remain excluded.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d9d8aab-6827-452f-90a9-9d1e8f795a26

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53655 and be376eb.

📒 Files selected for processing (3)

backend/main.py
backend/routes/admin_analytics.py
backend/tests/test_admin_analytics_routes.py

🚧 Files skipped from review as they are similar to previous changes (1)

backend/main.py

coderabbitai · 2026-07-27T16:07:14Z

+def _resolve_range(from_: str | None, to: str | None) -> tuple[str, str]:
+ """Default to the last 30 days; echo caller-supplied ISO bounds otherwise."""
+ now = datetime.now(timezone.utc)
+ to_iso = to or now.isoformat()
+ from_iso = from_ or (now - timedelta(days=30)).isoformat()
+ return from_iso, to_iso


🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate date bounds before building PostgREST filters.
Raw invalid timestamps can make select_with_count() raise and become a 500; from > to silently produces an empty report. Parse bounds centrally, require timezone-aware values, and reject inverted ranges with 422.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 123 - 128, Update _resolve_range to parse both supplied and default bounds as timezone-aware datetimes before constructing the returned ISO values. Reject malformed or naive timestamps, and reject ranges where from exceeds to by raising the route’s established 422 validation error so select_with_count() only receives valid, ordered bounds.

coderabbitai · 2026-07-27T16:07:14Z

+ if len(out) >= _SCAN_CAP:
+ logger.warning(
+ "admin_analytics scan hit cap %d on %r (total=%d); results truncated",
+ _SCAN_CAP, table_name, total,
+ )
+ break


🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Do not present capped scans as complete analytics.
When Line 153 hits _SCAN_CAP, callers receive partial rows but responses report totals derived from those rows. This underreports usage, users, and cost without informing API clients. Return a clear range-too-large error or expose an explicit truncation indicator with the actual count.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 153 - 158, Update the scan handling around the _SCAN_CAP check in the admin analytics route so capped scans are not returned as complete analytics: either return a clear range-too-large error, or include an explicit truncation indicator and the actual scanned count in the API response. Ensure callers can distinguish capped partial results from complete totals.

coderabbitai · 2026-07-27T16:07:14Z

+@router.get("/usage/summary", response_model=UsageSummary)
+def usage_summary(
+ request: Request,
+ from_: str | None = Query(None, alias="from"),
+ to: str | None = Query(None),
+) -> UsageSummary:
+ require_admin(request)
+ from_iso, to_iso = _resolve_range(from_, to)
+ rows = _scan_range("events", "event_type,user_id,created_at", from_iso, to_iso)
+
+ by_type: dict[str, int] = defaultdict(int)
+ users: set[str] = set()
+ for r in rows:
+ by_type[r.get("event_type") or ""] += 1
+ if r.get("user_id"):
+ users.add(r["user_id"])
+
+ return UsageSummary(
+ range=Range(from_=from_iso, to=to_iso),
+ total_events=len(rows),
+ distinct_active_users=len(users),
+ by_event_type=sorted(
+ (EventTypeCount(event_type=k, count=v) for k, v in by_type.items()),
+ key=lambda e: e.count, reverse=True,
+ ),
+ )


🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Apply the required private conditional-GET policy to all analytics endpoints.
These GET routes do not use make_etag, conditional, or cached_json; /usage/by-user, user-grouped /llm/cost, and /errors can return user-scoped identifiers without Cache-Control: private.
backend/routes/admin_analytics.py#L180-L205: add conditional caching for usage summaries.
backend/routes/admin_analytics.py#L208-L262: use a private conditional response for per-user usage.
backend/routes/admin_analytics.py#L265-L314: use a private conditional response when grouping by user.
backend/routes/admin_analytics.py#L317-L349: use a private conditional response for error records.
As per coding guidelines, “Use services/http_cache.py helpers (make_etag, conditional, and cached_json) for conditional GET caching. Responses containing user-scoped or decrypted data must always use Cache-Control: private, never public.”
📍 Affects 1 file
backend/routes/admin_analytics.py#L180-L205 (this comment)
backend/routes/admin_analytics.py#L208-L262
backend/routes/admin_analytics.py#L265-L314
backend/routes/admin_analytics.py#L317-L349
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/routes/admin_analytics.py` around lines 180 - 205, Apply the private conditional-GET policy to all four analytics handlers: usage_summary (backend/routes/admin_analytics.py:180-205), the per-user usage endpoint (208-262), user-grouped llm/cost endpoint (265-314), and the errors endpoint (317-349). Use make_etag, conditional, and cached_json from services/http_cache.py, ensuring every response containing user-scoped data sets Cache-Control to private.
Source: Coding guidelines

AndresL230 · 2026-07-28T01:50:39Z

Heads-up: PR #431 just merged with backend/db/migrations/0032_rooms_missing_columns.sql and 0033_realtime_publish_room_messages.sql. This PR's 0032_observability.sql now collides with the merged ledger — please rebase onto main and renumber it to 0034 before merging (migrations are append-only; db/migrate.py tracks by filename so nothing breaks at runtime, but the sequence must stay linear).

🤖 Generated with Claude Code

… (fixes staging session_expired) (#409) * feat(errors): extract FastAPI detail from thrown API errors (#361) `fetchJSON` rejects with `new Error(await res.text())`, so a FastAPI failure surfaces as an Error whose message is the raw JSON body. Add a dependency-free helper that reads the `detail` back out of it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover FastAPI detail extraction (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): recover the HTTP status off a thrown error (#361) `fetchJSON` only spells the status out (`HTTP 404`) when the response body is empty, so read it from an attached `status`/`statusCode`, the parsed body, or the `HTTP <code>` message as available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover HTTP status recovery (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): map HTTP statuses to friendly copy (#361) Add humanizeError: status-driven sentences for the cases users can act on (auth, missing, rate limit, 5xx), falling back to caller-supplied copy so it can never surface a raw body. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): declare the term label on EnrolledCourse (#140) /api/graph/{user_id}/courses has always returned the offering's term label; the client type never declared it, so every consumer had to cast through any to reach it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover the status-to-copy mapping (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(api): add getSemesters() for GET /api/semesters (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): add responsive layout primitives to globals.css (#109) Inline styles can't carry a media query, so the app's fixed multi-column shells (Admin's master/detail panes and metric row, Settings' profile field rows) get class hooks here instead. Driving them from CSS rather than `useIsMobile` also makes the first paint correct, since the hook can only flip after hydration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): prefer a human-readable server detail (#361) A FastAPI detail like "Exam not found." is better copy than generic status text, so surface it — but only when it reads like a sentence, so a serialized payload, markup or a stack can never reach the UI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin roles pane on mobile (#109) The role editor rail was pinned at `minmax(280px, 360px) 1fr` with no mobile branch, so the pane overflowed the viewport below ~640px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): scaffold the shared term helper module (#140) termRankFromLabel mirrors the sort_key formula from migration 0019 so a label-only fallback orders identically to the server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin achievements pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): assert no raw body, markup or stack ever reaches the UI (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Admin cosmetics pane on mobile (#109) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): resolve the current term by date (#140) Mirrors services/academics.py::current_term — today within [start_date, end_date], else the highest sort_key — so client and server never disagree about which semester is current. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): reflow the Admin overview metric row on mobile (#109) Four fixed metric cards squeezed to ~75px each at 375px. Drops to a 2x2 grid below 900px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(errors): add an isNotFound predicate (#361) Lets callers branch on "that thing is gone" without string-matching a response body at the call site. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): stack the Settings profile rows on mobile (#109) The username row and the display-name/bio/location/website rows were both hard-coded to `180px 1fr`, leaving ~150px for the input at 375px. They now share the `.settings-field-row` class and collapse to a label-above-control stack below 600px. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover current-term date resolution and the gap fallback (#140) Fixtures are the four terms seeded by migration 0019 verbatim, so a drift between this rule and the backend's shows up here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(errors): cover isNotFound detection (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the exam-load failure toast (#361) `String(err)` rendered the stringified FastAPI body straight into the toast. Keep the real error on the console and show a sentence instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ui): let Dialog consumers pick the initially focused element (#109) Dialog focuses the first focusable node in the panel, which is always the close button. Form dialogs need their first field instead, and `autoFocus` loses that race — React fires it at mount, before Dialog's focus pass. Opt-in and additive; existing consumers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): group courses by term label, most recent first (#140) Ordering keys on sort_key when the semesters payload is available and degrades to the label-derived rank otherwise. Courses with no term go to an 'Other' bucket rather than being dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the guide-load failure toast (#361) Also clear the stale guide so a failed load can't leave the previous exam's content on screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover term grouping, ordering and the unknown bucket (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): land on inline guidance when the exam is gone (#361) A missing exam is a normal state — a deleted assignment, or a stale "recent guides" entry — not a failure. Show the user where to go next instead of firing a red toast at them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move LetterScaleEditor onto the shared Dialog (#109) Drops the hand-rolled portal and its `minWidth: 360` — which overflowed a 360px viewport once the overlay's gutters were counted — for Dialog's `min(420px, 100vw - 32px)` panel. Also picks up the focus trap, Escape handling and scroll lock the hand-rolled version never had. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): partition courses into current and archive (#140) Only courses that rank strictly below the current term are archived. Undatable courses — and every course when /api/semesters gives us nothing — stay in the default list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(semesters): cover partition ordering and the no-semesters fallback (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): offer a retry when a guide genuinely fails to build (#361) Generation failures (502) are usually transient, so keep the message on screen next to a retry instead of leaving the user on a blank panel after the toast times out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): keep regenerate unreachable without a selected exam (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(semesters): derive ordered term labels for the gradebook chips (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the regenerate failure toast (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move EditWeightsModal onto the shared Dialog (#109) `minWidth: 520` made this the worst overflow of the four gradebook modals; it now sits in Dialog's `min(640px, 100vw - 32px)` panel. The footer wraps rather than crushing the "Total: n%" readout against the buttons on narrow screens. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): humanize the flashcard delete and generate toasts (#361) Last two raw-error toasts on this screen. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(study): sharpen the no-exam empty-state copy (#361) Say why an exam is needed, not just that none exist — that's the whole question a user lands on this state with. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(gradebook): read term (not semester) off the courses payload (#140) /api/graph/{user_id}/courses emits `term`; the landing read `(c as any).semester`, which is always undefined. `distinct` was therefore always empty and every signed-in user silently fell through to the hardcoded SAMPLE_SEMESTERS demo chips. The sample chips are now the logged-out preview only — a signed-in user with no terms gets their own empty state instead of another student's fake grades. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move SyllabusUploadFlow onto the shared Dialog (#109) Replaces `minWidth: 460` with Dialog's fluid panel, and lets the category/assignment rows shrink (`minWidth: 0` on the flex text inputs, wrapping on the assignment rows) so the date picker can't push them past the panel edge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study-guide): make the exam-not-found detail actionable (#361) The frontend now renders a FastAPI detail verbatim when it reads like a sentence, so tell the user what to do next instead of just naming the condition. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study-guide): pin the 404 detail as user-facing copy (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): pin the landing chips to the courses payload term (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(gradebook): type the CourseCard test stub instead of any (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): let the guide problem outrank the generic empty hints (#361) Opening a recent guide clears the exam selection, so a missing exam would otherwise stack "No exams for this course yet" on top of the guidance explaining what actually happened. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): move AssignmentModal onto the shared Dialog (#109) `minWidth: 420` overflowed any phone viewport, and the panel had no max-height at all — with the bell-curve section expanded the footer ran off-screen with nothing to scroll. Dialog fixes both and adds the focus trap, Escape handling and scroll lock. `autoFocus` is swapped for Dialog's `initialFocusRef` so the Title field still takes focus on open rather than the close button. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(study): retry the guide that actually failed (#361) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px touch targets for SideNav rows (#110) `8px` vertical padding around a 15px icon left the nav links ~31px tall. Collapsed, the rail is 64px wide minus 6px padding, so the `width: 100%` link already clears 44px horizontally — only the height needed a floor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(a11y): 44px collapse/expand controls in SideNav (#110) The collapse chevron was a 24x24 target and the expand bar 28px tall. Both now match Dialog's 44x44 close button. `flexShrink: 0` keeps the collapse button square when the account name is long — the name block beside it already ellipsizes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(study): cover the missing-exam guidance and retry paths (#361) Drives the screen through the recent-guides rail — the real path to a stale exam id — and asserts a missing exam produces guidance with no toast, while a genuine failure toasts a sentence and keeps a retry. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ui): make useIsMobile hydration-safe via useSyncExternalStore (#110) `useState(false)` + a `matchMedia` effect meant the value was stale for one paint after every mount, and each consumer registered its own listener. `useSyncExternalStore` pins the SSR/hydration snapshot to `false` (so server and first client render still agree, as React 19 requires) while sharing one `MediaQueryList` per breakpoint and updating as early as React allows. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): order the semester chips by the real term calendar (#140) Chips now sort by sort_key from /api/semesters and default to the date-derived current term instead of whichever term the courses payload happened to list first. A failed semesters fetch degrades to the label-derived order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(gradebook): open the term named by ?semester= (#140) Gives the dashboard archive somewhere to land: selecting an archived class opens that semester's gradebook rather than the current one. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): load the term calendar alongside the graph payload (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * polish(study): stop the failure card restating its own title (#361) When no server detail survives, the body falls back to "Couldn't build that study guide" — which was the title too. Give the card a heading that pairs with any reason. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ui): cover the useIsMobile SSR/hydration contract (#110) Seven cases: the server render reports desktop on a mobile viewport, hydration produces no recoverable error either way, the value flips after commit and tracks later changes, and the queried width matches the `max-width: 767px` rules globals.css relies on. Verified against a naive `useState(matchMedia(...).matches)` implementation — it fails three of them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): partition course progress into current and archive (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ui): hide the desktop rail pre-hydration on mobile (#110) The SSR shell always assumes desktop, so a phone painted a 232px SideNav rail until hydration swapped in TopNav. A width-based `@media` rule applies to that first frame, which no amount of hook work can reach. Pairs with the useIsMobile breakpoint, asserted in its test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(images): lazy-load and size the remote avatar images (#111) `Avatar` and `AvatarFrame` render user-supplied URLs with no intrinsic dimensions, so every one of them reserved zero space until it decoded. Explicit width/height give the browser the aspect ratio up front; the CSS `100%` sizing still wins for layout. The two `/sapling-icon.svg` logos in TopNav/SideNav are deliberately left eager — they're local, above-the-fold brand marks already sized by inline styles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(dashboard): extract CourseProgressRow from the courses panel (#140) Same markup, lifted so the current-term list, the archive and the graph overlay can all render a course line without a third copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): group the my-courses panel by semester with an archive (#140) Current-term courses show by default; earlier terms collapse behind an Archive toggle, grouped by label most recent first. Also covers the mobile 'My Courses' tab, which renders the same panel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): gate the hero canvas RAF behind prefers-reduced-motion (#111) The hero projects and sorts 226 nodes and runs an O(n^2) edge pass every frame, forever. globals.css only neutralizes CSS animation, so a reduced-motion visitor was still paying for all of it. Now it paints one static frame and parks, repainting on resize (which clears the backing store) and re-arming if the preference flips mid-session. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph courses key to the current term (#140) The floating course key now lists only current-term courses and offers past terms as a compact Archive that deep-links into each semester's gradebook. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): hoist the floating-card DOM and dataset reads out of the RAF (#111) The tick re-ran `querySelectorAll('.floating-card')` and re-parsed three `dataset` floats per card on every frame. Both are static, so they move to effect setup. The loop also parks under prefers-reduced-motion, keeping each card's resting tilt but dropping the drift, mouse tilt and scroll parallax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): cache spotlight card rects instead of measuring per mousemove (#111) `getBoundingClientRect()` on every pointer sample forces a layout flush. The rect is now taken on `mouseenter` and dropped on scroll/resize — the only things that can move a card relative to the viewport — so a sweep across a card costs one measurement, not one per sample. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(api): carry the HTTP status on failed requests (#361) fetchJSON discarded the status, so a FastAPI failure — which always has a JSON body — reached callers with no status at all. isNotFound had to infer "missing" from the words "not found", which would silently regress into a red toast the day someone reworded a server message. ApiError keeps `message` as the raw body, so existing callers that stringify or read `.message` are unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(courses): group the manage-courses list by semester (#140) Headings only appear once a student has courses in more than one term. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(landing): rAF-throttle the landing scroll handler (#111) `onScroll` wrote inline styles on the hero, the nav and the ambient glow on every scroll event, which fire well above frame rate. Coalesced to one write per frame; the mousemove and scroll listeners are also marked passive since neither calls preventDefault. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): scope the graph legend chips to the current term (#140) Keeps the top-nav legend consistent with the courses key overlay, which already lists only the current semester. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(dashboard): cover semester grouping, archive routing and degradation (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(dashboard): wire the archive toggle to its region for assistive tech (#140) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(gradebook): smoke-cover the four modals moved onto Dialog (#109) The migration is invisible to tsc — a modal that stops opening, loses its Cancel handler, or drops its accessible name still typechecks. These four had no tests at all, so the swap was landing unverified. Also pins initial focus landing on the title field rather than Dialog's close button, which is the specific reason initialFocusRef exists. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): prune the suppression the Landing fix made stale (#140) Reading `term` instead of `(c as any).semester` removed the only no-explicit-any in Landing.tsx, so its suppression entry no longer matches anything. eslint exits 2 on a stale suppression even with zero errors, which fails the CI lint gate — `main` exits 0, this branch did not. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(frontend): make DEPLOY_ENV the single source of truth for env config Staging login bounced to /?error=session_expired: the worker serving staging.saplinglearn.com ran with production config (BACKEND_URL= api.saplinglearn.com), so sign-in round-tripped through the prod backend and came back as a prod-signed .saplinglearn.com cookie that staging's middleware rejected under its own SESSION_SECRET. The deployGuard check that would catch a consistent-but-wrong-target build only arms when DEPLOY_ENV is set, and it wasn't set on either Workers Build. - deployGuard: add resolveFrontendEnv (derive apiUrl/cookieDomain from FRONTEND_ENVS when DEPLOY_ENV is set; fall back to explicit vars otherwise) plus expectedEnvForHost/detectHostConfigMismatch. Unit-tested. - middleware: derive API_URL via the resolver; on a protected route, flag a host/backend mismatch with a loud log + distinct `env_misconfig` code instead of the misleading `session_expired`. - session route: derive cookie Domain from the resolver. - next.config: derive build-time BACKEND_URL/NEXT_PUBLIC_API_URL/COOKIE_DOMAIN from DEPLOY_ENV. - wrangler.toml: set DEPLOY_ENV for [vars] and [env.staging.vars]. - SignInModal: user copy for env_misconfig. - docs: ADR 0020 (root cause + required deploy follow-up). Note: this hardens the repo but does not fix the running deployment — that needs a staging redeploy with DEPLOY_ENV=staging + `wrangler deploy --env staging` and the correct route binding (see ADR 0020). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(deploy): stop the build-command footgun that took staging down ADR 0020's operational follow-up told operators to set a `wrangler deploy --env staging` line and a DEPLOY_ENV build variable, but never said to keep the Build command as `npm run cf:build`. Wiring that up, the frontend-staging Workers Build's *build-command* field got overwritten with `npx wrangler deploy --env staging` — a deploy command in the build slot. That skips `opennextjs-cloudflare build`, so `.open-next/` is never produced and every build failed with "Could not find compiled Open Next config" (~16 red builds across all branches since 2026-07-20). Verified locally: `npm run cf:build` produces `.open-next/worker.js` (the `main` wrangler deploys); `npx wrangler deploy --env staging` alone does not. - ADR 0020: split the two Workers Builds fields explicitly, mandate the Build command stay `npm run cf:build`, and forbid putting a deploy command in it. - wrangler.toml: document the same Build vs Deploy field distinction at the point of configuration. The live fix is still a one-field dashboard revert (Build command back to `npm run cf:build`); this stops the docs from steering anyone into it again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): data-testid convention on six core E2E surfaces (#382) (#410) The browser suite (#385) needs stable selectors. Today shipped code has zero data-testid attributes, so Playwright would have to anchor on CSS classes (utility-ish, non-unique) or copy — both churn on every design pass. Adds a kebab-case `<surface>-<element>` convention, applies it to the six surfaces Chapter-1 drives (sign-in, approval gate, upload modal, tutor composer, quiz answer flow, graph container), and gates drift with a per-file ESLint rule. - docs/frontend-testids.md documents the naming rules, how repeated/list items are disambiguated (stable domain id first, render index as the fallback), the full current inventory, and how to onboard a new surface. - Testids land on the file that actually renders the element, which is not always the screen file: the tutor composer lives in ChatPanel.tsx (single consumer: screens/Learn.tsx) and every quiz control lives in QuizPanel.tsx (screens/Quiz.tsx only mounts it). - eslint.config.mjs gets a `no-restricted-syntax` block scoped to those six files: any <button>/<input>/<textarea> there without a data-testid is an error. Deliberately not repo-wide — the rest of the app has no browser coverage to protect. Attributes and lint config only; no behavior, styling, or logic changes. The SignInModal.tsx edit is strictly additive (open PRs #409/#359 touch that file). Closes #382 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(backend): keyless rag_service import + hermetic LLM egress guard (#411) #378 — services/rag_service.py built a module-level genai.Client with api_key=os.getenv("GEMINI_API_KEY", ""), and genai.Client(api_key="") raises ValueError at construction. That broke `import main` outright without a key (routes/quiz.py and routes/learn.py both pull the module in). Fall back to "dummy-key-for-import" the way services/gemini_service.py and agents/_providers.py already do: imports stay clean and the failure moves to call time, where it is actionable. No behaviour change when a real key is present. #379 — add the autouse `_hermetic_llm_transport` fixture to tests/conftest.py, the LLM sibling of `_hermetic_supabase_client`. It patches the google-genai transport CLASS (google.genai._api_client.BaseApiClient) rather than client instances, so every already-constructed module-level client is covered: gemini_service, rag_service, and pydantic-ai's GoogleProvider. Unstubbed calls now raise UnstubbedLLMEgress("unstubbed LLM egress: ...") instead of making a real, billable request. Unary, streaming, sync, async and the File API side channels are all blocked, and the fixture fails loudly if google-genai ever moves the seam rather than silently degrading to a no-op. Exemptions mirror the existing guards (e2e_staging, integration) plus a new `live_llm` marker for the three deliberately-live tests in test_ocr_pipeline.py. Their existing `_requires_gemini` skipif is invisible to `get_closest_marker`, so a real marker was required; the skipif still keeps them from running without a key. Verified: full suite 987 passed / 5 skipped / 1 pre-existing error (test_ocr_pipeline::test_save_to_db, unchanged from main); CI-equivalent lane 929 passed / 5 skipped; ruff clean; keyless `import main` succeeds. Closes #378 Closes #379 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) (#412) * test(backend): cookie-minting test-auth endpoint for local/test envs (#381) `GET /api/auth/dev-login` was removed and real Google OAuth is not headless-automatable, so pytest and Playwright had no sanctioned way to obtain an authenticated session. Unify the duplicated minter: - New `backend/services/session_tokens.py` owns the one implementation of the `<payload_b64>.<sig_b64>` format `auth_guard._decode_session` verifies, plus the canonical `SESSION_COOKIE_NAME`. - `db/e2e_staging_http.py` and `tests/integration/conftest.py` now use it instead of carrying verbatim copies; the OAuth-callback redirect handoff token in `routes/auth.py` uses it too (byte-identical output, TTL passed explicitly). `auth_guard` reads the cookie name from it. - `tests/test_auth_session_contract.py::_mint` stays an independent re-implementation on purpose: it pins the wire format from the outside. Add `POST /api/auth/test-login`: - Sets the `sapling_session` cookie with the same attributes as the real session AND returns the token in the body, so Playwright global setup can inject it via `context.addCookies()`. - Hard-gated on `APP_ENV in {"local", "test"}` — narrower than `config.IS_LOCAL`, which also covers `development`/`dev`. - The gate is evaluated per request off the live `config` module attribute and returns a stock 404 `{"detail": "Not Found"}` everywhere else, for every request shape (the body is parsed by hand so FastAPI's pre-handler 422 cannot disclose the route). `include_in_schema=False` keeps it out of /openapi.json in all environments. - No DB access: it does not create users or grant approval/roles. 47 new tests cover the production 404, the request-time gate, the real auth_guard round-trip, and byte-identical minting. Closes #381 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(auth): assert test-login mounting via router.routes, not app.routes `test_route_exists_but_is_gated` walked `client.app.routes` looking for `/api/auth/test-login`. How an included APIRouter flattens into the composed app's route list is not a stable API: under the pinned fastapi 0.138 / starlette 1.3 (CI) the sub-router contributes no `.path` entries there, so the set comprehension silently found nothing and the assertion failed — while every behavioural test against the same endpoint passed, because the route itself was mounted and serving correctly. Assert against `auth_module.router.routes` instead, which is a flat list of APIRoute objects with stable `.path` values across both versions. This keeps the test's original purpose: proving the 404 comes from the environment gate rather than from a route that was never mounted. Caught by CI; the local venv runs fastapi 0.136 / starlette 1.0, where the old introspection happened to work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): wave-2 handoff for epic #402 subcutaneous lane (#414) Session prompt for the next wave (#391, #397, #398), committed so a cloud session can pick it up from the repo rather than needing it pasted in. Records what wave 1 established and what it cost to learn: the baseline test counts and the one pre-existing OCR error not to chase, the shadowed grep/find, the missing venv/.env in fresh worktrees, why `env -u GEMINI_API_KEY pytest` can never work, and the local-vs-requirements.lock version skew that made a locally-green test fail CI. Also states the engineering constraints this lane turns on -- assert through a different layer than the one that wrote, make a test fail before trusting it, never weaken a hermetic guard to get green, and treat #398's findings as the deliverable rather than a blocker. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): add skills + autonomy guidance to the wave-2 handoff (#415) * docs(e2e): add skills + autonomy guidance to the wave-2 handoff The handoff covered environment traps and engineering constraints but said nothing about which skills to reach for or how independently to run, so a session picking it up would default to neither. Splits the tooling by what actually resolves where: /sync-context, the context-curator agent, /recall, /log-decision and /log-attempt are committed under .claude/ and work anywhere, while the superpowers and code-review skills are local plugins that may not exist in a cloud session -- those are listed conditionally with a manual fallback for the review fan-out. Calls out that CLAUDE.md already requires /sync-context before agent-building work, which #391 is, and that context-curator is meant to run before touching LLM integration. Adds an autonomy section: execute the wave without asking permission for reversible work, own CI failures rather than reporting a red PR as done, and never end a turn on a plan instead of doing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): make code review gate the merge, not trail it The handoff put /code-review at the end of the wave, after every PR had already merged. That ordering cannot prevent a bad change from landing -- it can only document one after the fact. Wave 1 was run this way and got lucky: the review found nothing above threshold, but anything it had found would already have been on main. Makes review a per-PR merge gate alongside CI, with every finding addressed or explicitly dismissed with a reason. Keeps a wave-end pass, but reframes it as covering interactions between merged PRs rather than as the only review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(e2e): fix false test claim + add destructive-truncate guardrail Review of PR #415 surfaced two real defects in the handoff: - Claimed all four tests in test_local_stack.py assert via table(); only two do. The other two assert on the app's HTTP response. Corrected so an agent doing find-and-replace isn't misled about the current shape. - #397's autouse truncate runs on a direct psycopg connection over SUPABASE_DB_URL, but the only local guard checks SUPABASE_URL, a separate var. .env.staging and .env.production both hold live direct-Postgres strings. Added a non-negotiable requirement to assert SUPABASE_DB_URL is local and fail loudly before any truncate, so an unsupervised run can't silently wipe real data. Same guardrail added to issue #397 and its acceptance criteria. Also flags the psycopg-in-tests pattern as a deliberate test-only exception to the table()-only rule, so a literal reader doesn't stall on the conflict or treat it as licence for psycopg in app code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) (#416) * feat(agents): SAPLING_MODEL_MODE FunctionModel test seam (#391) model_for() now dispatches on SAPLING_MODEL_MODE (default 'real', so production and the hermetic unit lane are unchanged): - real → GoogleModel, still honoring the per-task SAPLING_MODEL_<TASK> override from ADR 0008. - function → pydantic-ai FunctionModel bound to a per-task handler tests register via register_function_handler(). Scripted tool calls run through the real tool registration, arg-schema validation, and retry loop. - cassette → reserved (issue scope) but raises NotImplementedError. - anything else → ValueError (a typo'd mode never silently bills Gemini). The FunctionModel substitutes ABOVE the #379 transport guard: it never builds a google.genai request, so a function-mode run needs no hermetic exemption and runs clean in the default lane. Tests pin that invariant (rides-above-guard + the real-mode counter-check that still trips it). AC: an integration-style test drives note_chat_agent with a FunctionModel and asserts on the LLM-chosen search_course_materials_tool arguments after schema validation; a classifier test proves the retry loop runs for real. +13 tests, no regressions (976 → 989 passed in the CI-ignore lane). ADR 0019 records the decision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX * refactor(agents): review polish on the model-mode seam (#391) Self-review follow-ups, no behavior change: - annotate model_for/_function_model_for as -> Model (the pydantic-ai base) instead of GoogleModel + type: ignore — function mode genuinely returns a FunctionModel, so the honest supertype removes the type lie. - drop the unused unregister_function_handler and ModelMode alias to keep the seam's public surface to just register/clear. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX --------- Co-authored-by: Claude <noreply@anthropic.com> * test(backend): integration fixtures — psycopg raw-SQL seam, truncate isolation, seeded users (#397) (#417) The integration lane existed but only round-tripped through PostgREST both ways (testing the echo, not the DB) or asserted on the app's own JSON. This adds the raw-SQL seam the lane was missing and the fixtures #398 builds on: - db_conn: session-scoped psycopg connection on SUPABASE_DB_URL (dict rows, autocommit) — the raw-SQL assertion seam. Writes go through the app; reads come back through this, never through table(). - _require_local_db_url: the non-negotiable safety gate. SUPABASE_DB_URL is independent of the SUPABASE_URL that _require_local_stack checks, and .env.staging/.env.production hold live direct-Postgres strings, so the truncate could wipe a real project. The gate parses the host (strict, so 127.0.0.1.evil.com is rejected) and RAISES — never skips — on non-local. - _reset_between_tests: autouse truncate of every mutable table + reseed of the rich baseline before each test, making the suite order-independent. The denylist preserves the migration-seeded reference layer + catalog hierarchy (verified to carry no FK to users, so no CASCADE can reach it). - seeded_user factory (distinct approved users) and authed_client / other_user_client, replacing the per-test cookies.set boilerplate. test_local_stack.py is refactored onto the fixtures: the flagship test POSTs a note through the app and asserts the ciphertext at rest via raw SQL; a truncate -isolation pair proves ordering-independence; a distinct-users test and a seeded_user test cover the new fixtures. The safety gate is proven in the DEFAULT hermetic lane (tests/test_integration_ db_guard.py, pure URL logic, no DB) so it gates every PR: +13 tests there (976 → 989), the 9 DB-backed tests skip without RUN_INTEGRATION. No regressions. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * test(backend): migration order pins, encryption round-trip suite, e2e→subcutaneous rename (#398) (#418) Partial delivery of the subcutaneous write-path suite — the pieces provable or low-risk without a running stack: - test_migrations.py (default lane, VERIFIED): pins the runner's apply order. The 0021 pair is load-bearing — 0021_gradebook.sql CREATEs `assignments` and 0021_gradebook_curve.sql ALTERs it to add curve_* columns, so gradebook MUST apply first. sorted(glob()) does exactly that ('.' 0x2E < '_' 0x5F). This also corrects the issue comment, which claimed the sort yields "gradebook_curve before gradebook" — it does not; the pin guards against a rename flipping it. - tests/integration/test_encryption_roundtrip.py: reads every encrypted column from the seeded baseline via the #397 raw-SQL seam and asserts ciphertext at rest + decrypt round-trip across text (decrypt_if_present), numeric (decrypt_numeric, assignments.points_*), and JSON (decrypt_json, sessions.summary_json) — the "silent decrypt regression" sentinel. - tests/integration/test_migrations_ledger.py: the DB-backed half of the migration check (schema_migrations records every file on disk). - Renamed test_e2e_staging.py → test_subcutaneous_staging.py (it drives HTTP routes below the UI; not a browser E2E). Marker `e2e_staging` unchanged. Default lane: +5 verified migration tests (1002 → 1007), no regressions. The integration files are marked `integration` and skip without RUN_INTEGRATION. Remaining #398 scope (test_postgrest_semantics, test_constraints, test_authz_real_rows, and the actual run-to-find-bugs) needs the local stack and is tracked as a follow-up — #398 stays open. Claude-Session: https://claude.ai/code/session_018b3MtMEu2htdGVBGRcrmzX Co-authored-by: Claude <noreply@anthropic.com> * feat(ocr): transcribe text-layer-less pages with Gemini vision Scanned and photographed handwritten coursework carries no text layer, so there are no characters to copy out. Docling's OCR is meant to cover this but crashes on such documents -- `Stage preprocess failed for run 1, pages [13]: std::bad_alloc` -- and the error is swallowed, so the page comes back empty. Docling *does* flag those pages in `fallback_pages`, but the only consumer of that signal was gated behind `OCR_ENGINE=auto` + `GOT_OCR_ENABLED`, and the default engine is `docling`. So in practice the signal was computed and discarded, and a 13-page handwritten practice final extracted to "" -- which then reached the classify prompt as an empty `Content:` block and was answered with an invented summary. Add a Gemini-vision backend that transcribes a rendered page image, and wire it to that existing signal. Deliberately NOT gated on `OCR_ENGINE=auto`, since that gate is precisely why the rescue never fired for real uploads. Chosen over the alternatives for handwritten maths specifically: Tesseract is poor at handwriting, and GOT-OCR needs a ~2GB weight download and is impractical CPU-only. Gemini already backs every other AI path here, and returns LaTeX. Verified end to end on the document that triggered this, with OCR_ENGINE at its default: 0 chars -> 4,507 chars, including row reductions, characteristic polynomials, and \boxed answers. Off by default (`GEMINI_VISION_OCR_ENABLED`): it costs one LLM call per flagged page. Pages with a normal text layer are never flagged, so a text PDF costs nothing. Per-page failures keep whatever Docling produced for that page -- a partial document beats none -- while an unavailability error aborts the loop rather than burning a failed call for every page of a long scan. Also: - extract the OCR cache key into `_ocr_cache_key` and include the new flag, so enabling vision cannot serve the empty string cached from before it was on - correct the comment claiming OCR is deterministic. It no longer is, which matters for content-addressed chunk ids (ADR 0019): two students uploading the same scan only dedup to one embedding if they transcribe identically. Persisting OCR output content-addressed rather than merely caching it is the real fix, and is not attempted here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(observability): activate Logfire ops/error/LLM tracing (#119) (#406) * feat(observability): activate Logfire ops/error/LLM tracing (#119) Turn on Logfire safely and document it. The SDK was already configured (logfire.configure + instrument_pydantic_ai + the scrub_value scrubber), but two gaps kept the success criteria unmet: - instrument_fastapi was never called, so no FastAPI request traces would appear even with a token set. Wire it in main.py. - Enabling FastAPI instrumentation introduces a content-egress path the scrubber cannot reach: OTel records parsed endpoint arguments (request body + params) under `fastapi.arguments.values`, which Logfire does not route through scrub_value (a field named e.g. `body` matches no risky pattern). Drop those arguments at the source via a request_attributes_mapper that returns None, keep headers off (capture_headers=False), and keep the extra argument/endpoint spans off (extra_spans=False). No prompts, completions, chat messages, note bodies, quiz answers, or uploaded document text leave the process on request spans. Also: - Add LOGFIRE_TOKEN to .env.example (optional; dormant when unset via send_to_logfire="if-token-present") and surface it through config.py. - Document Logfire in docs/observability-logging-tracking.md: what it captures vs the owned Supabase events/llm_usage tables (independent, no double-count), how to enable, what is scrubbed, and the in-scope query-string caveat. - Tests: AST guards that fail if the argument-dropping mapper / header / span flags regress, plus an end-to-end test asserting a request body never lands in any exported span. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(flashcards): stop rate-limit retry-after overshooting the window check_rate_limit computed `int(_RATE_WINDOW_SEC - elapsed) + 1`, which returns 61 when the limited calls land in the same clock tick (elapsed == 0) — one second past the 60s window, and it tripped test_sixth_call_returns_retry_after (`assert 61 <= 60`). Use math.ceil of the true remaining time instead: it still rounds a sub-second remainder up to 1 (never 0) but is bounded by the window, so retry-after is always in [1, _RATE_WINDOW_SEC]. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(agents): bound note_chat orchestrator + remaining worker agents with usage limits (#345) * fix(agents): bound note_chat + remaining worker agents with usage limits (#329) Residual from #327/#243: three run-sites still executed without usage_limits, defaulting to library maximums. - note_chat now runs under ORCHESTRATOR_LIMITS; guardrail trips (UsageLimitExceeded / UnexpectedModelBehavior) degrade to an in-band reply with degraded=true instead of an uncaught 500 (no legacy fallback exists for this path per ADR 0017). - note_summary / note_concepts run under WORKER_LIMITS via a shared _run_note_worker helper that converts guardrail trips to 503. - syllabus_extraction in calendar_service now passes WORKER_LIMITS; its caller already degrades gracefully. - Tests pin the usage_limits kwarg at all four run-sites and the new degrade/503 behavior. Closes #329 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes): use noun form in summarize 503 detail (CodeRabbit nit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ori1maMbbFjkpCS7jPgjj * fix(notes,calendar): separate budget trips from model bugs in agent guardrails (#329) Review fixes for the usage-limit guardrails so a deterministic budget trip and a genuine model bug are no longer conflated: - notes worker (_run_note_worker): UsageLimitExceeded -> 413 with an honest "note too long, shortening may help" detail (no transient "try again" wording); UnexpectedModelBehavior -> 500 + logger.exception so a real bug pages us with a traceback instead of hiding behind a 503/WARNING. - note_chat: UsageLimitExceeded keeps the in-band degrade (its budget wording is now accurate); UnexpectedModelBehavior -> 500. Success path now returns degraded: false for schema symmetry with the degrade path. - calendar (extract_assignments_from_file): UsageLimitExceeded degrades with an honest "syllabus too long / split it" warning; model hiccups and bare exceptions keep the generic degrade. _degraded_result gains a `warning=` override. - tests: rewrite the guardrail tests to the new contract and dedup the fake-note fixture into one module-level factory. Note: this revises behavior previously asserted by test_503_when_guardrails_trip and the parametrized note_chat degrade test — UnexpectedModelBehavior is intentionally no longer treated as a budget trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci: re-trigger frontend-staging Workers build Empty commit to re-run CI after the frontend-staging Workers build-command config fix (npm run cf:build restored; the broken wrangler deploy --env staging skipped the OpenNext build). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: AndresL230 <190146319+AndresL230@users.noreply.github.com> * feat(frontend): test environment profile with same-origin API proxy (#380) (#421) Add build:test / start:test npm scripts that produce a production Next build targeting the local stack with ALL API traffic same-origin through the Next /api/:path* rewrite to the local FastAPI on :5000: - NEXT_PUBLIC_API_URL is set explicitly EMPTY so every client fetch is same-origin and the sapling_session cookie always rides along (the landing page falls back to cross-origin http://localhost:5000 when the var is merely unset). - BACKEND_URL=http://localhost:5000 bakes the rewrite destination and satisfies next.config.ts's production-build guard. - Local Supabase URL + demo anon key are inlined so the lazy lib/supabase.ts client initializes instead of throwing. - start:test supplies the runtime side: BACKEND_URL for the middleware session check and the fixed local SESSION_SECRET for the session route. All values are the committed-safe local defaults from .env.local.example, inlined in the scripts (real process env beats .env* files in Next, so the profile is deterministic regardless of a dev's .env.local). Zero new dependencies; middleware.ts and the production `npm run build` are untouched. Recipe documented in docs/local-supabase.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(ocr): route vision transcription through a Pydantic AI agent The vision OCR call built a raw genai.Client and invoked generate_content directly. Three reasons that is wrong here, the third load-bearing: - CLAUDE.md: new LLM-driven code belongs in backend/agents/ as a Pydantic AI agent, not a fresh client. - ADR-0008 made agents/_providers.py::model_for(task) the one place a model is chosen. GEMINI_VISION_OCR_MODEL was a competing knob that bypassed it; the slot is now SAPLING_MODEL_OCR_VISION like every other agent's. - Cost attribution. Logfire's instrument_pydantic_ai() tags every pydantic-ai span with tokens and USD; a raw client call is invisible to it and to the usage capture #118/PR #375 is building. Vision OCR is one metered call per scanned page — plausibly the largest per-document LLM spend in the app, and it would have been the one call the new cost dashboard could not see. The run is bounded by WORKER_LIMITS: it sits in a per-page loop, where an unbounded run multiplies a single runaway page across the whole document. Also fixes a latent bug this refactor surfaced. _extract_text_or_422 is sync but called from both async handlers (routes/documents.py:640, :771), so a bare asyncio.run raises there — and _apply_gemini_vision_fallback's per-page `except Exception: continue` would have swallowed it, silently turning vision OCR into a no-op on the main upload path. _run_from_anywhere hands the coroutine to a worker thread when a loop is already running, copying the context so agent.override and the active span survive. The module contract is unchanged: same function name and signature, same GeminiVisionUnavailableError semantics, GEMINI_VISION_OCR_ENABLED still the switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ocr): cache key, cost ceiling, sequential rescuers, accurate docs Four findings from the review of #420. Cache key omitted the model. _ocr_cache_key claimed to include "every flag that changes the output" but not the vision model, so switching models kept serving the old transcription for the full 30-day TTL. Model and page cap are now in the key, mixed in only when vision is enabled so the vision-off majority keeps its existing entries. GOT_OCR_MODEL_PATH has the same pre-existing gap; the docstring now names it instead of overclaiming. No cost ceiling. Each flagged page is one metered call, and nothing upstream bounds the count: routes/extract.py allows min(max_pages, 50) and the upload path has no rate limit at all. The #182 limit (10 req/60s) was sized when a request meant one bounded local OCR run. GEMINI_VISION_OCR_MAX_PAGES caps it per document, default 10, and logs how many pages it left behind — a silent cap reads downstream as a full transcription. if/elif made the rescuers mutually exclusive. Enabling both meant vision never ran, including on pages GOT-OCR failed to fill, recreating the exact "signal computed then dropped" bug this feature exists to fix. They now run in sequence — GOT-OCR first (local, free), then vision over what it could not fill. Both share one driver; GOT-OCR's gate is byte-for-byte unchanged. Three false claims. .env.example said an unreadable scan "is rejected" — it is not on this base; the upload paths convert only extraction *exceptions* to 422, so "" reaches the classify prompt and the model fabricates. That rejection is PR #419, still open. The module docstring said vision applies to "any engine"; it needs Docling to have run and succeeded. And the cache comment cited ADR 0019 (actually the SAPLING_MODEL_MODE test seam) for content-addressed chunk ids, whose dedup claim is untrue on main and becomes true only under PR #352. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag (#383) (#422) New src/lib/testMode.ts exports IS_TEST_MODE (build-time inlined), random() (mulberry32-seeded drop-in for Math.random), and now() (frozen 2026-03-11T12:00:00Z clock seam, overridable via globalThis.__SAPLING_TEST_NOW__). With the flag on: - KnowledgeGraph2D seeds its initial node positions and takes the reduced-motion path (synchronous fixed-tick settle) so two loads render identical coordinates. - KnowledgeGraph3D forces cooldownTicks=0 (the reduced-motion seam). - Landing page point cloud + floating cards park their rAF loops on a deterministic static frame; the frame's time read goes through now(). - AtmosphericBackdrop paints one still frame with seeded orbs. - HowItWorks/Study set framer-motion MotionGlobalConfig.skipAnimations. - Dashboard freezes the quote to index 0 and routes greeting, week strip, and relative labels through now(); Calendar (dueLabel, cursor, today) and Notetaker (relTime) do the same. Flag off, every seam passes through to Math.random()/Date.now() and no rAF/motion gate changes: production behavior is unchanged. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * test(infra): one-command local stack boot — make e2e-up / e2e-down (#384) (#423) Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: run the integration lane on every push to main (#402) (#427) The subcutaneous suite (#396–#398) currently runs only on manual workflow_dispatch — a real-DB lane that never runs protects nothing. Per epic #402's open decision 3 (lean: main-only first, promote to a PR gate once #388's stability bar holds), trigger it on every push to main while keeping manual dispatch. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci: gate test_extraction_service.py — it needs none of the OCR stack The CI pytest step ignored four files. Three genuinely need what requirements.lock deliberately excludes: transformers (test_extraction_backends), docling (test_docling_integration), live network (test_ocr_pipeline). test_extraction_service.py needs none of them — it stubs every backend it exercises. It was swept into the list with its heavy neighbours, and the consequence is that nothing in it has ever gated a PR: not the OCR engine gating, not the content-addressed cache key (#97), and not the cost ceiling and rescuer sequencing added alongside this change. #420's own fallback and cache-key tests were ungated for the same reason. Verified against the locked (non-OCR) dependency set CI actually installs, using CI's exact command and env: 1069 passed, 23 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): Playwright harness and fixtures (#385) (#428) * test(e2e): Playwright harness and fixtures (#385) Browser-lane foundation for epic #402 — #386/#387/#392–#395 build on this. - frontend/playwright.config.ts: chromium-only, workers=1 (serial to start), retries=2 gated on CI, trace/video/screenshot on failure, JSON reporter (e2e/results/last-run.json) with per-attempt retry indices for #390 flake tracking, timezoneId pinned to America/New_York for the frozen #383 clock. No webServer block: the boot contract belongs to make e2e-up (#384); global-setup fails fast with the exact fix when the stack is down. - e2e/global-setup.ts: health-check the stack, mint a session for rich-user-active via POST /api/auth/test-login (#381) through the same-origin proxy, persist as storageState. - e2e/support/db.ts: the single DB seam — pg over 127.0.0.1:54322 (loopback-exact guard, mirroring #397), TRUNCATE mutable tables RESTART IDENTITY CASCADE with the #397 denylist, re-seed via the canonical db/seed_local_rich.py. - e2e/support/fixtures.ts: auto fixture resets the DB before each test; specs import test/expect from here. - e2e/smoke.spec.ts: one harness proof (authed /dashboard renders app-shell), deliberately not a journey. - build:test now bakes NEXT_PUBLIC_TEST_MODE=1 (the #383 flag; this composition is what it was built for). - ShellFrame: data-testid="app-shell" on both layout variants — the stable authed-shell anchor per the #382 convention. Verified against a cold make e2e-up boot: npx playwright test green twice in a row (truncate/re-seed isolation holds), tsc --noEmit, eslint, vitest (204 passed), and a plain production build all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(e2e): review fixes — testid process + comment accuracy (#385) - Follow docs/frontend-testids.md 'Adding a surface' for app-shell (missed in the initial commit): App shell row in the owning-files table, an `app` inventory section noting ShellFrame.tsx and the smoke-spec anchor role, and ShellFrame.tsx added to the eslint no-restricted-syntax scope (passes clean — the frame renders no intrinsic button/input/textarea). Doc's 'six files' phrasing generalized now that the list has seven. - global-setup.ts: correct the cookie-flags comment — auth.py only sets Secure under an https FRONTEND_URL (config.py), so the local cookie is HttpOnly/Lax; we mint secure:true and Chromium accepts it on http://localhost. - smoke.spec.ts: correct both redirect comments — unauthed /dashboard goes to ${BACKEND_URL}/api/auth/google via the middleware (BACKEND_URL is always set under start:test), not to the landing page. Verified: npx tsc --noEmit clean; npx eslint . 0 errors with ShellFrame.tsx newly in scope (scoped run at --max-warnings=0 clean); vitest 204/204. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): send the transcription prompt in the user turn, not as system Caught by the first real Gemini call anyone has made against this feature. Moving the instruction to `system_prompt` during the agent refactor changed what the model produces. Measured on a rasterized syllabus with known ground truth (231 chars of source text, 0-char text layer): prompt as system_prompt -> 743 chars: \documentclass{article}, five \usepackage lines, \begin{document}, a tabular, \end{document} prompt in the user turn -> 359 chars: clean Markdown table Both transcribe the facts correctly — every assignment, date and type matches. The difference is that as a system prompt, "Use LaTeX for mathematics" reads as a document-format directive rather than an instruction about math notation, so the model emits a whole LaTeX file. The preamble is not cosmetic. extracted_text feeds the classify, summary and concept prompts and is chunked into course_chunks for RAG, so "amsmath" and "booktabs" become candidate concepts on a graph shared by every student in the course — the same pollution this feature exists to prevent, arriving by a different door. Restores the wire shape the original raw-client implementation used (contents=[image, prompt]), verified to produce 358 chars of clean Markdown on the same fixture. The agent seam, the ADR-0008 model slot and the cost attribution are all unaffected — only the placement changes. The test now pins placement in the user turn and asserts the instruction is absent from any system prompt. Revert-proof: reintroducing system_prompt fails it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — study room with two browser contexts (#394) (#431) Two signed-in contexts (rich-user-active + rich-user-second), one seeded room. Both contexts assert receipt of the other's message through the real propagation path — Supabase Realtime postgres_changes signal + decrypting REST re-fetch (#124) — and both users' knowledge graphs render. Zero waitForTimeout: cross-context sends only happen after each context's postgres_changes subscription is server-confirmed ("Subscribed to PostgreSQL" frame). Unblocking migrations (both verified-needed at runtime on the local migrations-only schema): - 0032: add the rooms columns routes/social.py already selects (topic/course/owner_id/updated_at/is_public) — bug #405 made every room listing endpoint 500 (verified: PostgREST 42703); columns stay nullable/unpopulated, the create_room semantics remain open in #405. - 0033: publish room_messages on supabase_realtime (guarded, idempotent) — verified empty publication locally; without it postgres_changes never fire, and the chat has no polling fallback. Harness additions (additive): e2e/support/session.ts mints a second user's storageState (cookie + the sapling_user localStorage identity that UserContext requires) via POST /api/auth/test-login; USER_SECOND joins stack.ts; global-setup.ts takes the #386 branch's localStorage fix verbatim so sibling PRs converge on identical content. Social.tsx joins the #382 data-testid convention (social-* inventory in docs/frontend-testids.md, eslint files array). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(ocr): per-run provider — every second vision call died on a closed loop Found by the live test added here, which is the only thing that could have found it: every other test in this feature substitutes the model, and a FunctionModel has no client and no event loop. Measured against the live API, calling the seam four times in one process: call 1: OK 302 chars call 2: RuntimeError: Event loop is closed call 3: OK 308 chars call 4: RuntimeError: Event loop is closed `_providers._provider` is a module-level GoogleProvider, so its async httpx client binds to the first loop `asyncio.run` creates and dies when that loop closes. Every `run_agent_sync` caller shares this — it is #354, and the sweep is still open in PR #358. Transcription is the only caller that runs in a LOOP, which turns a latent bug into an unusable feature: a 10-page scan alternates success and failure page by page, and `_apply_gemini_vision_fallback`'s per-page `except Exception: continue` keeps Docling's text without a word. Half a document silently degrades to the mangled OCR this feature exists to replace. So this path does not wait for #358. `fresh_ocr_vision_model()` builds a provider per run and is passed as a per-run `model=` override, leaving the shared `_provider` untouched so it cannot conflict with whatever #358 lands. It returns None outside SAPLING_MODEL_MODE=real, where the FunctionModel has no loop affinity and must not be overridden. Four consecutive live calls now pass. The fixture is an image-only math worksheet. A missing text layer alone is not enough to reach vision — Docling ships RapidOCR and reads rasterized prose fine. This page is reached because `_detect_math_without_latex` flags math-shaped content carrying no LaTeX, the scanned-math case the feature is for. Docling alone drops problem 3 entirely as ``; with vision it comes back as `$\sqrt{x^2 + 16} \leq 5$`. Tests live in the `live_llm` lane, not tests/integration/: they need Docling and a real model, not Postgres, and that lane's conftest mandates a running Supabase stack. Opt-in via RUN_LIVE_OCR=1 plus a real key; skipped otherwise, so CI's dummy key is a clean skip. One test guards the premise and fails loudly if Docling ever stops flagging the fixture, since the other two would then pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): journey — seeded session → dashboard (#386) (#429) * test(e2e): journey — seeded session → dashboard (#386) Co-Authored-By: Claude Fable 5 <norepl…

… admin analytics (#120) Rebase of PR #375 (which included stacked #376) onto current main, rebuilt as a fresh application of the branch diff plus the adaptations main now requires. Core (unchanged from #375/#376): - agents/usage.py: record_agent_usage(result, feature=, task=, user_id=) — one-line, never-raising usage capture for every Pydantic AI run. - services/events_service.py: bounded-queue fire-and-forget writer draining events/llm_usage rows through db/connection.table() off the request thread. - services/llm_pricing.py: token-field normalization + per-1K price map; unknown real models record cost_usd=NULL with a one-time warning. - gemini_service: call_gemini / call_gemini_multiturn log via _log_gemini_usage (feature= threaded from callers; json delegates). - routes/admin_analytics.py (+ mount): admin usage/cost rollup endpoints. - tests/test_usage_instrumentation_coverage.py: AST guard — a module that runs an agent without referencing record_agent_usage fails CI. Rebase adaptations: - Migration renumbered 0032_observability.sql -> 0035_observability.sql (main grew 0032-0034 in the meantime); content unchanged, header comment updated. - main.py lifespan: events_service.start_worker()/shutdown() coexists with #406's Logfire wiring (configure/instrument_pydantic_ai/instrument_fastapi). - Conflict resolutions keep main's guardrail semantics and add capture on top: notes.py wraps _run_note_worker results (WORKER_LIMITS + 413/500 mapping intact) and the note_chat try/except (ORCHESTRATOR_LIMITS + degraded reply intact); learn.py wraps the _prepare_chat_run-based _chat_via_agent; calendar_service keeps usage_limits=WORKER_LIMITS. - NEW run-sites landed on main since the branch: * SSE streaming tutor (#349): stream_agent_turn grows an optional on_usage(run_result) hook fed by the final AgentRunResultEvent, called once on the success path before on_complete (tokens are spent even if persistence fails); /chat/stream and /start-session/stream pass record_agent_usage(feature="chat_tutor", task="chat_tutor"). Error rungs and the Rung-1 legacy fallback don't fire it — legacy usage is captured inside call_gemini_multiturn(feature=). Covered in test_chat_stream.py. * gemini_vision_backend: per-page ocr_vision_agent run wrapped (feature="document", task="ocr_vision"; no user_id — extraction is content-addressed and user-agnostic). - SAPLING_MODEL_MODE=function: 'function:<task>' model names record with cost_usd=NULL and NO unpriced-model warning (they are the e2e/CI seam, not real spend); real unknown models keep the one-time warning. Tested. Verification: full backend suite 1275 passed / 27 skipped; ruff clean; AST guard green over all current run-sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:17:55Z

Code review

Found 1 issue:

One bad row silently drops an entire usage batch: _flush_batch inserts up to 50 queued rows (from many concurrent requests) in a single atomic PostgREST call, and on any failure catches the exception and drops the whole batch with only a warning — no per-row retry or bisection. For a table whose purpose is accurate cost/usage rollups, one constraint-violating row discarding up to 49 unrelated valid rows is silent data loss. (bug due to backend/services/events_service.py — except Exception: logger.warning(..., "dropping batch") around the single bulk table(table_name).insert(rows))

Sapling/backend/services/events_service.py

Lines 167 to 183 in f6bc9d3

def_flush_batch(items: list[dict]) ->None: 
"""Insert a batch grouped by table. Errors are swallowed + logged."""
ifnotitems: 
return
grouped: dict[str, list[dict]] = {} 
foriteminitems: 
grouped.setdefault(item["table"], []).append(item["row"]) 
fortable_name, rowsingrouped.items(): 
try: 
table(table_name).insert(rows) 
exceptException: 
logger.warning( 
"events flush failed for table %r (%d row(s)); dropping batch", 
table_name, len(rows), exc_info=True, 
 ) 

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

…ation flag, private caching, poison-row salvage, per-call-site usage guard - admin_analytics _resolve_range: validate from/to as ISO 8601 (422 naming the bad param) and reject from > to; validated strings echoed unchanged. - Surface the 100k _SCAN_CAP: _scan_range returns (rows, truncated) and UsageSummary/UsageByUser/LLMCost carry `truncated: bool = False` (the /errors feed paginates server-side, so it has no cap to surface). - All 4 analytics GETs now send `Cache-Control: private` via a Response param. - test_default_range_is_last_30_days freezes the module clock at 2026-07-21 so the fixture window can't rot after 2026-08-09. - events_service._flush_batch: a failed bulk insert now retries rows one at a time, dropping only the rows that individually fail (per-row debug log + one warning with the drop count); never-raise contract kept, unit-tested with a 3-row batch where only the poison row is lost. - test_usage_instrumentation_coverage: upgraded the file-level substring guard to a real per-call-site AST check — every agent-run site needs an enclosing record_agent_usage, with pass-through runner helpers (return await agent.run(...)) checked at their module-local callers instead. The docstring now states the exact granularity and the cross-module blind spot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

AndresL230 · 2026-07-29T11:24:54Z

The batch-drop issue from the review above is fixed in the latest push: a failed bulk insert now falls back to per-row inserts, dropping only rows that individually fail (one summary warning with the count). Also landed alongside: 422 validation for malformed/inverted from-to ranges, a truncated flag on capped rollups, Cache-Control: private on the four admin GETs, a frozen clock in the 30-day-range test, and the instrumentation guard upgraded from a per-file substring check to a per-call-site AST check.

- migrations counter 0030 -> 0036 - build_system_prompt line + the streamed tutor routes (#349) - admin analytics / usage-observability repo-map entry (#375) - seam bullet notes streamed function-mode runs Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…o end (#117) (#465) * feat(observability): instrument the capture seams — events flow end to end (#117) The #375 foundation (events/llm_usage tables, queue+worker write path, admin analytics API) shipped with zero producers: grep found no log_event call site outside events_service itself. This activates it. Taxonomy (12 events, domain.action, documented in events_service's docstring + a frozen EVENT_TAXONOMY constant tests pin): - error.4xx/error.5xx (middleware): path/method/status_code/duration_ms + the matched route template (bounded cardinality; raw path kept, query strings never recorded). user_id rides request.state, stamped by get_session_user_id (the shared ASGI scope is the only channel that propagates back out of BaseHTTPMiddleware's downstream task). No 2xx/3xx events by design — curated domain events cover success, and per-request rows would blow the analytics 100k scan cap. - Unhandled crashes are captured too: a non-HTTPException propagates THROUGH dispatch without ever producing a response object, so the except-path emits the error.5xx (real duration, request id) and re-raises — empirically-verified gap from the implementation pass; the middleware comment claiming the exception handler covered it was wrong. - auth.login at the two real session-mint sites (google_callback, test-login) — NOT on every decode, which would fire per-request (documented deviation from the issue's literal wording). - auth.permission_denied at all three guard 403s (not_self / not_admin / missing_role:<slug>); the 401 paths stay uninstrumented (the middleware error.4xx already counts them — no double-emitting). - document.upload / document.processed (agent paths AND the ADR-0001 legacy pipeline), quiz.started / quiz.completed (success path only, after the atomic claim), chat.message_sent (message via content= → fingerprint only), session.started (topic fingerprinted, never in payload) / session.ended, note.created. Never text/titles/bodies in payloads — metadata + SHA-256 fingerprints per the 0035 contract. Call sites are deliberately unwrapped: log_event's body already cannot raise; a test proves a raising logger doesn't break a route. Emitted shapes stay byte-compatible with test_admin_analytics_routes' consumer fixture. 28 red-first tests in test_event_capture_seams.py; new frontend/e2e/events.spec.ts proves rows FLOW in the live stack — student action + a 404 with a secret query string, then the seeded admin polls /usage/summary and /errors (secret asserted absent everywhere). Suites: backend 1376 passed + ruff clean; frontend tsc clean, 277 passed. Closes#117. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * review(#465): fix all three findings from the pre-merge review - chat.message_sent now also emitted inside _legacy_chat — fallback turns (agent guardrails tripped / unexpected agent failure, on BOTH chat routes) are fully persisted turns and must count; emitting there is exactly-once since both routes' fallback branches exit before the main-path emissions. request_id rides request.state (reliable inside the SSE generator where the contextvar is not). Mirrors the documents D6 treatment; red-first test forces the fallback. - document.upload moved BELOW the idempotency short-circuit on both upload routes — an X-Request-ID replay of an already-persisted upload no longer inflates the attempt count (red-first test). - auth_guard module comment rewritten: the 403 dual emission (auth.permission_denied audit row + the middleware's error.4xx) is deliberate and now documented as such — the audit row carries the denial reason and actor, which the HTTP-level row cannot know; the old comment's double-count rationale only applies to 401s. Backend 1378 passed, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…w r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(quiz): flip include_answer_key default to false (#546) The #537 client grades every question through POST /attempts/{id}/answer and has sent `include_answer_key: false` on every /api/quiz/generate call since it shipped (frontend/src/lib/quiz/api.ts, tested in api.test.ts and modeled by the e2e quiz specs), so the generate response's per-option answer key is dead weight for the real caller. Flip GenerateQuizBody.include_answer_key's default to false; the field itself stays accepted-but-logged for one release (explicit `include_answer_key: true` still returns `correct` booleans and logs the #546 deprecation breadcrumb) rather than being deleted outright. _INTERNAL_QUESTION_KEYS (provenance, question_hash) already stripped in both flag states, unaffected by this change. Tests: - test_quiz_answers_c.py::TestIncludeAnswerKey: rewritten for the new default (test_default_now_strips_the_key asserts the omitted-flag path is keyless and does NOT log; test_explicit_true_keeps_the_key_and_logs covers the still-accepted opt-in path) + a new hermetic test (test_default_response_is_the_keyless_projection_of_the_real_stored_key) that grounds the "no leak" claim against the server's own _strip_answer_key projection of the real encrypted questions_json, plus a generic recursive key-name walk, rather than just checking `correct` is absent. - test_quiz_routes.py::test_returns_agent_output_in_legacy_wire_shape: now opts in explicitly (`include_answer_key: true`) since it pins the full keyed wire shape, which is no longer the default. - tests/integration/test_quiz_subcutaneous_db.py (#545): added the real-Postgres twin of the hermetic grounding test above (test_generate_default_response_never_reveals_the_correct_option), marked integration; unrun here (needs the local stack + function mode), collection verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): non-circular no-leak anchor + full-response walk (#546 review r1) Fix round 1 on the task-3 review (task-3-findings-r1.md): Important 1 — the recursive key-name walk in both the hermetic test (test_quiz_answers_c.py) and its subcutaneous DB twin (test_quiz_subcutaneous_db.py) only covered `{"questions": served}`, missing generate's five other top-level fields (quiz_id, requested_difficulty, resolved_difficulty, requested_count, delivered_count). A future sibling key (e.g. `answer_key`) would have sailed past it. Both now walk the full response body. Important 2 — `served == _strip_answer_key(stored)` compared the route's default-path output against the SAME function that produced it, so it could not catch a leak introduced inside that projection (or an accidental widening of `_KEYLESS_*_KEYS`) — falsifiable by widening the allowlist while both sides still move together. Replaced with a non-circular anchor: each served question's key set must be a subset of, and each option's key set must equal exactly, a HARD-CODED literal written in the test (not imported from routes.quiz). Verified this actually discriminates by temporarily reverting the model default to True and confirming both tests fail with "unexpected key(s): {'explanation'}" before restoring it. Docstrings on both tests rewritten to stop overclaiming ("any leak... shows up as a diff") and describe the key-name walk as a heuristic backstop, not a proof. M4 — test_quiz_subcutaneous_db.py's new test now guards `_attempt_row(...)` with an `is not None` assertion before subscripting, matching the neighbouring test's pattern. M6 — both caplog assertions in TestIncludeAnswerKey now filter `caplog.records` by `rec.name == "routes.quiz"` before scanning for the deprecation breadcrumb, since caplog's handler captures every propagating logger. M3 (frontend/src/lib/quiz/api.ts:32-33 now overstates #546's tonight scope) is note-only per the reviewer; not fixed here — noted in the task report and PR body. Covering tests: tests/test_quiz_answers_c.py (16 -> unchanged count, docstrings/assertions only) + tests/test_quiz_provenance_e5_e6.py — 40 passed. tests/integration/test_quiz_subcutaneous_db.py collect-only verified (needs the real stack to execute). Full hermetic suite: 2225 passed, 80 skipped (unchanged from before this fix round). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(quiz): make the deprecated answer-key flag countable (#546 review r2) The one-release grace window for `include_answer_key` was gated on a `logger.info`: nothing rolls log lines up, and the breadcrumb only ever fired on an explicit true, so the population the window actually exists for — flag-unaware callers that OMIT the field and silently changed response shape at the flip — had no telemetry at all (F1). Distinguish the three populations by `model_fields_set` (was the field on the wire?) rather than by value, and emit an events_service event per population so the count lands in the #375 admin-analytics by_event_type rollup with no schema or endpoint work (F4, #117 convention): * explicit true -> quiz.answer_key_served (the count that must reach zero before the parameter is deleted; keeps the log line too) * omitted -> quiz.answer_key_flag_omitted (deletion is a no-op for them, but they are the shape-change population) * explicit false -> nothing; every shipped #537 client sends this on every generate and would swamp the rollup Two event types rather than one with a `flag` payload field because by_event_type does not break payloads out: one type would show a single number mixing the population that blocks deletion with the one that doesn't. Also canonicalizes the flag's lifecycle prose on the field declaration (F14) — it was restated at six sites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(quiz): ground the keyless-projection tests + comment sweep (#546 review r2) Test strength: - Both "keyless projection of the real stored key" tests asserted the served shape but never that it WAS the projection: add count equality vs the decrypted stored questions, per-index id/stem/option label+text correspondence, and delivered_count computed off the projected list (F2, F3). - The ~30-line leak check (allowlist + key-set loops + recursive walk) was duplicated across the hermetic and subcutaneous lanes and had already drifted; it moves to one `assert_keyless_projection` fixture in tests/conftest.py, which both lanes use (F11). Sharing a test-side helper is not the circularity the anchor guards against — the allowlists are still literals, never imported from routes.quiz. - Option key drift now reports the symmetric difference, so a MISSING key reports itself instead of only extras (F9). - Pin the exact top-level response key set (F12). - Consolidate the default-{} and explicit-false cases via parametrize, keeping the caplog-silence assert (F13). - Cover the three telemetry populations, including the no-event case. Comments: - test_quiz_routes: the keyed shape is not "what submit_quiz expects" — submit grades from stored questions_json and reads only question_id/selected_label (F5). - test_quiz_answers_c: the #537 client has ALWAYS sent explicit false; the omitting population is flag-unaware callers (F6). - test_quiz_provenance: the keyed branch is no longer the default — this PR is the flip (F7). - frontend quiz/api.ts: the default is false now, so the client's explicit false is belt-and-braces, not load-bearing (F8). - Fix the vacuity-guard wording: the anchors sit below, not above (F10). - Parse the subcutaneous response once (F15). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

github-code-qualityBot found potential problems Jul 21, 2026
View reviewed changes

Darkest-Teddy mentioned this pull request Jul 22, 2026
feat(observability): admin analytics + cost API /api/admin/analytics (#120) #376
Merged
6 tasks

coderabbitaiBot reviewed Jul 27, 2026
View reviewed changes

AndresL230 marked this pull request as draft July 27, 2026 23:50

This was referenced Jul 28, 2026
fix(ocr): review follow-ups for #420 — agent seam, cost ceiling, cache key, sequential rescuers #424
Merged
feat(ocr): transcribe text-layer-less pages with Gemini vision #420
Merged
test(e2e): journey — study room with two browser contexts (#394) #431
Merged

AndresL230 marked this pull request as ready for review July 29, 2026 11:02

AndresL230 force-pushed the feat/118-llm-usage-observability branch from be376eb to f6bc9d3Compare July 29, 2026 11:02

AndresL230 merged commit 8cf38c2 into mainJul 29, 2026
7 checks passed

This was referenced Jul 29, 2026
[P2] Observability: admin analytics + cost API (/api/admin/analytics) #120
Closed
[P1] Observability: create events + llm_usage tables (migration) #115
Closed
docs: CLAUDE.md refresh after the 2026-07-29 backlog clear #460
Merged

coderabbitaiBot mentioned this pull request Jul 29, 2026
fix(backend): close the #136 contract-audit tail + #340 + #72 #464
Merged

AndresL230 mentioned this pull request Jul 30, 2026
feat(observability): instrument the capture seams — events flow end to end (#117) #465
Merged

This was referenced Jul 30, 2026
feat(agents): structured-output retry + validation hardening (#153) #470
Merged
feat(analytics): #121 data layer — bucket param, typed client, hooks, raw admin page #478
Merged

AndresL230 deleted the feat/118-llm-usage-observability branch August 2, 2026 18:30

AndresL230 mentioned this pull request Aug 26, 2026
feat(quiz): flip include_answer_key default to false (#546) #590
Merged
11 tasks