feat(quiz): mine answers_json into the digest as a mistake profile (#554) by AndresL230 · Pull Request #571 · SaplingLearn/Sapling

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230 · 2026-08-22T08:47:51Z

Closes#554. Workstream H2 of epic #537.

The gold that was never read

quiz_attempts.answers_json has recorded which distractor a student picked, per question, on every submit since the column existed. Nothing has ever read it back.

What the post-submit digest agent actually received was results — labels only:

{ "question_id": "3", "selected": "B", "correct": false, "correct_answer": "C" }

"Picked B, the answer was C" is not something a model can turn into a misconception. It can only guess one. The option text — the thing that makes a wrong answer mean something — was one join away in questions_json the whole time.

The fix

services/quiz_distractors.py does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct.

Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs inside the post-submit BackgroundTask after the attempt is already graded and written. A crash there would cost the student the digest for a quiz they had already finished.

Per the issue: the join happens in the post-submit task, not at generation, so generation pays nothing for it.

Four things it deliberately does not report

Each would teach the digest something false:

case	why it's excluded
correct answers	a "mistake profile" made of right answers spends tokens reporting the absence of a problem
unanswered questions	skipped is not wrong — recording a blank as a distractor choice invents a misconception out of silence
items with no correct option (#129's shape, which grade wrong for everyone)	"the correct answer was nothing" is not a fact about the student
results whose question isn't in the attempt	nothing to say

Digest schema version

The other half of the issue. DIGEST_SCHEMA_VERSION is stamped into context_jsonby the model's default rather than by the LLM — a field the model has to remember to set is a field that goes missing — and the reader warns when it meets a version it doesn't understand.

The drift this exists to catch is #548's: the coercer looked for common_errors while the agent wrote common_mistakes, and the symptom was an empty digest, which is indistinguishable from a new student. Unversioned rows are pre-#554 and expected, not a discrepancy.

Tests

Ten, and they pin the wiring as well as the logic — because str.replace on a renamed placeholder is a silent no-op, so the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Also covers the garbage-in cases, since this code's contract is that it cannot throw.

Verification

Hermetic 2181 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 46 passed.

Two Playwright failures, neither from this PR:

landing-drag-field.spec.ts:332 — e2e lane red on main since #524: a dropped landing node doesn't scroll with the page #566, red on main, fixed by test(e2e): measure a dropped node against the copy it is welded to (#566) #568.
gradebook.spec.ts:35 — test(e2e): gradebook two-terms journey is intermittent in full-suite local runs #569, the intermittent I filed while investigating quiz H1: misconceptions tool filters offering_id with the abstract course id — verify live, then fix + seed test #553 (0 failures in 4 clean-main full-suite runs and 6 isolated runs; it flakes under load and its failure signature is indistinguishable from the regression it guards).

🤖 Generated with Claude Code

) `quiz_attempts.answers_json` has recorded which distractor a student picked, per question, on every submit since the column existed — and nothing has ever read it back. What the post-submit digest agent actually received was `results`, which carries LABELS: "question 3, picked B, the answer was C". That is not something a model can turn into a misconception; it can only guess one. The option TEXT — the thing that makes a wrong answer mean something — was one join away in `questions_json` the whole time. `services/quiz_distractors.py` does that join and hands the digest the wrong answers in words: the stem, the concept, what the student chose, and what was correct. Deliberately dumb — no model call, no I/O, pure data — and it never raises, because it runs in the post-submit BackgroundTask after the attempt is already graded and written, where a crash would cost the student the digest for a quiz they had already finished. Four cases it deliberately does NOT report, each of which would teach the digest something false: * correct answers — a mistake profile made of right answers spends tokens to report the absence of a problem; * unanswered questions — skipped is not wrong, and recording a blank as a distractor choice invents a misconception out of silence; * items with no correct option (#129's shape, which grade as wrong for everyone) — "the correct answer was <nothing>" is not a fact about the student; * results whose question is missing from the attempt. Digest schema version, the other half of the issue: `DIGEST_SCHEMA_VERSION` is stamped into `context_json` by the model's default rather than by the LLM (a field the model must remember to set is a field that goes missing), and the reader warns when it meets a version it doesn't understand. The drift this exists to catch is #548's: the coercer looked for `common_errors` while the agent wrote `common_mistakes`, and the symptom was an empty digest — indistinguishable from a new student. Tests pin the wiring as well as the logic, because `str.replace` on a renamed placeholder is a silent no-op: the profile would be computed, serialized, and dropped on the floor with nothing to show for it. Hermetic 2181 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-22T08:48:00Z

Warning

Review limit reached

@AndresL230, you've reached your PR review limit, so we couldn't start this review.

Next review available in:26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d01c32fe-14bd-4aba-9137-f8e87d141887

📥 Commits

Reviewing files that changed from the base of the PR and between c7f043c and bdbcf4f.

📒 Files selected for processing (8)

backend/agents/tools/quiz_history.py
backend/prompts/quiz_context_update.txt
backend/routes/quiz.py
backend/services/quiz_context_service.py
backend/services/quiz_distractors.py
backend/tests/integration/test_quiz_context_repair_db.py
backend/tests/test_quiz_context_service.py
backend/tests/test_quiz_distractor_profile.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

supabase · 2026-08-22T08:48:27Z

This pull request has been ignored for the connected project ybgqdonkoqftwrmweuyv because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

cloudflare-workers-and-pages · 2026-08-22T08:50:00Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	frontend-staging	`bdbcf4f`	Commit Preview URL Branch Preview URL	Aug 22 2026, 09:03 AM

Three findings, all about the schema-version half rather than the profile. **The version was LLM-controlled.** `schema_version` was a field on the digest agent's output_type, and `submit_quiz` persists `model_dump()` verbatim. Worse, this PR's own prompt feeds the previous digest back in under "update your notes" — so from now on the model would SEE `schema_version: 2` and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is now off the agent schema entirely (so it costs no decoding budget either) and `save_quiz_context` stamps it server-side, which also covers every future writer. **The guard couldn't catch the drift it cited.** A version comparison only fires for a writer NEWER than the reader — a mixed-deploy window. #548 was a key RENAME at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is precisely the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its OUTCOME: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest cannot be all three at once — the agent always writes at least `questions_seen_summary` or `notes` — so that combination means the keys moved. Logged with the actual key set. **The docstring understated the blast radius.** It claimed this runs in the post-submit BackgroundTask. It does not: `submit_quiz` builds the whole prompt string synchronously and hands only the finished string to `add_task`. So an escaping exception 500s the submit AFTER the atomic `completed_at` claim, the mastery write and the score update have landed — the student sees a failed submit for a quiz that scored, and the retry 409s with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure. The review also found a live instance of this same drift class in the OTHER reader of `context_json`: `course_context_service` harvests `effective_explanations`, a key no agent has ever written, so that column has persisted an empty array for every offering since it existed. Out of scope here, filed as #572. Hermetic 2184 passed / 9 skipped, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

`test_save_quiz_context_upserts_one_row_and_encrypts` asserts the payload round-trips exactly, so server-stamping `schema_version` in `save_quiz_context` changed what comes back. The hermetic twin was updated with the fix; this is its real-DB counterpart, and only the integration lane could see it — which is the whole argument for that lane. Integration 56 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

AndresL230 · 2026-08-22T09:04:02Z

Review round — three findings, all fixed

All three were about the schema-version half, not the profile.

1. The version was the model's to change.schema_version was a field on the digest agent's output_type, and submit_quiz persists model_dump() verbatim — so the model owned it. Worse, this PR's own change feeds the previous digest back into the prompt under "update your notes", so from here on the model would seeschema_version: 2 and be invited to update it. A model that helpfully bumped it to 3 would trip the reader's unknown-shape warning on every later read for that (user, concept), forever, with no real drift; one that lowered it would kill the guard just as quietly. The field is off the agent schema entirely now (it also stops costing decoding budget), and save_quiz_context stamps it server-side — which covers every future writer rather than just this one.

2. The guard couldn't catch the drift it cited.version > DIGEST_SCHEMA_VERSION only fires for a writer newer than the reader — a mixed-deploy window. But #548 was a key rename at the same version, and catching that with a version number requires remembering to bump it in the same commit as the rename, which is exactly the discipline that failed the first time. The version now claims only what it can prove, and the rename case is caught by its outcome: a row that is present, stamped with a version this reader understands, and yet yields nothing readable. A real digest can't be all three at once — the agent always writes at least questions_seen_summary or notes — so that combination means the keys moved. Logged with the actual key set, so the next person sees what it drifted to.

3. The docstring understated the blast radius. It said this runs in the post-submit BackgroundTask. It doesn't: submit_quiz builds the whole prompt string synchronously and hands only the finished string to add_task. So an escaping exception 500s the submit after the atomic completed_at claim, the apply_graph_update mastery write and the score update have all landed — the student sees a failed submit for a quiz that scored, and their retry gets a 409 with the score never returned. The guard was right; the comment beside it invited a future reader to delete it as cheap background-task failure.

A live instance of the same bug class, elsewhere

The review also found that course_context_service._parse_quiz_context_to_arrays harvests effective_explanations from context_json — a key no agent has ever written. Its two siblings in the same function read the correct live keys, which is what makes it easy to miss. So offering_concept_stats.effective_explanations has persisted an empty array for every offering since the column existed, indistinguishable from "this class produced none".

I verified it before filing (common_misconceptions ← common_mistakes ✅, prerequisite_gaps ← weak_areas ✅, effective_explanations ← dead ❌). Out of scope here, filed as #572 — it needs a product decision (make it real vs. delete it) rather than a quiet patch.

The integration lane earned its keep

Server-stamping the version broke test_save_quiz_context_upserts_one_row_and_encrypts, which round-trips the payload against a real database. The hermetic twin was updated alongside the fix; only the real-DB lane caught the second one.

Verification

Hermetic 2184 passed / 9 skipped, ruff clean, oracles 0 findings, integration 56 passed, Playwright 47 passed (the one failure is #566, fixed by #568). CI green.

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

AndresL230and others added 2 commits August 22, 2026 04:54

AndresL230 merged commit e139954 into mainAug 22, 2026
8 checks passed

Uh oh!

Conversation

AndresL230 commented Aug 22, 2026

The gold that was never read

The fix

Four things it deliberately does not report

Digest schema version

Tests

Verification

Uh oh!

coderabbitaiBot commented Aug 22, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

supabaseBot commented Aug 22, 2026

Uh oh!

cloudflare-workers-and-pagesBot commented Aug 22, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

AndresL230 commented Aug 22, 2026

Review round — three findings, all fixed

A live instance of the same bug class, elsewhere

The integration lane earned its keep

Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

AndresL230 commented Aug 22, 2026

The gold that was never read

The fix

Four things it deliberately does not report

Digest schema version

Tests

Verification

Uh oh!

coderabbitaiBot commented Aug 22, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

supabaseBot commented Aug 22, 2026

Uh oh!

cloudflare-workers-and-pagesBot commented Aug 22, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

AndresL230 commented Aug 22, 2026

Review round — three findings, all fixed

A live instance of the same bug class, elsewhere

The integration lane earned its keep

Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

AndresL230 commented Aug 22, 2026

The gold that was never read

The fix

Four things it deliberately does not report

Digest schema version

Tests

Verification

Uh oh!

coderabbitaiBot commented Aug 22, 2026•
edited
Loading

cloudflare-workers-and-pagesBot commented Aug 22, 2026•
edited
Loading

coderabbitaiBot commented Aug 22, 2026•
edited
Loading

cloudflare-workers-and-pagesBot commented Aug 22, 2026•
edited
Loading

coderabbitaiBot commented Aug 22, 2026•
edited
Loading

cloudflare-workers-and-pagesBot commented Aug 22, 2026•
edited
Loading

coderabbitaiBot commented Aug 22, 2026•
edited
Loading

cloudflare-workers-and-pagesBot commented Aug 22, 2026•
edited
Loading