Found by the code review on #554 (H2), which was specifically about making this class of drift visible. This is a live instance of it, in the reader #554 did not touch.
The bug
services/course_context_service.py::_parse_quiz_context_to_arrays builds three arrays out of each student's decrypted quiz_context.context_json:
formincj.get("common_mistakes", []): ... # ✅ the key QuizContext writesforwincj.get("weak_areas", []): ... # ✅ the key QuizContext writesforexpincj.get("effective_explanations", []): # ❌ nothing writes thisagents/quiz_context.py::QuizContext — the only producer of context_json — has exactly five fields: weak_areas, common_mistakes, questions_seen_summary, recommended_difficulty, notes. There is no effective_explanations.
Grepping the whole repo for the name finds it only in two migrations, seed_local_rich.py (which seeds []), and this reader. No writer anywhere.
Consequence
offering_concept_stats.effective_explanations has been persisting an empty array for every offering since the column existed, and an empty array is indistinguishable from "this class produced no effective explanations". That is the #548 failure mode verbatim — the coercer looking for common_errors while the agent wrote common_mistakes, symptom being an empty result that looks like an honest absence.
Its two siblings in the same function read the right keys, which is what makes this easy to miss.
Verified
common_misconceptions ← common_mistakes — correct, reads the live key.prerequisite_gaps ← weak_areas — correct.effective_explanations ← effective_explanations — dead.
(The separate observation that common_misconceptions is empty on staging and prod today — 0 of 72 and 0 of 73 rows — is explained by #529 having starved this table's input for 51 days, not by this bug.)
The decision this needs
Two ways to close it, and they are not equivalent:
- Make it real — add
effective_explanations to QuizContext so the post-submit digest records what explanations worked for a student, and the class aggregate becomes meaningful. That is a genuine product capability nobody is currently getting; "what explanation lands for this class" is arguably the most useful thing this table could hold. - Delete it — drop the read (and consider dropping the column) so nothing implies data that never existed.
Option 1 is the better product answer, but it widens what class-derived text the tutor may surface, so it should go through the same consent framing as #558's k-anonymity floor before shipping.
Either way, the array should stop silently claiming to be an answer.
Found by the code review on #554 (H2), which was specifically about making this class of drift visible. This is a live instance of it, in the reader #554 did not touch.
The bug
services/course_context_service.py::_parse_quiz_context_to_arraysbuilds three arrays out of each student's decryptedquiz_context.context_json:agents/quiz_context.py::QuizContext— the only producer ofcontext_json— has exactly five fields:weak_areas,common_mistakes,questions_seen_summary,recommended_difficulty,notes. There is noeffective_explanations.Grepping the whole repo for the name finds it only in two migrations,
seed_local_rich.py(which seeds[]), and this reader. No writer anywhere.Consequence
offering_concept_stats.effective_explanationshas been persisting an empty array for every offering since the column existed, and an empty array is indistinguishable from "this class produced no effective explanations". That is the #548 failure mode verbatim — the coercer looking forcommon_errorswhile the agent wrotecommon_mistakes, symptom being an empty result that looks like an honest absence.Its two siblings in the same function read the right keys, which is what makes this easy to miss.
Verified
common_misconceptions←common_mistakes— correct, reads the live key.prerequisite_gaps←weak_areas— correct.effective_explanations←effective_explanations— dead.(The separate observation that
common_misconceptionsis empty on staging and prod today — 0 of 72 and 0 of 73 rows — is explained by #529 having starved this table's input for 51 days, not by this bug.)The decision this needs
Two ways to close it, and they are not equivalent:
effective_explanationstoQuizContextso the post-submit digest records what explanations worked for a student, and the class aggregate becomes meaningful. That is a genuine product capability nobody is currently getting; "what explanation lands for this class" is arguably the most useful thing this table could hold.Option 1 is the better product answer, but it widens what class-derived text the tutor may surface, so it should go through the same consent framing as #558's k-anonymity floor before shipping.
Either way, the array should stop silently claiming to be an answer.