Uh oh!
There was an error while loading. Please reload this page.
DB Modular Redesign: monolith → 8 bounded domains (migrations 0019–0028) - #279
Conversation
Design spec (conventions charter, per-domain target DDL, catalog transform, migration sequencing) and the epic rollout plan (10 PR slices into epic/db-modular-redesign, PR1 detailed, staging->prod promotion runbook). Docs only; seeds the epic branch. Supersedes #137/#138/#142/#259/#260. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docs(db): modular redesign spec + epic rollout plan
Authors the complete modular target schema as one coherent set: terms+schools+updated_at trigger (0019), academics catalog/offering/enrollment split with data-preserving catalog transform (0020), gradebook re-keyed to enrollment (0021), analytics re-keyed to offering (0022), graph integrity + mastery events (0023), identity profile split (0024), study/sessions integrity (0025), ops cleanup (0026). Text PKs, FKs+ON DELETE, real types, CHECK enums read off code, encryption columns kept TEXT. Validated 0001->0026 end-to-end against Postgres 15 incl. the catalog transform. Source of truth for the per-domain code PRs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nts ids The renamed baseline tables inherited a TEXT PRIMARY KEY with no default, unlike every other table in the redesign. Add gen_random_uuid()::text so the app does not have to supply ids on insert for these two tables.
… redesign Folds the in-flight DB changes from origin/Gradebook and origin/chore/staging-ops-scripts into the redesign so nothing is lost and the schema stays the single source of truth: drop-lowest -> gradebook_categories.drop_lowest; bell-curve policy -> enrollments.curve_*; per-assignment curve stats + gradescope_assignment_id -> assignments; gradescope_credentials + gradescope_course_links (link re-targeted to enrollment_id) -> 0027; newsletter_emails.approved_at -> 0026. Those branches' migration files are now superseded; their CODE rewire is tracked in filed issues. Validated 0001->0027 against Postgres 15. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(db): schema-foundation migrations 0019-0026 (target schema)
…rollments (epic slice PR2) Code-only slice against the already-landed academics split (migrations 0019-0027). The public API keeps the abstract `course_id`; the term/semester becomes a real second axis; enrollment resolves to a per-term offering internally. - services/academics.py (new): term/offering/enrollment resolver — current_term (date-derived, latest-term fallback), list_terms, resolve_offering (current term, create-if-missing so new enrollments land in the real current semester), offering_course_id, user_offering_ids_for_course, term_for_offering. - graph_service: enrollments→course_offerings→courses/terms join reshaped to the legacy flat shape; graph stays keyed on the ABSTRACT course_id; add/color/nickname/ delete resolve offerings; get_courses now surfaces `term`. - course_context_service: offering-scoped analytics (offering_concept_stats/ offering_summary); resolves offering→abstract course for graph_nodes; semester gone. - graph_read: misconceptions read offering_concept_stats by offering_id. - onboarding/graph routes: enroll into the current-term offering; GET /api/semesters (routes/academics.py) from terms; learn/profile course resolution via enrollments. `gradebook.py` is intentionally left to db/gradebook-code (PR3, with curve+drop-lowest). Other-slice callers (documents/quiz) degrade gracefully until their slices land. Tests: +test_academics; updated graph_service/shared_course_context/graph_read_tools/ onboarding/learn suites. Full backend suite 724 passed; ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
db/academics-code: rewire app onto courses/offerings/terms/enrollments (epic slice PR2)
…ice) Rewire the ops domain's code onto the already-landed 0026_ops schema, which dropped the SERIAL integer PKs on feedback/issue_reports and recreated them with TEXT PKs (gen_random_uuid()::text) plus real FKs to users(id) — and sessions(id) ON DELETE SET NULL for feedback. - routes/feedback.py: hand-build the text PK with str(uuid.uuid4()) on both the feedback and issue_reports inserts, per the repo convention (academics.py, graph_service.py), instead of relying on the dropped SERIAL default. The new user_id/session_id FKs are already satisfied by the request body / session. - tests/test_feedback_routes.py: new coverage for the two POST endpoints (previously untested) — asserts the insert carries a UUID text PK and the body fields round-trip, using the MagicMock-per-table factory pattern. - routes/admin.py allowlist approve/revoke already read/write newsletter_emails.approved_at as 0026 declares it (issue #267) — verified, no code change needed. Plan: docs/superpowers/plans/2026-06-24-db-ops-code.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewire the identity domain onto the 0024 schema split. Public-profile fields (name/first_name/last_name/username/avatar_url/bio/location/website/year/ majors/minors/learning_style) now live on a 1:1 user_profiles table; users keeps only identity + auth + activity. One source of truth per field — nothing is written to both users and user_profiles, nor duplicated onto user_settings. - routes/profile.py: _get_user_or_404 reads users (id/email/streak/created_at) and merges user_profiles; new _get_or_create_profile helper (ensure-row + decrypt); username uniqueness + writes go to user_profiles; avatar_url persists to user_profiles; _SETTINGS_COLS drops the moved columns. - routes/auth.py: get_me reads name/username/avatar_url from user_profiles; google_callback writes profile fields to user_profiles (insert/upsert) and keeps only email/auth/activity on users; oauth_tokens.expires_at sent as None (not "") when absent, since it is TIMESTAMPTZ now. - routes/onboarding.py: the profile-field write moves to a user_profiles upsert; onboarding_completed stays on users. Enrollment loop untouched (academics owns it). - models: drop display_name from UpdateProfileBody and the moved fields from SettingsResponse. - tests: profile/onboarding/decrypt-boundary updated for the split; added an ensure-row test and a per-table column-contract pin. Encryption boundary preserved: name/first_name/last_name/bio/location stay 🔒 TEXT (encrypt_if_present at write, decrypt_if_present at read) on user_profiles; users.email stays 🔒 on users. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ed schema Rewire the gradebook onto the academics-split schema (migration 0021): key gradebook_categories + assignments on enrollment_id instead of user_id+course_id. The public API still speaks the abstract course_id plus an optional `semester` (term label, default = current term); routes resolve (course_id, semester) -> the user's enrollment via services.academics. - services/gradebook_service.py: add drop-lowest (per-category, lowest earned/possible ratio), apply_curve (linear z-score bell curve: avg_target + (raw-mean)*(new_sd/sd), clamped 0-100), and credit-weighted GPA (gpa_points + weighted_gpa). - routes/gradebook.py: enrollment resolver (_resolve_enrollment, semester->term via terms.label); curve folded into current_grade; new PATCH /curve and GET /gpa (per-semester + cumulative/transcript). points stay encrypted at write, decrypted at read. - models: semester + drop_lowest fields, assignment_type enum, SetCurveBody. - tests: enrollment-keyed route tests (filter-aware fake table patching both routes.gradebook.table and services.academics.table), service tests for drop-lowest/curve/GPA incl. a hand-computed credit-weighted GPA fixture (3.7*3 + 2.7*4)/7 = 3.1285714. Fixed test_response_decrypt_boundary for the new resolver shape. Plan: docs/superpowers/plans/2026-06-24-db-gradebook-code.md Out of scope: gradescope sync (gradescope_assignment_id column preserved). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…courses table
The academics split (0020/0022) already re-keyed class analytics to the
offering (course_concept_stats→offering_concept_stats, course_summary→
offering_summary) and offering-keyed course_context_service.py. The lone
holdout was routes/social.py::get_students, which still read
table("courses").select("user_id,course_name") — a query against the old
offering-shaped courses table that no longer has user_id or per-enrollment rows.
Resolve a user's courses through the enrollment chain instead
(enrollments → course_offerings → courses) via the PostgREST embedded join,
deduping across offerings of the same abstract course. Response shape of
GET /api/social/students is unchanged.
Adds tests/test_social_students.py (5 tests). No new migrations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>…ents Rewire services/graph_service.py onto the integrity guarantees from migration 0023_graph_integrity.sql: - graph_nodes / graph_edges writes now use UNIQUE-backed upserts (on_conflict on the new (user_id, course_id, concept_name) and (user_id, source_node_id, target_node_id, relationship_type) constraints), replacing the non-atomic select-then-insert dedup. - Mastery changes append a row to the new node_mastery_events table instead of rewriting the dropped graph_nodes.mastery_events JSONB blob, fixing the non-atomic read-modify-write (#247). get_graph batch-reads that table to compute learning_velocity and the trimmed per-node event history (API contract preserved). - Delete db/dedup_nodes.py — its (user_id, concept_name) dedup is superseded by the UNIQUE constraint (#181). Academics-owned graph_service.py logic (enrollment reshape, course CRUD, offering resolution, update_course_context call sites) and the abstract-course graph key are left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Study artifacts (documents/notes/sessions/study_guides/flashcards) now key
on offering_id (0025); the knowledge graph + course context stay on the
abstract course_id. Routes resolve the abstract course id from the API
boundary to the current-term offering via services.academics before
reading/writing, and translate offering -> abstract for every graph path.
- notes_service: offering_id column + soft-delete (deleted_at) on read/delete.
- documents: persist offering_id; soft-delete; resolve offering on upload;
study-guide cache + doc reads key on offering; graph/syllabus/course-context
keep the abstract course id.
- study_guide / flashcards: read+write the offering; expose abstract course id
in responses; flashcards.import_commit resolves offering (nullable).
- learn: sessions key on offering_id; doc context reads by offering; wire the
shared-context block to the session's abstract course (resolves the
db/study-code TODO, no more {} degrade).
- quiz: validate difficulty against the 0025 CHECK enum; stop touching the
dropped graph_nodes.mastery_events column — route mastery writes through
services.graph_service.apply_graph_update (sanctioned path), keyed on the
abstract course id. IDOR 404 still fires before any write.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>db/ops-code: feedback/issue_reports PKs + FKs (epic slice PR8)
db/analytics-code: social.py offering re-keying (epic slice PR4)
db/identity-code: user_profiles split (epic slice PR6)
db/graph-code: graph integrity + node_mastery_events (epic slice PR5)
db/study-code: study artifacts → offering_id (epic slice PR7)
db/gradebook-code: semester-aware gradebook + curve + drop-lowest (epic slice PR3)
…0024 identity split
Migration 0024 moved the public profile (incl. the 🔒-encrypted display `name`)
out of `users` into a 1:1 `user_profiles` table, and renamed
`users.room_id` -> `users.current_room_id`. The identity slice updated identity
files but left cross-domain readers/writers of `users.name` untouched; these
break against the real DB (mocked tests didn't catch them).
- services/graph_service.ensure_user_exists: stop INSERTing `name` into `users`
(column no longer exists). Insert only `{id, streak_count}`; do not create a
user_profiles row (onboarding/oauth own it).
- services/profiles.py (new): get_display_name / get_display_names read
user_profiles by user_id and decrypt the name. Tolerate missing rows.
- Repoint name reads onto the helper, preserving each response shape:
main.list_users (also source room_id from current_room_id),
routes/social.py (room detail, room activity, match_partners, school_match,
get_students), routes/quiz.py (quiz-context student name),
routes/learn.py get_user_name, services/users_search.paginate_users.
- services/users_search: CODE-ONLY fix — it reads table("users") directly, not a
DB view, so no 0028 migration is needed; names now come from user_profiles.
- services/flashcard_import_service.dedup_against_existing: filter the flashcards
link column `offering_id` (0025 renamed it from `course_id`); behavior identical.
Tests: new test_profiles_service; assert ensure_user_exists omits `name`; update
roster/social/users_search/flashcard tests for the new sources. Gate green:
2 known env-only test_storage_service failures only; ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>db/identity-reconcile: route users.name readers through user_profiles + flashcard column (epic follow-up)
…ma (#258) Self-contained, idempotent staging-only seed that lays a small fake demo dataset on top of migrations 0019-0027 so the live app renders the knowledge graph, gradebook, and courses-with-term against a real DB. Runs via `python -m db.seed_staging`; safe to re-run. - 1 demo school, 3 abstract courses, 4 offerings (CS101 in 2 terms — graph mastery is cumulative across terms since it's keyed on the abstract course). - 1 slim user + user_profiles (encrypted name fields), 4 enrollments (incl. CS101 in both terms), 9 graph_nodes / 4 graph_edges / 6 node_mastery_events. - Gradebook (4 categories w/ drop_lowest, 6 assignments w/ encrypted points), plus a document + note on an offering for study endpoints. - Idempotent via deterministic seed-… ids + upsert-on-UNIQUE / insert-if-absent; re-runs add nothing. All 🔒 columns go through encrypt_if_present; enum values read straight off the migration CHECK sets (no guesses). Reuses pre-seeded terms (read-only) — never touches the real catalog. - Hermetic tests patch db.seed_staging.table to a recording FakeTable and assert insertion coverage, FK consistency, enum validity, multi-term, encryption, and idempotency (2nd run adds no rows). Checklist Step 6 references the new command. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
db/seed-staging: idempotent demo seed for the new schema (#258)
… (0028) 0020 renamed courses→course_offerings and dropped the abstract columns but missed course_code, which stayed NOT NULL. Existing rows have it populated, but every NEW offering insert (app resolve_offering/add_course AND seed_staging) omits it → 23502 not-null violation. The abstract course_code lives on `courses` now. Surfaced by seeding staging; would also break add_course against the real DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(db): drop vestigial course_code NOT NULL on course_offerings (0028)
user_profiles is keyed on user_id and has no `id` column, so the idempotency pre-check's `select=id` 400'd against staging. Select a column we're already filtering on (the natural/PK key) instead — works for every table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(db): seed _exists_by uses a real column, not hardcoded id
…ular DB redesign + add dev guide /update-mds across the knowledge docs (per-doc base..HEAD windows): new schema (courses/course_offerings/terms/enrollments, user_profiles split, gradebook→enrollment, analytics→offering, node_mastery_events), services/academics.py + services/profiles.py, the db.migrate / db.seed_staging commands + .env.staging note, and the encryption list (name fields now on user_profiles). Adds docs/db-modular-redesign-dev-guide.pdf for the team. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Warning Review limit reached
More reviews will be available in 49 minutes and 1 second. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughBackend guidance, schema migrations, services, routes, tests, and staging E2E checks were updated to match the modular database redesign. The PR shifts academics, identity, analytics, gradebook, graph, study, ops, and related docs/code paths to the new offering- and enrollment-based schema. ChangesDB modular redesign
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related issues
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The epic cutover. Restructures the Postgres/Supabase schema from one offering-shaped
coursestable + ausersmega-table into 8 bounded domains of typed tables with foreign keys, CHECK enums, a realtermsentity, and one source of truth per fact — and rewires the entire backend onto it. Delivered schema-first (all migrations as one source of truth, then per-domain code slices).What changed
coursessplit into abstractcourses+course_offerings(per term) +terms;user_courses→enrollments(offering_id). New resolverservices/academics.py. API boundary still uses the abstractcourse_id; the split is resolved server-side.usersslimmed; newuser_profiles(1:1) holds name/profile fields (names still 🔒). Newservices/profiles.pyfor decrypted display names.enrollment_id(gradebook_categories, enrollment-keyedassignments); per-semester + cumulative GPA, bell-curve + drop-lowest;decrypt_numericon 🔒 points.graph_nodes.mastery_eventsJSON column → append-onlynode_mastery_eventstable; UNIQUE-backed node/edge upserts.course_concept_stats/course_summary→offering_concept_stats/offering_summary(offering-keyed; free-text semester gone).offering_id+ soft-delete + CHECK enums.feedback/issue_reportsint PK → text PK + FKs.How it was built & validated
Schema landed as migrations 0019–0028; code rewired in per-domain PRs that accumulated on this epic (#264 schema, #268 academics, #269–#274 the 6 domain slices, #275 reconciliation, #276 seed, #277/#278 staging fixes).
ruffclean(92.0,'A-'), user_profiles decrypt, offering analytics, learn/graph-read).0028(vestigialcourse_offerings.course_code NOT NULL, would've brokenadd_coursein prod) and the seed_exists_byid assumption.For reviewers
docs/db-modular-redesign-dev-guide.pdf(the id model, rename cheat-sheet, what to change). Design spec + epic plan underdocs/superpowers/.CLAUDE.md,docs/architecture.md,README.md,ROADMAP.md.migrate --baselineto record 0001–0018, thenmigrateto apply 0019+; the 0020 catalog transform is data-driven. Never run dashboard DDL.Known follow-ups (non-blocking)
Frontend term picker (#260), gradescope code rewire (#265),
schoolspopulation (school surfaces blank), duplicate subject-root hub for multi-term courses.Summary by CodeRabbit
Resolved issues (auto-close on merge to main)
The DB modular redesign resolves these — semesters DB+backend, the missing indexes/FKs/UNIQUE-dedup, atomic mastery, graph write-integrity, social/students, and the staging catalog+environment:
Closes#100
Closes#128
Closes#137
Closes#138
Closes#158
Closes#160
Closes#161
Closes#176
Closes#177
Closes#178
Closes#179
Closes#180
Closes#181
Closes#195
Closes#247
Closes#258
Closes#259
Closes#266
Closes#267
Partially addressed (NOT closed — follow-ups remain): #142 (frontend term UI #139/#140/#141/#260), #265 (Gradescope code rewire), #126 (assignment-notes encryption + response-boundary leaks).