Uh oh!
There was an error while loading. Please reload this page.
fix(db): reconcile staging's ledger by recovering three out-of-band migrations (#316, #265) - #510
Conversation
…igrations Staging's schema_migrations holds three rows whose filenames exist nowhere in this repo, and `git log --all` finds nothing for any of them — they were applied out-of-band. migrate-staging.yml's preflight refuses to apply anything on top of that, correctly, so all 11 pending migrations are stuck. That includes the fixes for #316 (avatars bucket still private, every avatar renders broken) and #265 (assignments_source_check still rejects 'gradescope', so every synced row would violate it). Neither needs new code; both need this unblocked. filename is the ledger's primary key, which leaves exactly one reconciliation move that doesn't involve hand-editing a live ledger: restore the files under their exact recorded names. Staging then sees them as recorded-and-present, while prod and fresh local databases see them as pending and apply them for real. A timestamped name would leave the orphan in place AND re-run the DDL. All three are idempotent, because environments genuinely disagree about whether they ran. 0019_newsletter_approved_at is a no-op everywhere but the ledger — 0026_ops.sql already carries the column and says so in a comment. 0032_retire_summer_2026 changes data and behaviour, and is worth reading before it reaches an environment that matters. It moves Fall 2026's start_date back to absorb the Summer window, because 0019 seeds deliberately contiguous ranges so exactly one term contains any date; deleting Summer without that leaves a 98-day hole where current_term() returns nothing and resolve_offering can't place an enrollment. Consequence: a date in the old Summer window now resolves to Fall. 0033_offering_section_not_null is the one with an actual design argument. 0036 patched NULL-section duplicates with a partial index, but NULL was never the only way to say "no section" — all three seeders write '' while resolve_offering omitted the key and wrote NULL, so the pair 0036 exists to catch could sit in the table as ('' , NULL) and the index could not see it. Collapsing NULL into '' leaves one representation, covered directly by 0020's existing course_offerings_unique. resolve_offering needs no code change; its docstring did, since it cited 0036 by number for a guarantee that index no longer provides. 0036 stays (already applied elsewhere; applied migrations are immutable) and becomes a no-op, dropped by a timestamped migration so it can't be mistaken for the thing holding the invariant up. _LEGACY_NUMERIC_COUNT 45 -> 48 is the one sanctioned exception to #509's guard: these numbers were already spoken for by rows in a live ledger, so they are recovered history rather than newly claimed. The count stays closed at 48. The e2e lane caught the one real defect here: the rich seed pins an offering to summer-2026, so 0032 broke it with an FK violation surfacing as a 409. Retargeted to fall-2026 — the same thing the migration does to real rows. The su26 ids are deliberately NOT renamed: on an existing local database the old row survives and would collide with the new one on course_offerings_unique. Verification: 1553 passed, 38 skipped; ruff clean. Full flocked e2e cycle with a genuine from-empty replay (supabase db reset --no-seed, then all 49 migrations): up/reset/playwright/oracles all 0, 33 journeys green including semester-scope, oracles 0 findings. Stops short of prod deliberately — 0032 is a product decision, and prod's ledger state needs checking first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Warning Review limit reached
Next review available in:21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (14)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with |
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs | frontend-staging | 6422966 | Commit Preview URL Branch Preview URL | Aug 01 2026, 06:59 AM |
AndresL230
commented
Aug 1, 2026
Code reviewFound 4 issues:
Sapling/backend/db/migrations/0033_offering_section_not_null.sql Lines 47 to 49 in b2331aa
Sapling/backend/db/migrations/0032_retire_summer_2026.sql Lines 35 to 39 in b2331aa
Lines 86 to 88 in b2331aa
Sapling/backend/db/seed_local_rich.py Lines 126 to 131 in b2331aa Below the bar but worth folding in while these files are open: 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
… collide Self-review caught two real defects in this PR's own migrations, both invisible to every lane that verified it. course_offerings_unique is UNIQUE (course_id, term_id, section), and Postgres treats NULL as DISTINCT — so (c, t, NULL) and (c, t, '') coexist legally. That is exactly the mixed state 0033's comment describes, since the seeders write '' and resolve_offering wrote NULL. 0033's blind `UPDATE ... SET section = '' WHERE section IS NULL` collapses that pair onto one key and raises a duplicate key error. Two NULL rows for the same course+term collide the same way, and 0036 cannot prevent it because 0036 applies after 0033. 0032's summer->fall repoint has the identical shape: a course with an offering in both terms lands on an occupied key. Not exotic — resolve_offering(create=True) never sets section, so every app-created offering shares the same default. Either failure is worse than one bad statement. apply_migration runs the whole file plus its ledger INSERT in ONE transaction with no per-file recovery, so a collision rolls the migration back AND stops everything queued behind it. Neither verification lane could see this, which is the part worth remembering: the e2e replay starts from `supabase db reset`, so 0033 always ran against a zero-row table, and the hermetic suite mocks the DB layer entirely. "1553 passed, oracles 0 findings" was true and proved nothing here. Both migrations now detect the collision first and RAISE with the offending groups named. Deliberately not auto-merged: the colliding rows are two distinct offerings and enrollments/documents/notes hang off one id or the other, so choosing a survivor is a data call, not something a migration should do quietly. Verified against a scratch Postgres 15 by running the real migration files: 0033 colliding -> aborts, names course_id/term_id/rows, section still NULLABLE 0033 clean -> succeeds, nullable=NO default=''::text, 0 NULL rows 0032 colliding -> aborts, names the collision, summer-2026 still present 0032 clean -> succeeds, summer gone, fall start 2026-05-18, offering moved Also from the review: - CLAUDE.md and db/migrations/README.md still said the legacy NNNN_ set was frozen at 45. It is 48, and the reconciliation exception is now documented as the ONLY sanctioned reason to add one — previously that rationale lived just in a test comment and a PR description. - _PINNED_PAIRS now covers the three duplicate-prefix groups this PR creates, and _RECOVERED_ORPHANS pins all three recovered files by exact name. The count guard could not catch a rename: it would stay at 48 and pass CI while silently reopening the orphan the file exists to close. - The seed comment named the wrong constraint. The upsert conflicts on (course_id, term_id, section), so course_offerings_unique is the conflict TARGET and routes into an UPDATE; a renamed id actually fails on enrollments.offering_id's FK, which has no ON UPDATE clause. - routes/onboarding.py still described resolve_offering as creating a NULL-section offering. 1554 passed, 38 skipped; ruff clean. Full flocked e2e cycle green with the guards in place: from-empty replay applied all 49, 33 journeys passed, oracles 0 findings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Uh oh!
There was an error while loading. Please reload this page.
Staging's migration ledger has three rows recorded under filenames that exist nowhere in this repo — and
git log --allfinds nothing for any of them, so they were applied out-of-band.migrate-staging.yml's preflight refuses to apply anything on top of that drift, correctly, which means all 11 pending migrations are stuck, including the fixes for #316 and #265.Two of them were applied within hours of this being written (
2026-08-01 01:55and03:51), so this is ongoing drift, not archaeology.Why transcription, and why these exact filenames
schema_migrations.filenameis the primary key, and the runner treats any basename it hasn't recorded as pending. That gives exactly one reconciliation move that doesn't involve hand-editing a live ledger: restore the files under their exact recorded names. Then staging sees them as recorded-and-present (neither pending nor orphan), while prod and every fresh local database see them as pending and apply them for real.A timestamped name would not work — it would leave the orphan in place and re-run the DDL.
All three are written idempotently, because environments genuinely disagree about whether they ran: staging recorded them, prod got the newsletter column via
0026_ops.sql, and a fresh database gets them here first.What each one actually does
0019_newsletter_approved_at.sql— a no-op everywhere except the ledger.0026_ops.sql:30already carries the column with a comment saying "Absorbed from 0019_newsletter_approved_at (drift fix; prod already has this column)". Restored purely to clear the orphan.0032_retire_summer_2026.sql— this one changes data and behaviour. It deletes the Summer 2026 term and moves Fall 2026'sstart_dateback to2026-05-18to close the 98-day hole that would otherwise open in0019's deliberately contiguous date cover (current_term()resolves by date; a date in the hole resolves to no term, andresolve_offering(create=True)then can't place an enrollment). Consequence worth stating: a date like today resolved tosummer-2026before and resolves tofall-2026after. Offerings are repointed before the term row is deleted, sincecourse_offerings.term_idis the only FK intoterms(verified against the live schema) and it'sON DELETE RESTRICT.0033_offering_section_not_null.sql—sectionbecomesNOT NULL DEFAULT ''. This is the one with a real design argument behind it, below.The section design, and why staging's version wins
0020madesectionnullable andcourse_offerings_uniqueisUNIQUE (course_id, term_id, section). Plain UNIQUE treats NULLs as distinct, so two NULL-section rows for the same course+term both survive — which is what0036patches, with a partial unique index overWHERE section IS NULL.But NULL was never the only way to say "no section". Every seeder in this repo writes the empty string:
sectionasdb/seed_staging.py:119""db/seed_local_rich.py:140""db/e2e_staging_http.py:69""services/academics.py::resolve_offeringNULLSo a seeded offering and a
resolve_offering'd one for the same course+term were two different values, and0036— scoped toWHERE section IS NULL— couldn't see the pair. The duplicate-offering bug0036exists to prevent walks in through the''door.Collapsing NULL into
''removes the second door: one representation of "no section", covered directly by the constraint that was already there.resolve_offeringneeds no code change — it still omits the key, the DEFAULT supplies'', and a lost race still surfaces as the 409 its existing handler re-selects on. Its docstring and the 409 comment did need updating, since both cited0036by number for a guarantee it no longer provides.0036is left untouched (it's already applied elsewhere; applied migrations are immutable) and becomes a permanent no-op — a partial index over a predicate no row can satisfy.20260801062439_drop_dead_null_section_index.sqlremoves it, so it can't be mistaken for the thing holding the invariant up.The guard bump
_LEGACY_NUMERIC_COUNTgoes 45 → 48, with the reasoning in the file. These aren't newly claimed numbers — the numbers were already spoken for by rows in a live ledger. The count stays closed at 48; a genuinely newNNNN_file still fails CI.What this unblocks
Both of these are already fixed in
mainand blocked solely on the backlog:0041_avatars_bucket_public.sqlis pending; staging still reportsavatars public=false, so every avatar renders broken. That migration's own header says it: "writing this migration does not by itself fix staging — it converges on the nextpython -m db.migraterun."0042_assignments_source_gradescope.sqlis pending; staging'sassignments_source_checkis stillCHECK (source IN ('manual','syllabus')), so everysource='gradescope'insert the merged sync code writes would violate it.Verification
ruff checkclean.test_migrations.py: 0033-before-0036 (reversed, 0036 would index live NULL rows and 0033 would then silently empty it), 0036-before-the-drop, and 0019-before-0032 (reversed, the DELETE matches nothing and 0019 re-seeds the term).Note for whoever does prod
This PR deliberately stops short of prod.
0032is a product decision about which terms exist, and it will apply there on the next run. Prod's ledger state also needs checking first — it may not have one at all, in which casedb.migratewould treat all 48 as pending.🤖 Generated with Claude Code