[EPIC] Chapter 1 — E2E Regression Suite

[AndresL230]

Problem

The audit (verified against 6765f49, 2026-07-22) found 970 backend tests, all against per-module MagicMock DB stubs; zero browser tests; and CI that never starts a database or applies a migration. The persistence layer is therefore structurally untested — a misspelled column, a NOT NULL violation, a wrong .eq() chain, a broken upsert conflict target, or an encrypt-on-write / decrypt-on-read mismatch would all pass today, because the mock agrees with whatever the caller asserts. Negative-authz tests have the same problem: RLS does not exist (zero ROW LEVEL SECURITY statements in backend/db/migrations/; the backend connects with the service-role key via db/connection.py:10, bypassing it), all access control lives in services/auth_guard.py, and every IDOR test runs against mocks — so a guard that is correct in mock-land but wrong about PostgREST filter semantics passes. Meanwhile the local Supabase stack is running and used by no test (open PR #377 starts closing that gap for the subcutaneous lane).

What this chapter is

The lane split is by how each suite is run and trusted, not by subject matter. Chapter 1 asserts conformance to a spec we wrote, finds regressions, requires determinism, runs on every PR, gates merge, tolerates zero flake, and outputs pass/fail. Two lanes:

• subcutaneous — HTTP → FastAPI → real Postgres; assertions read back with raw SQL (test(backend): integration marker + conftest gating for the real-DB lane #396, test(backend): integration fixtures — psycopg, truncate isolation, seeded users #397, test(backend): subcutaneous write-path suite — encryption, PostgREST semantics, constraints, authz, migrations #398)
• browser — Playwright → Next → FastAPI → Postgres (test(e2e): Playwright harness and fixtures #385–test(e2e): journey — graph render integrity vs DB #395)

plus a plumbing lane that unblocks both (#378–#384).

Success criteria

• ≥6 browser journeys
• Flake <1% over 20 consecutive CI runs
• E2E lane <8 min wall clock
• 100% of encrypted columns covered by a raw-SQL round-trip assertion
• Migration replay from empty gated on every PR

Non-goals

• Replacing the existing mocked suite (it is fast and it stays)
• Adopting RLS policies
• Visual regression / screenshot diffing
• Automating real Google OAuth
• LLM-as-judge scoring as a PR gate

Dependency graph

Regenerated from the per-issue Dependencies sections:

#378 ─┬─> #379 ─> #391 ─────> #392, #393 (each also needs #385)
 └─> #396 ─> #397 ─> #398
 └──────> #400 (Ch2 oracles; also needs #399)

#380 ─> #384 ─┐
#381 ─────────┼─> #385 ─┬─> #386 ─┬─> #388 ─> #389 ─> #390
#382 ─────────┤ ├─> #387 ─┘
#383 ─────────┤ ├─> #394
 └─────────┴─> #395 (needs #383 + #385)

#388 + #391 ──> #399 ─> #400 ─> #401 (Chapter 2, epic tracked separately)

Critical path to a first green browser test: #380 → #384 → #385 → #386, with #381/#382/#383 landing in parallel.

Open decisions

• 1. CI container runtime — Supabase CLI on a Docker runner, or plain postgres + db.migrate? The latter is simpler and kills the Podman/Docker divergence, but loses PostgREST, which the backend actually talks to. Leaning: full Supabase CLI, since PostgREST semantics are half the point.
• 2. CI Postgres version — match local (PG17) or prod (PG15)? Leaning: PG15 to match prod, plus a separate issue to close the local skew.
• 3. Does the E2E lane gate PRs or only main? Leaning: main-only for two weeks, then promote once test(e2e): browser-lane stability gate — 20 consecutive green runs #388 has held.
• 4. Where does Chapter 2 live — this repo, or a sibling so its churn doesn't pollute this one?
• 5. Seed strategy — reuse db.seed_staging, or a separate smaller db.seed_e2e? Leaning: separate, so changing demo data can't break tests. PR Local-Supabase follow-ups: rich seed + integration suite (#363, #362; #365 reverted) #377's db.seed_local_rich (built for feat(seed): rich local dataset for E2E testing #363) is a strong candidate base.

Risks

Risk	Mitigation
Podman/Docker divergence breaks CI	Decision 1; validate on a runner before building on it
PG15/PG17 skew hides a prod bug	Decision 2 + one test asserting server version
Browser lane becomes slow and gets muted	Hard 8-min budget; #388 gate before it can block PRs
data-testid convention decays	Lint rule in #382
Agent lane produces noisy false positives	Non-gating by design; only promoted findings count
#398 surfaces many real bugs at once	Expected. Triage into follow-ups; do not block the epic on fixing them

Children

Plumbing

• fix(backend): align rag_service GEMINI_API_KEY fallback so keyless import works #378 — fix(backend): align rag_service GEMINI_API_KEY fallback so keyless import works
• test(backend): hermetic LLM guard fixture — fail on unstubbed Gemini egress #379 — test(backend): hermetic LLM guard fixture — fail on unstubbed Gemini egress
• feat(frontend): test environment profile with same-origin API proxy #380 — feat(frontend): test environment profile with same-origin API proxy
• test(backend): cookie-minting test-auth endpoint for local/test envs #381 — test(backend): cookie-minting test-auth endpoint for local/test envs
• feat(frontend): data-testid convention on six core E2E surfaces #382 — feat(frontend): data-testid convention on six core E2E surfaces
• feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag #383 — feat(frontend): NEXT_PUBLIC_TEST_MODE determinism flag
• test(infra): one-command local stack boot — make e2e-up / e2e-down #384 — test(infra): one-command local stack boot — make e2e-up / e2e-down

Browser

• test(e2e): Playwright harness and fixtures #385 — test(e2e): Playwright harness and fixtures
• test(e2e): journey — seeded session → dashboard #386 — test(e2e): journey — seeded session → dashboard
• test(e2e): journey — upload → SSE → document appears #387 — test(e2e): journey — upload → SSE → document appears
• test(e2e): browser-lane stability gate — 20 consecutive green runs #388 — test(e2e): browser-lane stability gate — 20 consecutive green runs
• ci: E2E job — real Postgres, migrations from empty, Playwright #389 — ci: E2E job — real Postgres, migrations from empty, Playwright
• ci: flake tracking — surface per-run retry counts #390 — ci: flake tracking — surface per-run retry counts
• feat(backend): pydantic-ai test seam via FunctionModel (SAPLING_MODEL_MODE) #391 — feat(backend): pydantic-ai test seam via FunctionModel (SAPLING_MODEL_MODE)
• test(e2e): journey — tutor conversation persists to messages #392 — test(e2e): journey — tutor conversation persists to messages
• test(e2e): journey — quiz answer → mastery update (UI + DB) #393 — test(e2e): journey — quiz answer → mastery update (UI + DB)
• test(e2e): journey — study room with two browser contexts #394 — test(e2e): journey — study room with two browser contexts
• test(e2e): journey — graph render integrity vs DB #395 — test(e2e): journey — graph render integrity vs DB

Subcutaneous

• test(backend): integration marker + conftest gating for the real-DB lane #396 — test(backend): integration marker + conftest gating for the real-DB lane (closed: already delivered by PR Local-Supabase follow-ups: rich seed + integration suite (#363, #362; #365 reverted) #377 + fix(backend): keyless rag_service import + hermetic LLM egress guard (#378, #379) #411)
• test(backend): integration fixtures — psycopg, truncate isolation, seeded users #397 — test(backend): integration fixtures — psycopg, truncate isolation, seeded users
• test(backend): subcutaneous write-path suite — encryption, PostgREST semantics, constraints, authz, migrations #398 — test(backend): subcutaneous write-path suite — encryption, PostgREST semantics, constraints, authz, migrations

---

Supersedes #362 and #363 (absorbed; PR #377 implements slices of #396/#397 and the rich seed).