Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -106,7 +106,7 @@ make explore # Chapter 2: bounded AI exploration of the ru

## Gotchas

- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), and `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Column-level encryption is on for sensitive columns: `user_profiles.name`/`first_name`/`last_name`/`bio`/`location` (these moved off `users` to `user_profiles` in the 0024 identity split), Google OAuth tokens, `messages.content`, `room_messages.text`, `sessions.summary_json`, `documents.summary` + `concept_notes` + `extracted_text` (the RAG OCR text added in 0030), `notes.title`/`body`/`last_summary`, `assignments.notes`/`points_possible`/`points_earned` (the enrollment-keyed gradebook table; points columns carry numeric semantics — use `decrypt_numeric` at read), `feedback.comment`/`topic` + `issue_reports.topic`/`description` (free-text user input, #520), `quiz_attempts.questions_json`/`answers_json` + `quiz_context.context_json` (quiz performance data, #521; scalar analytics columns — score/total/difficulty/completed_at — stay plaintext), and `flashcards.front`/`back` + `study_guides.content` + `room_summaries.summary` (derived content, #518; `study_guides.content` uses the JSON pair, `room_summaries.summary` keys its cache on the separate plaintext `member_hash` column so encryption doesn't affect cache-hit lookups). Helpers live in `backend/services/encryption.py`; use `encrypt_if_present` at write boundaries and `decrypt_if_present` / `decrypt_numeric` at read boundaries (including before injecting into AI prompts). `ENCRYPTION_KEY` must be set (32 bytes as 64 hex chars; generate via `python -c "import secrets; print(secrets.token_hex(32))"`). Deliberate exception: `newsletter_emails.email` stays plaintext (ADR 0026) — the UNIQUE constraint, lookup index, and both subscribe/allowlist upserts key on the value, and AES-GCM's per-call nonce breaks value equality.
- Knowledge-graph mastery is now an append-only `node_mastery_events` table (replaced the `graph_nodes.mastery_events` JSON column in 0023); node/edge dedup is enforced by UNIQUE constraints. Don't read/write a `mastery_events` column.
- Optional cross-worker cache (#97): `services/cache.py` wraps Redis and is **off by default** — with no `REDIS_URL` set it's a zero-overhead no-op and never fails a request (any Redis error → clean miss + warning). Currently backs the content-addressed OCR/extraction cache (`extraction_service.extract_text_from_file`, keyed on `sha256(file_bytes)` + engine). The `redis` dependency is only imported when `REDIS_URL` is set.
- HTTP caching (#99): conditional GETs use `services/http_cache.py` (`make_etag`/`conditional`/`cached_json`). `Cache-Control` on these routes is **always `private`, never `public`** — the responses carry user-scoped, app-decrypted columns that must never be cached at a shared proxy/CDN. Derive the ETag from cheap change-keys (ids, `updated_at`, existing content hashes), not from the fully-built payload.
15 changes: 15 additions & 0 deletions backend/db/backfill_encryption.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -302,6 +302,18 @@ def backfill_quiz_context(apply: bool) -> dict:
return _encrypt_json_column("quiz_context", "context_json", pk="id", apply=apply)


def backfill_flashcards(apply: bool) -> dict:
return _encrypt_text_column("flashcards", ["front", "back"], pk="id", apply=apply)


def backfill_study_guides(apply: bool) -> dict:
return _encrypt_json_column("study_guides", "content", pk="id", apply=apply)


def backfill_room_summaries(apply: bool) -> dict:
return _encrypt_text_column("room_summaries", ["summary"], pk="room_id", apply=apply)


RUNNERS: dict[str, Callable[[bool], dict]] = {
"users": backfill_users,
"user_settings": backfill_user_settings,
Expand All@@ -315,6 +327,9 @@ def backfill_quiz_context(apply: bool) -> dict:
"issue_reports": backfill_issue_reports,
"quiz_attempts": backfill_quiz_attempts,
"quiz_context": backfill_quiz_context,
"flashcards": backfill_flashcards,
"study_guides": backfill_study_guides,
"room_summaries": backfill_room_summaries,
}


Expand Down
26 changes: 22 additions & 4 deletions backend/db/seed_local_rich.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -553,7 +553,7 @@ def seed_notes_documents() -> None:
)


# (fc_id, user_id, offering_id, topic, front, back) — plaintext, grouped by topic.
# (fc_id, user_id, offering_id, topic, front, back) — 🔒 front/back, grouped by topic.
_FLASHCARDS = [
("rich-fc-cs-1", USER_ACTIVE, OFF_CS_F25, "CS Basics",
"What is a variable?", "A named storage location for a value."),
Expand All@@ -579,8 +579,9 @@ def seed_flashcards() -> None:
"user_id": user_id,
"offering_id": off_id,
"topic": topic,
"front": front,
"back": back,
# 🔒 front / back (#518)
"front": encrypt_if_present(front),
"back": encrypt_if_present(back),
},
)

Expand DownExpand Up@@ -625,11 +626,26 @@ def seed_study_guides() -> None:
"offering_id": off_id,
"exam_id": exam_id,
"generated_at": generated_at,
"content": content,
# 🔒 content (#518)
"content": encrypt_json(content),
},
)


def seed_room_summaries() -> None:
# #518: room_summaries.summary is 🔒. PK is room_id (no id column), so this
# can't go through insert_if_absent.
if not table("room_summaries").select("room_id", filters={"room_id": f"eq.{ROOM_STUDY}"}):
table("room_summaries").insert({
"room_id": ROOM_STUDY,
"summary": encrypt_if_present("The group is reviewing recursion before the midterm."),
"member_hash": "rich-member-hash-v1",
})
h.record("room_summaries", created=True)
else:
h.record("room_summaries", created=False)


# (qa_id, concept_node_id, difficulty, score, total, questions_json, answers_json, completed_at)
_QUIZ_ATTEMPTS = [
("rich-qa-cs-variables-1", "rich-node-cs-variables", "easy", 9, 10,
Expand DownExpand Up@@ -766,6 +782,7 @@ def seed_sessions() -> None:
"schools", "courses", "course_offerings", "users", "user_profiles", "user_roles",
"enrollments", "graph_nodes", "graph_edges", "node_mastery_events",
"gradebook_categories", "assignments", "rooms", "room_members", "room_messages",
"room_summaries",
"notes", "documents", "flashcards", "study_guides", "quiz_attempts", "quiz_context",
"sessions", "messages", "feedback", "issue_reports",
]
Expand All@@ -785,6 +802,7 @@ def main() -> None:
seed_notes_documents()
seed_flashcards()
seed_study_guides()
seed_room_summaries()
seed_quiz()
seed_feedback()
seed_sessions()
Expand Down
4 changes: 4 additions & 0 deletions backend/e2e_oracles/gather.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -195,6 +195,10 @@ def run_counts(args: argparse.Namespace) -> tuple[list[Finding], int]:
("quiz_attempts", "id", "questions_json"),
("quiz_attempts", "id", "answers_json"),
("quiz_context", "id", "context_json"),
("flashcards", "id", "front"),
("flashcards", "id", "back"),
("study_guides", "id", "content"),
("room_summaries", "room_id", "summary"),
)


Expand Down
23 changes: 17 additions & 6 deletions backend/routes/flashcards.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,7 +14,7 @@
from services.academics import resolve_offering, term_id_for_label
from services.auth_guard import require_self, get_session_user_id
from services.achievement_service import check_achievements
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import decrypt_if_present, decrypt_json, encrypt_if_present
from services.flashcard_import_service import (
dedup_against_existing,
check_rate_limit,
Expand DownExpand Up@@ -236,8 +236,8 @@ def generate(body: GenerateFlashcardsBody, request: Request):
"id": str(uuid.uuid4()),
"user_id": body.user_id,
"topic": body.topic,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand All@@ -261,8 +261,16 @@ def generate(body: GenerateFlashcardsBody, request: Request):
except Exception:
pass

# rows_to_insert holds the ciphertext just written; the response the
# frontend renders (freshly generated cards, before any list re-fetch)
# must carry plaintext front/back, not the encrypted insert payload.
response_cards = [
{**row, "front": decrypt_if_present(row["front"]), "back": decrypt_if_present(row["back"])}
for row in rows_to_insert
]

return {
"flashcards": rows_to_insert,
"flashcards": response_cards,
"context_used": {
"documents_found": len(documents),
"weak_concepts_found": len(weak_concepts),
Expand DownExpand Up@@ -291,6 +299,9 @@ def get_flashcards(
"id,user_id,topic,offering_id,front,back,times_reviewed,last_rating,last_reviewed_at,created_at",
filters=filters, order="created_at.desc"
) or []
for r in rows:
r["front"] = decrypt_if_present(r.get("front"))
r["back"] = decrypt_if_present(r.get("back"))
if semester:
# Term scoping (#141). This route is user-wide (no course id), so
# the filter works on the cards' offering: cards from the selected
Expand DownExpand Up@@ -397,8 +408,8 @@ def import_commit(body: ImportCommitBody, request: Request):
"user_id": body.user_id,
"topic": body.topic,
"offering_id": offering_id,
"front": c["front"],
"back": c["back"],
"front": encrypt_if_present(c["front"]),
"back": encrypt_if_present(c["back"]),
"times_reviewed": 0,
"last_reviewed_at": None,
"created_at": now,
Expand Down
23 changes: 19 additions & 4 deletions backend/routes/study_guide.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -4,6 +4,7 @@
Study guide generation and caching.
"""

import logging
import uuid
from datetime import datetime, timezone

Expand All@@ -24,10 +25,17 @@
)
from services.graph_service import get_courses as graph_get_courses
from services.auth_guard import require_self
from services.encryption import decrypt_if_present, decrypt_json
from services.encryption import (
decrypt_if_present,
decrypt_json,
decrypt_json_column,
encrypt_json,
)
from services.http_cache import cached_json, conditional, make_etag
from services.request_context import current_request_id

logger = logging.getLogger(__name__)

router = APIRouter()


Expand DownExpand Up@@ -198,7 +206,7 @@ def _generate_and_insert(user_id: str, offering_id: str, exam_id: str) -> dict:
"offering_id": offering_id,
"exam_id": exam_id,
"generated_at": now,
"content": content,
"content": encrypt_json(content),
}
table("study_guides").insert(row)

Expand DownExpand Up@@ -245,7 +253,14 @@ def get_cached_guides(user_id: str, request: Request):

result = []
for g in guides:
content = g.get("content") or {}
try:
content = decrypt_json_column(g.get("content")) or {}
except Exception:
logger.warning(
"get_cached_guides: content decrypt failed for guide %s; degrading",
g.get("id"),
)
content = {}
course_id = offering_to_course.get(g.get("offering_id"))
result.append({
"id": g["id"],
Expand DownExpand Up@@ -330,7 +345,7 @@ def get_guide(
)
if cached:
row = cached[0]
return {"guide": row["content"], "generated_at": row["generated_at"], "cached": True}
return {"guide": decrypt_json_column(row["content"]), "generated_at": row["generated_at"], "cached": True}

result = _generate_and_insert(user_id, offering_id, exam_id)
return {"guide": result["content"], "generated_at": result["generated_at"], "cached": False}
Expand Down
5 changes: 4 additions & 1 deletion backend/services/flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -33,6 +33,7 @@
from agents.flashcard import flashcard_agent
from agents.usage import record_agent_usage
from services import extraction_service
from services.encryption import decrypt_if_present

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -76,7 +77,9 @@ def dedup_against_existing(
filters["topic"] = f"eq.{topic}"

existing = table("flashcards").select("front", filters=filters) or []
existing_norm = [_normalize(r.get("front", "")) for r in existing]
existing_norm = [
_normalize(decrypt_if_present(r.get("front", "")) or "") for r in existing
]

keep: list[Card] = []
skipped: list[Card] = []
Expand Down
5 changes: 3 additions & 2 deletions backend/services/social_cache_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,6 +12,7 @@
from datetime import datetime, timezone

from db.connection import table
from services.encryption import decrypt_if_present, encrypt_if_present


def _compute_hash(member_summaries: list[str]) -> str:
Expand All@@ -26,15 +27,15 @@ def get_cached_summary(room_id: str, member_summaries: list[str]) -> str | None:
filters={"room_id": f"eq.{room_id}"},
)
if rows and rows[0]["member_hash"] == current_hash:
return rows[0]["summary"]
return decrypt_if_present(rows[0]["summary"])
return None


def save_summary(room_id: str, member_summaries: list[str], summary: str) -> None:
table("room_summaries").upsert(
{
"room_id": room_id,
"summary": summary,
"summary": encrypt_if_present(summary),
"member_hash": _compute_hash(member_summaries),
"updated_at": datetime.now(timezone.utc).isoformat(),
},
Expand Down
15 changes: 15 additions & 0 deletions backend/tests/integration/test_encryption_roundtrip.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,6 +46,13 @@
"Upload stuck"),
("issue_reports.description", "issue_reports", "id", "rich-issue-1",
"description", "Syllabus upload spins forever."),
("flashcards.front", "flashcards", "id", "rich-fc-cs-1", "front",
"What is a variable?"),
("flashcards.back", "flashcards", "id", "rich-fc-cs-1", "back",
"A named storage location for a value."),
("room_summaries.summary", "room_summaries", "room_id",
"rich-room-study-group", "summary",
"The group is reviewing recursion before the midterm."),
]

# (label, id_value, column, expected_number)
Expand DownExpand Up@@ -123,6 +130,14 @@ def test_quiz_context_context_json_is_ciphertext_and_decrypts(db_conn):
assert decoded["asked"] == 2


def test_study_guides_content_is_ciphertext_and_decrypts(db_conn):
"""#518: study_guides.content needs the JSON pair (encrypt_json/decrypt_json)."""
raw = _raw(db_conn, "study_guides", "id", "rich-guide-cs-f25-mid", "content")
assert isinstance(raw, str), "content stored as PLAINTEXT JSONB — encryption regressed"
decoded = decrypt_json(raw)
assert decoded["exam"] == "Midterm Exam"


def test_documents_concept_notes_uses_json_encryption(db_conn):
"""documents.concept_notes is stored via encrypt_json (a list of concepts)."""
raw = _raw(db_conn, "documents", "id", "rich-doc-cs-syllabus", "concept_notes")
Expand Down
22 changes: 22 additions & 0 deletions backend/tests/test_flashcard_import_service.py
Original file line numberDiff line numberDiff line change
Expand Up@@ -59,6 +59,28 @@ def test_filters_by_topic_when_offering_id_is_none(self):
assert "offering_id" not in str(call_kwargs)
assert "course_id" not in str(call_kwargs)

def test_dedup_matches_against_encrypted_existing_cards(self):
# #518: dedupe must be decrypt-aware — existing rows are ciphertext.
from services.encryption import encrypt
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [
{"front": encrypt("What is a variable?")}
]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new

def test_dedup_still_matches_legacy_plaintext_rows(self):
# Pre-backfill rows are still plaintext — decrypt_if_present falls
# back to the raw value, so dedupe keeps working unchanged.
with patch("services.flashcard_import_service.table") as t:
t.return_value.select.return_value = [{"front": "What is a variable?"}]
new = [{"front": "What is a variable?", "back": "A named storage location."}]
keep, skipped = svc.dedup_against_existing("u1", "c1", new)
assert keep == []
assert skipped == new


# ── check_rate_limit ─────────────────────────────────────────────────────────

Expand Down
Loading
Loading