diff --git a/.claude/skills/README.md b/.claude/skills/README.md index 7f3aa53..038a17d 100644 --- a/.claude/skills/README.md +++ b/.claude/skills/README.md @@ -114,7 +114,7 @@ index plus the relevant [Architecture](../../docs/architecture/) doc. | Skill | When to use | |-------|-------------| -| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Redis, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, MinIO, LiveKit, Meilisearch. | +| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Valkey, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, SeaweedFS, LiveKit, Meilisearch. | | [seed-tenant](seed-tenant/SKILL.md) | Provisioning a tenant (with its default organization, customization data, seed users) for local development. | ## Authoring a new skill diff --git a/.claude/skills/add-feature-key/SKILL.md b/.claude/skills/add-feature-key/SKILL.md index 5769f64..3da4af8 100644 --- a/.claude/skills/add-feature-key/SKILL.md +++ b/.claude/skills/add-feature-key/SKILL.md @@ -265,9 +265,11 @@ public async Task LimitKey_Soft_SurfaceBanner_DoesNotBlock() { ... } - **Writing a plan-projected key to `tenant_feature_flags`.** Architecture test rejects. Plan keys belong to the entitlement projection only. - **Reading the key from raw SQL.** Forbidden; use `IFeatureFlags`. -- **Hot path without Redis cache.** Each `IsEnabledAsync` call could become - DB-bound. The Redis layer (60s TTL, eager-invalidated by Dapr event) is - load-bearing. +- **Hot path without the cache stack.** Each `IsEnabledAsync` call could become + DB-bound. The two cache layers (L1 in-process `IMemoryCache` = 60s TTL; + L2 Dapr state → Valkey = 15-min upper bound; both eager-invalidated by the + `learnstack.hub.entitlement` Dapr event) are load-bearing — see Standards 20 + § Configuration / Eager invalidation. - **Killswitch without runbook.** The runbook is part of the deliverable. CI does not enforce its presence today; review must. - **Removing a key without a deprecation cycle.** A rename / remove follows the diff --git a/.claude/skills/add-integration-test/SKILL.md b/.claude/skills/add-integration-test/SKILL.md index f3c29f0..93177cd 100644 --- a/.claude/skills/add-integration-test/SKILL.md +++ b/.claude/skills/add-integration-test/SKILL.md @@ -3,7 +3,7 @@ name: add-integration-test description: > Write a Testcontainers-backed integration test in `backend/tests/LearnStack.Tests.Integration` that exercises a real Postgres + - Redis + (optionally) Dapr stack and asserts behaviour under tenant + organization + Valkey + (optionally) Dapr stack and asserts behaviour under tenant + organization context. USE FOR: cross-tenant / cross-org isolation tests (mandatory for every new `[TenantOwned]` / `[OrganizationScoped]` entity), outbox → consumer round trips, audit-pipeline assertions, RLS-effective-isolation tests. DO NOT USE FOR: @@ -31,7 +31,7 @@ architecture test) plus any other invariant the change touches. See - Outbox → consumer round-trip. - Audit-pipeline-writes-the-expected-row. - Provider-adapter contract test that talks to a containerised real provider - (LiveKit OSS, MinIO, Meilisearch). + (LiveKit OSS, SeaweedFS, Meilisearch). ## When not to use @@ -45,7 +45,7 @@ architecture test) plus any other invariant the change touches. See |-------|----------|-------------| | Scenario | Yes | A short name + setup + act + assert. | | Seed | Yes | Minimum tenants / orgs / users / customization data the scenario needs. | -| Required containers | Yes | Postgres always; add Redis / Kafka / Meilisearch / LiveKit / MinIO as needed. | +| Required containers | Yes | Postgres always; add Valkey / Kafka / Meilisearch / LiveKit / SeaweedFS as needed. | | Tenant context | Yes | Which tenant + org the act phase runs as. | ## Workflow @@ -54,7 +54,7 @@ architecture test) plus any other invariant the change touches. See The project ships `TestFixture` that: -- Spins Postgres + Redis (+ optional Kafka via Dapr) via Testcontainers. +- Spins Postgres + Valkey (+ optional Kafka via Dapr) via Testcontainers. - Applies all migrations (per module). - Seeds a baseline platform admin, two tenants, two orgs per tenant. - Exposes `fixture.AsTenant(tenantId, organizationId?)` to scope a block. diff --git a/.claude/skills/local-dev-setup/SKILL.md b/.claude/skills/local-dev-setup/SKILL.md index eab515a..31c69e2 100644 --- a/.claude/skills/local-dev-setup/SKILL.md +++ b/.claude/skills/local-dev-setup/SKILL.md @@ -1,8 +1,8 @@ --- name: local-dev-setup description: > - Bring up the LearnStack local stack — Postgres, Redis, Vault, Kafka, Dapr - sidecar, APISIX, Keycloak (two realms), MinIO, LiveKit OSS, Meilisearch — via + Bring up the LearnStack local stack — Postgres, Valkey, Vault, Kafka, Dapr + sidecar, APISIX, Keycloak (two realms), SeaweedFS, LiveKit OSS, Meilisearch — via `docker-compose` plus the project's `make dev` orchestrator. USE FOR: first-time workstation setup, restoring a broken local environment, switching between `DeploymentMode` for testing. DO NOT USE FOR: production deployment (separate @@ -15,7 +15,7 @@ description: > ## Purpose Stand up a full LearnStack stack on a developer workstation so backend + frontend -can run against real Postgres / Redis / Kafka / Vault / Keycloak / MinIO / +can run against real Postgres / Valkey / Kafka / Vault / Keycloak / SeaweedFS / LiveKit / Meilisearch / APISIX — the same components production uses ([12-infrastructure.md § Local Infrastructure](../../../docs/standards/12-infrastructure.md), [20-infrastructure-stack.md](../../../docs/standards/20-infrastructure-stack.md)). @@ -45,7 +45,7 @@ LiveKit / Meilisearch / APISIX — the same components production uses | Docker Desktop | Yes | Required for every container. | | .NET 10 SDK | Yes | `dotnet --version` returns `10.0.x`. | | Node 20+ + pnpm | Yes | For the frontend. | -| Deployment mode | Yes | `Development` (default) / `SaaS` / `Dedicated` / `SelfHosted`. | +| Deployment mode | Yes | `Development` (default) / `SaaS` / `Dedicated` / `SelfHostedOnline` / `SelfHostedAirGapped` (per [Standards 12 § Deployment Modes](../../../docs/standards/12-infrastructure.md)). | | `.env` (gitignored) | Optional | Local overrides; `.env.example` is the source of truth. | ## Workflow @@ -98,12 +98,12 @@ The components and their default ports: | Component | Port | Purpose | |-----------|------|---------| | Postgres | 5432 | Main DB. | -| Redis | 6379 | Cache + Dapr state. | +| Valkey | 6379 | Cache + Dapr state. | | Kafka | 9092 | Dapr pub/sub backend. | | Kafka UI | 9094 | Optional UI for topics. | | Vault (dev mode) | 8200 | Secrets backend; `root` token, **not** for production. | | Keycloak | 8080 | Two realms: `learnstack` (tenants) + `learnstack-hub` (operators). | -| MinIO | 9000 (API) / 9001 (console) | Object storage. | +| SeaweedFS | 9000 (S3 API) / 9001 (filer UI) / 9333 (master) | Object storage (single dev binary: master + volume + filer + S3 gateway). | | Meilisearch | 7700 | Search. | | LiveKit | 7880 (API) / 7881 (TLS) / 7882 (RTC) | Live classroom. | | coturn | 3478 / 5349 | TURN for LiveKit. | @@ -111,8 +111,7 @@ The components and their default ports: | Mailhog | 8025 | Captures outbound email in dev. | | OTel Collector | 4317 (gRPC) | Local observability. | | Dapr sidecar (per service) | 3500 (HTTP) / 50001 (gRPC) | Building-block runtime. | -| APISIX | 9080 (gateway) / 9180 (admin) | API gateway. | -| APISIX dashboard (optional) | 9000 | Route inspection. | +| APISIX | 9080 (HTTP gateway) / 9443 (HTTPS) / 9091 (Prometheus metrics) | File-driven standalone (`data_plane`) per ADR-0015 — no Admin API, no dashboard companion. `apisix.yaml` is the only source of truth. | | LearnStack API | 5100 | Backend host. | | LearnStack Web (`apps/web`) | 3000 | Frontend dev server. | @@ -128,7 +127,7 @@ The first `make dev` run additionally: 4. Seeds the two demo tenants' customization data (`TenantContentType`, `TenantPageBlock`, `TenantLevelTaxonomy`, …) so the page renderer has something to render. -5. Creates the MinIO buckets. +5. Creates the SeaweedFS buckets. 6. Creates the Meilisearch indexes. For a clean re-seed: @@ -150,8 +149,8 @@ curl -fsS http://localhost:9080/healthz | jq open http://localhost:8080/realms/learnstack/account open http://localhost:8080/realms/learnstack-hub/account -# MinIO console -open http://localhost:9001 # minioadmin / minioadmin +# SeaweedFS filer UI (replaces the MinIO console of the prior stack) +open http://localhost:9001 # S3 access: learnstack / learnstack-dev-secret # Web app open http://localhost:3000 # one of the demo tenants @@ -168,9 +167,10 @@ Edit `.env` to flip `DEPLOYMENT_MODE`: | Value | What happens | |-------|--------------| | `Development` (default) | `InProcessEventBus` + `InMemoryCacheService` + env vars for secrets. Dapr sidecar is still present but not exercised. | -| `SaaS` | `DaprEventBus` (Kafka) + `DaprCacheService` (Redis) + `DaprSecretProvider` (Vault) + `HubEntitlementProvider` pointing at the local Hub. Requires the `learnstack-hub` repo's `make dev` to be running. | +| `SaaS` | `DaprEventBus` (Kafka) + `DaprCacheService` (Valkey) + `DaprSecretProvider` (Vault) + `HubEntitlementProvider` pointing at the local Hub. Requires the `learnstack-hub` repo's `make dev` to be running. | | `Dedicated` | Same as `SaaS` for the composition; in practice the Hub is dedicated to one tenant. | -| `SelfHosted` | `SignedLicenseKeyEntitlementProvider` reads `.lic` from `./secrets/license.lic`; no Hub interaction. | +| `SelfHostedOnline` | `HubEntitlementProvider` against the LearnStack-hosted Hub (phone-home daily, 30-day cached-projection grace per ADR-0020). | +| `SelfHostedAirGapped` | `SignedLicenseKeyEntitlementProvider` reads `.lic` from `./secrets/license.lic`; no Hub interaction. | After changing `.env`, restart the API: diff --git a/.claude/skills/run-tests-locally/SKILL.md b/.claude/skills/run-tests-locally/SKILL.md index e6a5abb..244aa2e 100644 --- a/.claude/skills/run-tests-locally/SKILL.md +++ b/.claude/skills/run-tests-locally/SKILL.md @@ -60,7 +60,7 @@ docker info >/dev/null && echo "docker OK" backend/tests/ LearnStack.Tests.Unit/ # No DB, no Docker. Pure unit tests. LearnStack.Tests.Architecture/ # Reflection + Roslyn + migration-scan rules. - LearnStack.Tests.Integration/ # Testcontainers (Postgres, Redis, optional Dapr). + LearnStack.Tests.Integration/ # Testcontainers (Postgres, Valkey, optional Dapr). LearnStack.Tests.EndToEnd/ # Real HTTP API + frontend smoke. frontend/apps/web/ # Vitest + axe-core + Playwright (E2E). @@ -96,14 +96,14 @@ Common failure messages and fixes: | `Every_TenantOwned_Table_HasRls_With_AppTenantId` | Migration missing `ENABLE ROW LEVEL SECURITY` + the policy. | | `Integration_Event_Handlers_Use_InboxGuard` | Handler skipped `IsAlreadyProcessedAsync`; see [add-integration-event](../add-integration-event/SKILL.md). | | `Dapr_PubSub_TopicNames_FollowConvention` | Topic isn't `learnstack.{module}.{aggregate}`. | -| `Modules_Do_Not_Inject_Redis_Directly` | Use `ICacheService` not `IConnectionMultiplexer`. | +| `Modules_Do_Not_Inject_Valkey_Directly` | Use `ICacheService` not `IConnectionMultiplexer`. | | `LearnStack_Modules_DoNotReference_Hub` | Hub URL or namespace referenced outside the dedicated adapter. | | `No_Source_Folder_Named_Verticals` | A `Verticals/` folder exists; ADR-0018 forbids. | | `Core_Modules_HaveNo_DomainSpecific_Names` | A `Cefr`, `English`, `Asana`, etc. name appears in core. | ### Step 5: Run integration tests -Testcontainers spin Postgres + Redis per class fixture; runtime depends on Docker +Testcontainers spin Postgres + Valkey per class fixture; runtime depends on Docker performance. ```bash @@ -123,7 +123,7 @@ Common failure shapes: | `TenantContextMissingException` | Test forgot `using fixture.AsTenant(...)`. | | Empty result where rows should exist | Wrong tenant context — RLS works as designed. | | `relation "" does not exist` | Migration didn't apply; check the module's `Persistence/Migrations`. | -| Docker container fails to start | Port collision (5432, 6379) — stop local Postgres / Redis. | +| Docker container fails to start | Port collision (5432, 6379) — stop local Postgres / Valkey. | | Test hangs | A handler awaiting Dapr in the dev fallback path; check `IEventBus` registration. | ### Step 6: Run frontend tests @@ -226,7 +226,7 @@ dotnet test --blame-hang --blame-hang-timeout 5min change — the lockfile must match. - **Skipped architecture test.** Forbidden. If a test is marked `[Skip]`, treat it as a bug. -- **Port collisions.** Local Postgres / Redis on default ports collides with +- **Port collisions.** Local Postgres / Valkey on default ports collides with Testcontainers. Stop them. - **`--no-build` after a source change.** Drop the flag — the test would run against stale binaries. diff --git a/.claude/skills/standards-check/SKILL.md b/.claude/skills/standards-check/SKILL.md index 4ea8da7..1a76c18 100644 --- a/.claude/skills/standards-check/SKILL.md +++ b/.claude/skills/standards-check/SKILL.md @@ -92,7 +92,7 @@ Walk every item in [CLAUDE.md § Hard rules](../../../CLAUDE.md) and - [ ] **Tenant + organization isolation defense-in-depth from day one** — marker + filter + RLS + arch test wherever a tenant-owned entity lands. - [ ] **Self-hosted infrastructure preferred** (Keycloak with two realms, - LiveKit OSS, MinIO, Meilisearch, Kafka, Vault). + LiveKit OSS, SeaweedFS, Meilisearch, Kafka, Vault). - [ ] **No domain-specific code in any module.** Domain shape = tenant data per ADR-0018. - [ ] **Foundation building blocks are Day-1.** Dapr / APISIX / audit diff --git a/.gitignore b/.gitignore index b402073..8b4eb5c 100644 --- a/.gitignore +++ b/.gitignore @@ -37,9 +37,9 @@ logs/ # Local infrastructure data .data/ -minio-data/ +seaweedfs-data/ postgres-data/ -redis-data/ +valkey-data/ # Local-only analyses (not shared via git) docs/analysis/ diff --git a/CLAUDE.md b/CLAUDE.md index 876c25d..2020284 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,15 +21,17 @@ Self-Hosted — backed by the companion **`learnstack-hub`** repository ## What state this is in -**Phase 01 in progress.** The repository now has the .NET 10 solution -scaffold under `backend/` (core + 7 modules × 4 projects + 4 test -projects including the non-skippable `LearnStack.Tests.Architecture`), +**Phase 01 packets 1-6 shipped.** The repository now has the .NET 10 +solution scaffold under `backend/` (core + 7 modules × 4 projects + 4 +test projects including the non-skippable `LearnStack.Tests.Architecture`), the `pnpm` frontend monorepo under `frontend/` (`apps/web` Next.js App -Router + `packages/{config,ui,sdk}`), and the core local-dev compose -stack at `infra/compose/dev.yml` (Postgres / Redis / MinIO / Mailpit / -Meilisearch). The remaining Phase-01 packets — Keycloak, LiveKit, Kafka, -Vault, Dapr, APISIX, `make` targets, `.env.example`, CI, `make seed` — -land incrementally; see +Router + `packages/{config,ui,sdk}`), and the full local-dev compose +stack at `infra/compose/dev.yml` — PostgreSQL 18, Valkey, SeaweedFS, +Mailpit, Meilisearch, Keycloak (two realms), LiveKit OSS + Coturn, +Kafka + kafka-ui, Vault, Dapr sidecar + placement, APISIX in file- +driven standalone mode. The remaining Phase-01 packets (7-8) — `make` +orchestrator, `.env.example`, pre-commit hook, `e2e.yml`, GitHub Actions +CI, `make seed` — land incrementally; see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). Module-level code references in the docs (e.g. `LearnStack.Modules.Education.Application`, `ILiveClassProvider`, @@ -88,7 +90,7 @@ let the entry point pick it. - **Standards changes cite an ADR.** A new standard rule or a change to an existing one is paired with an ADR when the rule is non-trivial. - **Modular monolith with four cross-module mechanisms** ([ADR-0010](docs/decisions/0010-cross-module-communication.md)): application contract, intra-module domain event, integration event via outbox (dispatched through Dapr pub/sub per Amendment 1), read-model projection. No fifth. - **Tenant + organization isolation is defense-in-depth from day one** ([ADR-0003 Amendment 1](docs/decisions/0003-tenant-isolation-defense-in-depth.md), [ADR-0017](docs/decisions/0017-tenant-organization-hierarchy.md)): tenant + organization context + EF query filters + PostgreSQL RLS + architecture tests. -- **Self-hosted infrastructure preferred** for Keycloak (auth, with two realms — `learnstack` + `learnstack-hub`), LiveKit OSS (live classroom), MinIO (object storage), Meilisearch (search), Kafka (pub/sub backend), Vault (secrets). See ADRs 0004, 0005, 0014. +- **Self-hosted infrastructure preferred** for Keycloak (auth, with two realms — `learnstack` + `learnstack-hub`), LiveKit OSS (live classroom), SeaweedFS (object storage), Meilisearch (search), Kafka (pub/sub backend), Vault (secrets). See ADRs 0004, 0005, 0014. - **The core platform stays domain-generic.** Domain-specific shapes (CEFR levels, English placement-test scoring, kyu/dan ranks, yoga asana catalogs, …) live as **tenant customization data** ([ADR-0018](docs/decisions/0018-tenant-driven-customization-model.md)), never as code in any module. There is no `Verticals/` folder. ADR-0011 is superseded. - **Foundation building blocks are Day-1, not Phase-11.** Dapr (`IEventBus`/`ICacheService`/`ISecretProvider`), APISIX gateway, audit infrastructure, organization scope, entitlement projection socket, host-to-tenant resolver, and architecture tests all ship in Phase 02a — not as later hardening. - **Provider adapters everywhere.** Payments, auth, storage, search, live classroom, notifications, **event bus, cache, secrets, Hub HTTPS contract, entitlement source, host resolver** — all sit behind interfaces. No SaaS lock-in in `Domain` or `Application`. See [20-infrastructure-stack.md](docs/standards/20-infrastructure-stack.md). diff --git a/README.md b/README.md index d2f9e32..f444ff4 100644 --- a/README.md +++ b/README.md @@ -17,25 +17,29 @@ Dedicated control plane, plan editor, custom-domain admin, and license-key issua ## Status -Phase 01 in progress. The repository now holds the .NET 10 solution scaffold -(7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` +Phase 01 packets 1-6 shipped. The repository now holds the .NET 10 solution +scaffold (7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` architecture test), the `pnpm` frontend monorepo (`apps/web` + `packages/{config,ui,sdk}`), -and the core local-dev `docker-compose` stack (Postgres / Redis / MinIO / Mailpit / -Meilisearch). Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, and -CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). +and the full local-dev `docker-compose` stack — PostgreSQL 18, Valkey, SeaweedFS, +Mailpit, Meilisearch, Keycloak (two realms), LiveKit OSS + Coturn, Kafka + kafka-ui, +Vault, Dapr sidecar + placement, APISIX (file-driven standalone). The remaining +Phase-01 packets — `make` targets, `.env.example`, pre-commit, `e2e.yml`, GitHub +Actions CI, `make seed` — land in packets 7-8; see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). ## Direction At A Glance - **Backend:** .NET 10, ASP.NET Core, Entity Framework Core, MediatR. -- **Database:** PostgreSQL 16, with Row-Level Security from day one. Tenant + **Organization** +- **Database:** PostgreSQL 18, with Row-Level Security from day one. Tenant + **Organization** defense in depth ([ADR-0003 Amendment 1](docs/decisions/0003-tenant-isolation-defense-in-depth.md), [ADR-0017](docs/decisions/0017-tenant-organization-hierarchy.md)). -- **Cache / Pub-Sub / Secrets:** Redis 7, Kafka, HashiCorp Vault — all accessed via - **Dapr** building blocks (`IEventBus`, `ICacheService`, `ISecretProvider`) per +- **Cache / Pub-Sub / Secrets:** Valkey 8 (RESP-protocol Linux-Foundation BSD fork + per [ADR-0030](docs/decisions/0030-redis-compatible-store-valkey.md)), Kafka, + HashiCorp Vault — all accessed via **Dapr** building blocks (`IEventBus`, + `ICacheService`, `ISecretProvider`) per [ADR-0014](docs/decisions/0014-adopt-dapr.md). -- **API Gateway:** **APISIX** in standalone YAML-reload mode per +- **API Gateway:** **APISIX** in file-driven standalone (`data_plane`) mode per [ADR-0015](docs/decisions/0015-api-gateway-apisix.md). -- **Object storage:** MinIO locally, S3-compatible storage in production. +- **Object storage:** SeaweedFS locally, S3-compatible storage in production. - **Search:** Meilisearch initially. - **Frontend:** Next.js 15 (App Router), TypeScript, React. **One** application (`apps/web`) with route segments for public, studio, and portal — multi-app split @@ -57,7 +61,7 @@ CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository - **Live classroom:** In-app WebRTC; **self-hosted LiveKit OSS** is the default; LiveKit Cloud available behind the same `ILiveClassProvider` interface. A custom WebRTC SFU is explicitly out of scope. -- **Recording:** Supported via LiveKit Egress to S3/MinIO; tenant-configurable; +- **Recording:** Supported via LiveKit Egress to S3/SeaweedFS; tenant-configurable; consent-aware; off by default. - **Deployment:** Triple deployment model per [ADR-0020](docs/decisions/0020-triple-deployment-hybrid-license.md) — diff --git a/backend/Directory.Packages.props b/backend/Directory.Packages.props index ca29456..b4ef912 100644 --- a/backend/Directory.Packages.props +++ b/backend/Directory.Packages.props @@ -37,6 +37,12 @@ suite only runs the WebApplicationFactory smoke test. --> + diff --git a/backend/README.md b/backend/README.md index 58e52dc..c6c28a3 100644 --- a/backend/README.md +++ b/backend/README.md @@ -37,7 +37,7 @@ backend/ tests/ LearnStack.Tests.Unit/ - LearnStack.Tests.Integration/ # Testcontainers (Postgres/Redis) + LearnStack.Tests.Integration/ # Testcontainers (Postgres/Valkey) LearnStack.Tests.Architecture/ # NetArchTest + filesystem invariants LearnStack.Tests.Contract/ ``` diff --git a/docs/architecture/02-domain-model.md b/docs/architecture/02-domain-model.md index 1217491..0edb2ba 100644 --- a/docs/architecture/02-domain-model.md +++ b/docs/architecture/02-domain-model.md @@ -348,7 +348,7 @@ These entities cover the **runtime** of a Live Session: actual rooms, tokens, re | `LiveRoom` | Yes | Runtime room created via `ILiveClassProvider`. Lives for the duration of a `LiveSession`. | | `LiveRoomToken` | Inside LiveRoom | Short-lived join token; scoped to user + room + role. | | `LiveRoomProvider` | Reference | The provider implementation that owns this room. | -| `LiveRecording` | Yes | Recording metadata, consent state, retention. File lives in MinIO/S3. | +| `LiveRecording` | Yes | Recording metadata, consent state, retention. File lives in SeaweedFS/S3. | | `LiveSessionEvent` | Append-only | join / leave / screen-share / recording started / network drop. Feeds `ClassroomEvent` analytics. | See [In-App Live Classroom](07-in-app-live-classroom.md) for the provider abstraction. diff --git a/docs/architecture/03-module-boundaries.md b/docs/architecture/03-module-boundaries.md index 94b830c..4b05c37 100644 --- a/docs/architecture/03-module-boundaries.md +++ b/docs/architecture/03-module-boundaries.md @@ -178,7 +178,7 @@ Per-tenant block shapes resolve through Tenant Customization (`TenantPageBlock`). ### Media -Owns media assets, object storage metadata, file lifecycle, variants, asset access policies. Knows about MinIO/S3 via the storage provider adapter. +Owns media assets, object storage metadata, file lifecycle, variants, asset access policies. Knows about SeaweedFS/S3 via the storage provider adapter. ### Education Catalog Owns programs, courses, course versions, categories, levels, tags, instructor profiles, @@ -261,7 +261,7 @@ backend/ LearnStack.Api/ # ASP.NET host (single deployment unit) LearnStack.Application/ # composition root, MediatR pipeline LearnStack.Domain/ # shared kernel domain pieces - LearnStack.Infrastructure/ # EF, Redis, MinIO, OpenTelemetry, Dapr wiring + LearnStack.Infrastructure/ # EF, Valkey, SeaweedFS, OpenTelemetry, Dapr wiring LearnStack.Infrastructure.Audit/ # audit interceptor + state capture + MediatR behavior LearnStack.SharedKernel/ # ids, audit fields, errors, paging, IEventBus, # ICacheService, ISecretProvider, IEntitlementProvider diff --git a/docs/architecture/04-technical-architecture.md b/docs/architecture/04-technical-architecture.md index 9cab4fd..0c082fb 100644 --- a/docs/architecture/04-technical-architecture.md +++ b/docs/architecture/04-technical-architecture.md @@ -7,12 +7,12 @@ | Backend runtime | .NET 10, ASP.NET Core Web API | | Language | C# | | ORM | Entity Framework Core | -| Database | PostgreSQL 16+ (shared schema + RLS isolation; ADR-0003) | -| Cache & coordination | **Redis 7+ via Dapr State Store** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | +| Database | PostgreSQL 18.x (major pinned per [ADR-0031](../decisions/0031-postgresql-major-version.md); shared schema + RLS isolation; ADR-0003) | +| Cache & coordination | **Valkey 8.x via Dapr State Store** (Linux-Foundation BSD-3 fork of Redis 7.2.4 per [ADR-0030](../decisions/0030-redis-compatible-store-valkey.md); [29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | | Pub/Sub | **Apache Kafka via Dapr Pub/Sub** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) — outbox dispatch target | | Secrets | **HashiCorp Vault via Dapr Secret Store** (or env-var fallback in Dev) | | Distributed runtime | **Dapr 1.14+** sidecar pattern (pub/sub, state, secrets) | -| Object storage | MinIO (local), S3-compatible (production) | +| Object storage | SeaweedFS (local), S3-compatible (production) | | Background jobs | Hangfire (Postgres storage) | | Search | Meilisearch (initial), OpenSearch (later, if needed). See [ADR 0012](../decisions/0012-search-strategy.md) | | Auth | Keycloak (self-hosted OIDC) — two realms: `learnstack` (tenant users) + `learnstack-hub` (operators); ADR-0004 Amendment 1 | @@ -52,8 +52,8 @@ flowchart LR subgraph data["Data Plane"] pg[(PostgreSQL)] - redis[(Redis)] - minio[(MinIO / S3)] + redis[(Valkey)] + seaweedfs[(SeaweedFS / S3)] meili[(Meilisearch)] kafka[(Kafka)] vault[(HashiCorp Vault)] @@ -87,13 +87,13 @@ flowchart LR daprd2 --> vault api --> pg - api --> minio + api --> seaweedfs api --> meili api --> kc_main api --> lk workers --> pg - workers --> minio - egress --> minio + workers --> seaweedfs + egress --> seaweedfs lk --> egress hubapi -- "mTLS + signed JWT + HMAC
POST /api/internal/*" --> api @@ -123,7 +123,7 @@ Module boundaries and dependency rules are in [Module Boundaries](03-module-boun | `Api` | HTTP endpoints, auth middleware, request binding, OpenAPI emission, tenant resolution middleware. | | `Application` | Use cases (MediatR commands/queries), validation, transactions, pipeline behaviors. | | `Domain` | Entities, aggregates, value objects, domain services, domain events. | -| `Infrastructure` | EF Core, Redis, MinIO, Hangfire, OpenTelemetry, external adapters. | +| `Infrastructure` | EF Core, Valkey, SeaweedFS, Hangfire, OpenTelemetry, external adapters. | | `Modules.*` | Bounded feature areas, each with their own `Application` / `Domain` / `Infrastructure` internals and a public `Application.Contracts` surface. | ## Multi-Tenancy @@ -207,8 +207,8 @@ Detailed conventions: [Frontend Architecture](14-frontend-architecture.md) and [ ``` postgres -redis -minio + minio-console +valkey +seaweedfs # single dev binary: master + volume + filer + S3 gateway meilisearch keycloak livekit-server diff --git a/docs/architecture/05-mvp-scope.md b/docs/architecture/05-mvp-scope.md index 668e104..7757e2c 100644 --- a/docs/architecture/05-mvp-scope.md +++ b/docs/architecture/05-mvp-scope.md @@ -52,7 +52,7 @@ test pass. ### Platform Kernel - Dapr building blocks wired through `IEventBus` (pub/sub → Kafka), `ICacheService` - (state → Redis), `ISecretProvider` (secrets → Vault). See + (state → Valkey), `ISecretProvider` (secrets → Vault). See [29-dapr-integration.md](29-dapr-integration.md) and [ADR-0014](../decisions/0014-adopt-dapr.md). - APISIX gateway in standalone mode (YAML hot-reload) — JWT verification, CORS, @@ -75,7 +75,7 @@ test pass. per-organization override. - Map custom domains via Hub's `CustomDomain` aggregate; LearnStack mirrors the host→tenant mapping. See [27-custom-domain-tls.md](27-custom-domain-tls.md). -- Per-tenant feature flags (catalog-defined; runtime via Redis cache + Postgres). See +- Per-tenant feature flags (catalog-defined; runtime via Valkey cache + Postgres). See [21-feature-flags.md](21-feature-flags.md). - Per-tenant settings (locale, timezone, default notification sender). @@ -121,7 +121,7 @@ customization aggregates are first-class in MVP: - Draft / published workflow with preview tokens. ### Media -- Upload to MinIO via signed URLs. +- Upload to SeaweedFS via signed URLs. - Asset metadata, folders, tags. - Image variants on upload. - Use assets inside content and page blocks. diff --git a/docs/architecture/06-extension-model.md b/docs/architecture/06-extension-model.md index 8e01ed9..f5583a0 100644 --- a/docs/architecture/06-extension-model.md +++ b/docs/architecture/06-extension-model.md @@ -35,7 +35,7 @@ The core platform talks to external systems through **interfaces** in | Recording egress | `IRecordingEgressProvider` | | Identity provider | (covered by Keycloak baseline; ADR-0004) | | Pub/Sub | `IEventBus` → Dapr → Kafka (ADR-0014) | -| Cache | `ICacheService` → Dapr → Redis (ADR-0014) | +| Cache | `ICacheService` → Dapr → Valkey (ADR-0014) | | Secret store | `ISecretProvider` → Dapr → Vault (ADR-0014) | Implementations live in `LearnStack.Infrastructure..` projects. diff --git a/docs/architecture/07-in-app-live-classroom.md b/docs/architecture/07-in-app-live-classroom.md index d9a5a75..26bbef6 100644 --- a/docs/architecture/07-in-app-live-classroom.md +++ b/docs/architecture/07-in-app-live-classroom.md @@ -64,8 +64,8 @@ flowchart LR end subgraph Infra - REDIS[(Redis)] - S3[(S3 / MinIO)] + REDIS[(Valkey)] + S3[(S3 / SeaweedFS)] TURN[Coturn TURN/STUN] end @@ -87,7 +87,7 @@ The flow: 2. LearnStack verifies enrollment / session permission and issues a **scoped, short-lived LiveKit join token**. 3. The client connects to LiveKit OSS via WebRTC (`wss://livekit.`), using TURN relay only when direct connectivity is blocked. 4. LiveKit emits webhooks (`participant_joined`, `track_published`, `room_finished`, etc.); LearnStack records them as `LiveSessionEvent` rows. -5. Optional recording: LearnStack triggers a LiveKit Egress job; the Egress worker writes the composite recording to S3/MinIO, and LearnStack stores `LiveRecording` metadata. +5. Optional recording: LearnStack triggers a LiveKit Egress job; the Egress worker writes the composite recording to S3/SeaweedFS, and LearnStack stores `LiveRecording` metadata. ## Provider-Agnostic Core @@ -180,7 +180,7 @@ Defaults: - Recording is **off by default** at the tenant level. - A tenant administrator can enable it for the entire tenant, for a course, or for a specific session. - Composite (single-file) recording is the default mode. Track-based recording is available for advanced post-processing scenarios. -- Recordings are written to S3/MinIO; retention defaults to **30 days** and is configurable per tenant up to a tenant-wide retention cap. See [16-media-pipeline.md](16-media-pipeline.md) § Recordings for the storage pipeline and [23-data-protection.md](23-data-protection.md) § Right to Erasure for deletion under KVKK / GDPR. +- Recordings are written to S3/SeaweedFS; retention defaults to **30 days** and is configurable per tenant up to a tenant-wide retention cap. See [16-media-pipeline.md](16-media-pipeline.md) § Recordings for the storage pipeline and [23-data-protection.md](23-data-protection.md) § Right to Erasure for deletion under KVKK / GDPR. - Recordings require consent: the classroom UI shows a "Recording" indicator while active, and the tenant onboarding agreement covers consent. ## MVP Scope for the Classroom @@ -212,10 +212,10 @@ Deferred to later phases: A working self-hosted deployment needs: 1. **LiveKit server** — Docker container, behind a TLS reverse proxy on `wss://livekit.`. -2. **Redis** — for multi-node coordination (single-node deployments can skip but multi-node needs it). +2. **Valkey** — for multi-node coordination (single-node deployments can skip but multi-node needs it). 3. **TURN server** — Coturn behind UDP/TCP ports, plus TLS for TURNS. Required for users behind restrictive NATs and corporate networks. 4. **Egress workers** — separate containers, started on demand by LiveKit when a recording is requested. -5. **Object storage** — S3 or MinIO with a `recordings/` bucket. +5. **Object storage** — S3 or SeaweedFS with a `recordings/` bucket. 6. **Monitoring** — LiveKit exports Prometheus metrics; Grafana dashboards are available upstream. 7. **Webhooks** — `https://api./webhooks/livekit` with HMAC signature verification. diff --git a/docs/architecture/08-livekit-cost-model.md b/docs/architecture/08-livekit-cost-model.md index 46de38e..368ff84 100644 --- a/docs/architecture/08-livekit-cost-model.md +++ b/docs/architecture/08-livekit-cost-model.md @@ -34,7 +34,7 @@ LearnStack owns: - Compute for the SFU and Egress workers. - Outbound bandwidth (often the largest single line item). - TURN/STUN infrastructure (Coturn). -- Redis for multi-node coordination. +- Valkey for multi-node coordination. - Object storage for recordings. - Monitoring, alerting, scaling, regional deployment, incident response. @@ -65,7 +65,7 @@ This would mean building a custom SFU on top of an open-source WebRTC library (M |---|---|---|---|---| | Base / minutes | $50 + $165 overage | $0 | $0 | $0 | | Bandwidth | $381 | ~$0 (in plan) | ~$310 | ~$0 (in plan) | -| Compute (SFU + Redis + TURN) | included | ~$150 | ~$200 | ~$150 | +| Compute (SFU + Valkey + TURN) | included | ~$150 | ~$200 | ~$150 | | Storage (recordings) | included up to 600 min | $5 | $5 | $5 | | Engineering (amortised) | ~$0 | ~$300/mo SRE slice | ~$300/mo SRE slice | **~$3,000–5,000/mo for 12 months, then ongoing** | | **Monthly steady-state** | **~$596** | **~$155** | **~$515** | **~$3,155–5,155** | @@ -136,7 +136,7 @@ A single 4-core SFU node comfortably handles ~250 concurrent participants for ty | Mode | Estimated monthly cost | |---|---| | LiveKit Cloud Ship | $50 (well within included quota) | -| LiveKit OSS Hetzner | ~$130 (1 small SFU + Redis + TURN) | +| LiveKit OSS Hetzner | ~$130 (1 small SFU + Valkey + TURN) | | LiveKit OSS AWS | ~$140 | At this scale, **LiveKit Cloud is cheaper** because the fixed infra cost of self- @@ -144,7 +144,7 @@ hosting is not yet amortised. > **Derivation (2026-05-14 rates).** Cloud: 12,000 minutes / 150,000 included = 8% of > quota, 84 GB / 250 GB = 34% — both inside Ship plan; $50 base only. -> Hetzner self-host: CX21 SFU (~$5) + CX11 Redis (~$3) + CX11 TURN (~$3) ≈ $11 +> Hetzner self-host: CX21 SFU (~$5) + CX11 Valkey (~$3) + CX11 TURN (~$3) ≈ $11 > infra; remaining ~$120 is the SRE-slice amortised cost (~10% of a senior engineer's > on-call rotation at this small workload). > AWS: same compute (~$30 t3.medium × 3) + ~$0 egress (84 GB inside 100 GB pooled @@ -222,7 +222,7 @@ flips the decision. | Mode | Estimated monthly cost | |---|---| | LiveKit Cloud Scale | $500 base + ~$120 transfer overage + $4,380 recording overage ≈ **$5,000** | -| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Redis + TURN + 1 TB recording storage ≈ **$650–800** | +| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Valkey + TURN ≈ **$650–800** compute. Recording storage at 300,000 min × 60 MB/min ≈ **18 TB** → ~€70/mo on Hetzner BX21-tier (€3.81/TB-month linear above 1 TB). **Total ≈ $720–870** | By this scale, the self-hosted advantage is ~6×. @@ -242,7 +242,7 @@ This is the basis for the firm decision in [ADR 0005](../decisions/0005-live-cla - Each tenant can enable recording, with retention configurable up to a tenant-level cap. - Track classroom cost metrics (participant minutes, bandwidth, recording minutes, egress CPU) from the first production deployment. See [Phase 09](../roadmap/phase-09-billing-integrations-analytics.md). - Default retention: 30 days. Long-term retention requires explicit tenant action. -- Recording files live in S3/MinIO; metadata lives in PostgreSQL. Deletion is a two-step process. +- Recording files live in S3/SeaweedFS; metadata lives in PostgreSQL. Deletion is a two-step process. - Bandwidth provider choice for the SFU node is a real architectural decision. Default to a bandwidth-friendly provider; only run the SFU on AWS/GCP when there is a regulatory or proximity reason. ## Recommendation diff --git a/docs/architecture/09-tenant-isolation.md b/docs/architecture/09-tenant-isolation.md index dc987db..35b003c 100644 --- a/docs/architecture/09-tenant-isolation.md +++ b/docs/architecture/09-tenant-isolation.md @@ -30,7 +30,7 @@ two scopes (tenant + organization): | PostgreSQL | RLS policy `tenant_id = current_setting('app.tenant_id', true)::uuid` | RLS policy `organization_id IS NULL OR organization_id = current_setting('app.organization_id', true)::uuid` | | Identity | Single-realm `learnstack` with `tenant_id` JWT claim (default per [ADR-0004](../decisions/0004-authentication-strategy.md); realm-per-tenant is an opt-in for enterprise isolation only) | `organization_id` JWT claim populated from active org membership | | Cache | Cache key auto-prefixed `{tenant_id}:{key}` | `{tenant_id}:{org_id}:{key}` when org context set | -| Files (MinIO) | Object key prefix `tenants/{tenant_id}/...` | `tenants/{tenant_id}/organizations/{org_id}/...` for org-scoped assets | +| Files (SeaweedFS) | Object key prefix `tenants/{tenant_id}/...` | `tenants/{tenant_id}/organizations/{org_id}/...` for org-scoped assets | | Search (Meilisearch) | `tenant_id` as mandatory filter | `organization_id = X OR organization_id IS NULL` clause when org context | | Jobs (Hangfire) | `JobParams.TenantId` mandatory | `JobParams.OrganizationId` nullable | | Audit (ADR-0016) | `audit_log.tenant_id` mandatory | `audit_log.organization_id` nullable | @@ -187,20 +187,20 @@ public abstract class PlatformJob : LearnStackJob | `Hangfire_JobPayloads_IncludeTenantId` | Reflection: every `LearnStackJob` subclass's `TParams` has `TenantId`. | | `LearnStackJob_RunAsync_SetsTenantBeforeExecute` | Source-grep + reflection: `RunAsync` is non-virtual; `SetTenant(...)` precedes `ExecuteAsync(...)`. | | `No_DirectDaprClient_OutsideInfrastructure` | Roslyn source scan: `Dapr.Client.*` only in `LearnStack.Infrastructure.{Caching, Messaging, Secrets}`. | -| `Provider_SDK_Types_NotImported_InDomain` | Provider SDK types (Stripe, Iyzico, LiveKit, Keycloak admin, MinIO) only in `LearnStack.Infrastructure.*` adapters. | +| `Provider_SDK_Types_NotImported_InDomain` | Provider SDK types (Stripe, Iyzico, LiveKit, Keycloak admin, SeaweedFS) only in `LearnStack.Infrastructure.*` adapters. | ## Storage, cache, search, audit, logs Tenant + org isolation applies outside PostgreSQL too: -### Storage (MinIO) +### Storage (SeaweedFS) -``` +```text tenants/{tenant_id}/organizations/{org_id}/courses/{course_id}/... ← org-scoped tenants/{tenant_id}/brand/... ← tenant-wide ``` -### Cache (Dapr State Store / Redis) +### Cache (Dapr State Store / Valkey) ``` {tenant_id}:{org_id}:{module}:{entity}:{id} ← org context set diff --git a/docs/architecture/16-media-pipeline.md b/docs/architecture/16-media-pipeline.md index 6be523a..deab72b 100644 --- a/docs/architecture/16-media-pipeline.md +++ b/docs/architecture/16-media-pipeline.md @@ -18,14 +18,14 @@ Both share the same object storage and tenant-scoped key layout described below. ## Object Storage -- **Local development**: MinIO running in Docker Compose, S3-compatible API. -- **Production**: S3-compatible storage. Specific provider chosen per deployment (AWS S3, Backblaze B2, Wasabi, Cloudflare R2, MinIO on owned infrastructure). The provider choice is operational, not architectural — the application sees the S3 API. +- **Local development**: SeaweedFS running in Docker Compose, S3-compatible API. +- **Production**: S3-compatible storage. Specific provider chosen per deployment (AWS S3, Backblaze B2, Wasabi, Cloudflare R2, SeaweedFS on owned infrastructure). The provider choice is operational, not architectural — the application sees the S3 API. Buckets are per-environment, not per-tenant. Tenant isolation in storage is enforced by the key prefix and signed-URL scoping; bucket-per-tenant explodes operational complexity and is not used. ## Key Layout -Per [09-tenant-isolation.md § Storage (MinIO)](09-tenant-isolation.md) the canonical +Per [09-tenant-isolation.md § Storage (SeaweedFS)](09-tenant-isolation.md) the canonical key prefix is `tenants/{tenant_id}/...` (with `organizations/{org_id}/...` segment for org-scoped assets). The full key layout used by the media module: @@ -59,7 +59,7 @@ Direct-to-S3 with a server-issued, scoped, time-limited PUT URL. The .NET API ne sequenceDiagram participant UI as Studio participant API as .NET API - participant S3 as S3 / MinIO + participant S3 as S3 / SeaweedFS participant W as Worker UI->>API: POST /v1/media/upload-intent (filename, mime, size) @@ -152,7 +152,7 @@ CDN-cached URLs (the public mode) are versioned by content hash so cache invalid A CDN sits in front of the storage backend for public and tenant-scoped reads. The CDN: - Honours `Cache-Control` set by the API. -- Honours signed URLs (CloudFront-style or signed-cookie equivalent for private content; for fully self-hosted MinIO the application proxies private reads via a short-lived URL with no CDN caching). +- Honours signed URLs (CloudFront-style or signed-cookie equivalent for private content; for fully self-hosted SeaweedFS the application proxies private reads via a short-lived URL with no CDN caching). - Provides per-tenant access metrics (bytes served, request count) that feed into the analytics pipeline. ## Recordings (Live Classroom) @@ -188,7 +188,7 @@ sequenceDiagram participant Portal participant API participant Provider as Live Provider - participant Storage as MinIO/S3 + participant Storage as SeaweedFS/S3 Learner->>Portal: Join recorded session Portal->>API: Request join token @@ -267,4 +267,4 @@ A managed adapter takes over transcoding, manifest generation, and CDN; the Lear - **Re-encoding profile changes** invalidate previous variants. Source retention buys re-encode capability; the schedule for retention extension is a per-tenant configuration. - **MIME-type lying** — clients can claim a different MIME than the file body. Always re-detect after upload using a magic-byte check before processing. - **Public-bucket misconfiguration** — buckets are private by default; "public" assets are served by ACL on the object, not by making the bucket public. A `public-read` bucket is an operational red flag. -- **Storage egress in self-hosted MinIO** — when MinIO runs on owned infrastructure, the bottleneck shifts to the colo's bandwidth. Plan capacity accordingly. +- **Storage egress in self-hosted SeaweedFS** — when SeaweedFS runs on owned infrastructure, the bottleneck shifts to the colo's bandwidth. Plan capacity accordingly. diff --git a/docs/architecture/18-webrtc-build-vs-adopt.md b/docs/architecture/18-webrtc-build-vs-adopt.md index b432752..75e4021 100644 --- a/docs/architecture/18-webrtc-build-vs-adopt.md +++ b/docs/architecture/18-webrtc-build-vs-adopt.md @@ -39,7 +39,7 @@ flowchart TD Learner --> SFU["SFU Media Server"] Instructor --> SFU SFU --> Recording["Recording / Egress"] - Recording --> Storage["MinIO / S3"] + Recording --> Storage["SeaweedFS / S3"] Backend["LearnStack Backend"] --> Signaling Backend --> Storage ``` diff --git a/docs/architecture/21-feature-flags.md b/docs/architecture/21-feature-flags.md index 13510f5..8392919 100644 --- a/docs/architecture/21-feature-flags.md +++ b/docs/architecture/21-feature-flags.md @@ -139,7 +139,7 @@ Rules: the inbound Dapr event from Hub. See [ADR-0021](../decisions/0021-feature-based-entitlement.md) and [29-dapr-integration.md](29-dapr-integration.md). -- A short-TTL Redis cache (60 s) fronts both tables for hot-path reads. Eager +- A short-TTL Valkey cache (60 s) fronts both tables for hot-path reads. Eager invalidation flows from `learnstack.cache.invalidation` (intra-instance) and from `learnstack.hub.entitlement` (cross-deployment). @@ -160,11 +160,11 @@ Resolution precedence for `IsEnabledAsync(FeatureKey key, ct)`: genuinely need to read cross-tenant go through a separate `IEntitlementAdminQuery` interface. 2. **If the key's catalog descriptor says `Source = PlanProjected`:** read from - `platform_entitlement_cache.features` (via Redis cache → Postgres). A missing entry + `platform_entitlement_cache.features` (via Valkey cache → Postgres). A missing entry resolves to the catalog default. Per-tenant `tenant_feature_flags` are **never** consulted for plan-projected keys. 3. **If the key's catalog descriptor says `Source = TenantFlag`:** read from - `tenant_feature_flags` (via Redis cache → Postgres). Missing entry → catalog + `tenant_feature_flags` (via Valkey cache → Postgres). Missing entry → catalog default. 4. **Killswitch overlay** (last word): if the corresponding killswitch is flipped `false` platform-wide, the answer becomes `false` regardless of the per-tenant @@ -264,13 +264,13 @@ Both surfaces are MUST-audit security-events (see hasn't refreshed sees the old feature set. Eager invalidation via the Dapr event keeps the typical refresh within seconds; the 15-min TTL is the upper bound. - **Performance.** Hot paths that read flags per call become DB-bound without the - Redis cache; the 60s TTL is the default trade-off. + Valkey cache; the 60s TTL is the default trade-off. ## Roadmap Touchpoints - **Phase 02a** — `tenant_feature_flags` table created in the Tenancy module; the `FeatureKeys` / `LimitKeys` / `KillswitchKeys` catalogs land here. `IFeatureFlags`, - the Redis cache, and the architecture tests ship here. + the Valkey cache, and the architecture tests ship here. - **Phase 02c** (parallel Hub Foundation) — `platform_entitlement_cache`, `IEntitlementProvider` with `NullEntitlementProvider` default + `HubEntitlementProvider` + `SignedLicenseKeyEntitlementProvider` diff --git a/docs/architecture/23-data-protection.md b/docs/architecture/23-data-protection.md index d46a729..e9036d3 100644 --- a/docs/architecture/23-data-protection.md +++ b/docs/architecture/23-data-protection.md @@ -128,8 +128,8 @@ a single LearnStack deployment. In practice: Within a regional instance, the following components must run in-region: - PostgreSQL primary + WAL archive. -- Redis (entitlement cache, L1 invalidation). -- MinIO / S3-compatible object storage (recordings, media). +- Valkey (entitlement cache, L1 invalidation). +- SeaweedFS / S3-compatible object storage (recordings, media). - Meilisearch. - Kafka. - LiveKit SFU + Egress (a learner joining from another continent still has their @@ -167,7 +167,7 @@ These are tracked as Phase-11+ work and are not implementation blockers for the For tenants subject to KVKK / GDPR, LearnStack acts as a **data processor** while the tenant is the **data controller**. The processor agreement template lives outside this repository (legal). Engineering-side commitments: -- Sub-processor list maintained in the tenant onboarding pack (Keycloak host, LiveKit host, S3 / MinIO provider, email provider, SMS provider, payment provider). +- Sub-processor list maintained in the tenant onboarding pack (Keycloak host, LiveKit host, S3 / SeaweedFS provider, email provider, SMS provider, payment provider). - Sub-processor changes are notified to tenants 30 days in advance via the Tenancy module's notification channel. - Security incident notification timeline: within 72 hours of discovery (GDPR Article 33 baseline). @@ -175,7 +175,7 @@ For tenants subject to KVKK / GDPR, LearnStack acts as a **data processor** whil - **PII redaction in logs.** Configured at the logging pipeline; the `[PiiSensitive]` attribute on a property excludes it from log emission ([10-observability.md](../standards/10-observability.md)). - **PII redaction in error reports.** Sentry receives redacted payloads; integration tests assert the redaction layer cannot be bypassed. -- **Encryption at rest.** Storage provider (S3 / MinIO) + PostgreSQL volumes use the provider's at-rest encryption; specific configuration documented in [12-infrastructure.md](../standards/12-infrastructure.md). +- **Encryption at rest.** Storage provider (S3 / SeaweedFS) + PostgreSQL volumes use the provider's at-rest encryption; specific configuration documented in [12-infrastructure.md](../standards/12-infrastructure.md). - **TLS in transit.** [11-security.md](../standards/11-security.md) § Transport. - **Backups.** Per-tenant deletion must propagate to backups within the platform's RPO + 1 backup cycle; the `delete-from-backups` job runs nightly. - **Subject-access timeline.** GDPR Article 12 requires response within 30 days. Engineering target: export bundle ready within 7 days of request; deletion completed within 30 days. diff --git a/docs/architecture/24-learnstack-hub.md b/docs/architecture/24-learnstack-hub.md index 0cd8bf5..16f90aa 100644 --- a/docs/architecture/24-learnstack-hub.md +++ b/docs/architecture/24-learnstack-hub.md @@ -523,7 +523,7 @@ learnstack-hub/ ``` Hub does NOT share Dapr instance with LearnStack — Hub has its own sidecar, its own -`pubsub` component, its own state store (Hub Redis instance), its own secret store +`pubsub` component, its own state store (Hub Valkey instance), its own secret store namespace (`secret/learnstack-hub/*`). **Kafka cluster topology:** Hub and LearnStack share the **same Kafka cluster** in diff --git a/docs/architecture/25-deployment-models.md b/docs/architecture/25-deployment-models.md index c051b73..7a9c7c8 100644 --- a/docs/architecture/25-deployment-models.md +++ b/docs/architecture/25-deployment-models.md @@ -65,8 +65,8 @@ Internet [ LearnStack.Host pods (auto-scaled 2-8) ] │ ├──► PostgreSQL (managed, RLS policies enforced) - ├──► Redis (managed, multi-tenant via key prefix) - ├──► MinIO (multi-tenant via key prefix) + ├──► Valkey (managed, multi-tenant via key prefix) + ├──► SeaweedFS (multi-tenant via key prefix) ├──► Meilisearch (per-tenant filter + per-locale index) ├──► Keycloak (single realm with multi-tenant claims) ├──► LiveKit (shared SFU pool) @@ -94,8 +94,8 @@ Internet [ LearnStack-managed Kubernetes namespace per customer ] │ ├──► Dedicated PostgreSQL instance (no shared rows; still RLS-protected for org scope) - ├──► Dedicated Redis instance - ├──► Dedicated MinIO instance + ├──► Dedicated Valkey instance + ├──► Dedicated SeaweedFS instance ├──► Dedicated Meilisearch ├──► Dedicated Keycloak realm (could be in shared Keycloak cluster with realm-per-tenant) ├──► Shared LiveKit pool (with per-tenant resource caps) OR dedicated SFU @@ -117,8 +117,8 @@ Internet [ Customer-owned Kubernetes cluster, customer admin ] │ ├──► PostgreSQL (customer-managed) - ├──► Redis (customer-managed) - ├──► MinIO or S3-compatible (customer-managed) + ├──► Valkey (customer-managed) + ├──► SeaweedFS or S3-compatible (customer-managed) ├──► Meilisearch (customer-managed) ├──► Keycloak (customer-managed; LearnStack ships pre-configured realm export) ├──► LiveKit (customer-managed OR fall back to LiveKit Cloud) diff --git a/docs/architecture/29-dapr-integration.md b/docs/architecture/29-dapr-integration.md index 48673fa..8de0750 100644 --- a/docs/architecture/29-dapr-integration.md +++ b/docs/architecture/29-dapr-integration.md @@ -5,7 +5,7 @@ [ADR-0010](../decisions/0010-cross-module-communication.md). LearnStack uses **Dapr** (Distributed Application Runtime) for three building blocks: -**pub/sub** (Kafka), **state store** (Redis), **secret store** (Vault). Application code +**pub/sub** (Kafka), **state store** (Valkey), **secret store** (Vault). Application code interacts with Dapr only through SharedKernel abstractions (`IEventBus`, `ICacheService`, `ISecretProvider`) — never via `DaprClient` directly. @@ -22,12 +22,12 @@ flowchart LR subgraph Backends[Backends] Kafka[(Kafka)] - Redis[(Redis)] + Valkey[(Valkey)] Vault[(HashiCorp Vault)] end Daprd -- pubsub.kafka --> Kafka - Daprd -- state.redis --> Redis + Daprd -- state.redis --> Valkey Daprd -- secretstore.hashicorp.vault --> Vault subgraph Subscribers[Subscriber side] @@ -40,7 +40,7 @@ flowchart LR The sidecar shares the network namespace of the app pod (Docker compose `network_mode: "service:learnstack-api"`; Kubernetes via `dapr.io/enabled` annotation). -The app talks to the sidecar via `localhost`, never directly to Kafka / Redis / Vault. +The app talks to the sidecar via `localhost`, never directly to Kafka / Valkey / Vault. ## 2. Components @@ -80,7 +80,16 @@ Topics follow the convention `learnstack.{module}.{aggregate}`. Examples: - `learnstack.hub.entitlement` (Hub-side) - `learnstack.cache.invalidation` (cross-instance L1 cache invalidation) -### `statestore.yaml` — Redis state store +### `statestore.yaml` — Valkey state store + +> The component below uses `spec.type: state.redis` and `redisHost` metadata — +> these are **Dapr provider-type / RESP-protocol identifiers**, NOT vendor +> brand markers. The actual backend is Valkey 8.x per +> [ADR-0030](../decisions/0030-redis-compatible-store-valkey.md); Valkey is +> drop-in compatible on the RESP wire protocol so the Dapr `state.redis` +> adapter consumes it unchanged. Operators read `redisHost` as "where to +> reach the RESP-compatible store"; in dev compose the value points at the +> `valkey` service (`infra/dapr/components/statestore-redis.yaml`). ```yaml apiVersion: dapr.io/v1alpha1 @@ -137,7 +146,7 @@ Secret path schema: secret/learnstack/postgres connection-string, ssl-cert secret/learnstack/redis password secret/learnstack/keycloak base-url, admin-username, admin-password -secret/learnstack/minio endpoint, access-key, secret-key +secret/learnstack/seaweedfs endpoint, access-key, secret-key secret/learnstack/meilisearch master-key, public-key secret/learnstack/livekit api-key, api-secret, ws-url secret/learnstack/coturn shared-secret diff --git a/docs/architecture/32-tenant-customization-model.md b/docs/architecture/32-tenant-customization-model.md index bbc5fe9..099e7d4 100644 --- a/docs/architecture/32-tenant-customization-model.md +++ b/docs/architecture/32-tenant-customization-model.md @@ -367,7 +367,7 @@ LearnStack core owns: - **Aggregate lifecycle** — `Course → CourseVersion → Module → Lesson → LessonItem` hierarchy. Tenant can't redefine that an enrollment binds to a `CourseVersion`. - **Auth flow** — Keycloak realm structure, JWT shape, MFA enforcement. -- **Storage layout** — MinIO bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...`. +- **Storage layout** — SeaweedFS bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...`. - **Outbox / event-bus contract** — integration event shapes are LearnStack-defined. - **Primitive set** — adding `whiteboard` or `3d-model` primitive is a LearnStack release. - **Audit pipeline** — modules cannot opt out of audit; per-(module, operation) toggling diff --git a/docs/decisions/0002-initial-architecture.md b/docs/decisions/0002-initial-architecture.md index e8abea6..c1921f2 100644 --- a/docs/decisions/0002-initial-architecture.md +++ b/docs/decisions/0002-initial-architecture.md @@ -2,7 +2,22 @@ ## Status -Accepted +Accepted with two amendments — see the bottom of this document for the dated +amendment blocks. Each amendment supersedes a single backend row of the +original Decision section without rewriting the rest of the ADR: + +- **Amendment 1 (2026-05-19):** storage backend MinIO → SeaweedFS per + [ADR-0029](0029-object-storage-seaweedfs.md). +- **Amendment 2 (2026-05-19):** cache + state backend Redis → Valkey per + [ADR-0030](0030-redis-compatible-store-valkey.md); PostgreSQL major + version pinned to 18.x per + [ADR-0031](0031-postgresql-major-version.md). + +Every other choice in the Decision section below — .NET 10, ASP.NET Core, +EF Core, modular monolith, Next.js — stands. The Decision + Consequences +text is the original, immutable form per CLAUDE.md's "never edit an +Accepted ADR's decision section" rule; read it together with the +Amendment blocks at the bottom for the current backend choices. ## Decision @@ -23,3 +38,55 @@ The team has stronger familiarity with .NET, so .NET 10 is preferred over Go for - MinIO should be used locally for S3-compatible object storage. - Next.js should be used for public rendering, admin studio, and portals initially. +--- + +## Amendment 1 — Storage backend: MinIO → SeaweedFS (2026-05-19) + +The original storage row picked MinIO. Two things changed: + +1. The `minio/minio` container repository was archived in 2026-04 with no + ongoing free-tier image stream. +2. MinIO Inc.'s licensing trajectory removed the no-phone-home, + no-license-key posture that the Self-Hosted Air-Gapped deployment mode + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires. + +[ADR-0029](0029-object-storage-seaweedfs.md) records the replacement +decision: SeaweedFS sits behind the same `IStorageProvider` S3-shaped +contract (no code-level adapter contract change), Apache 2.0 licensed, +single-binary self-hostable. The rest of ADR-0002 is unchanged. + +Every doc that previously said "MinIO" should be read as "SeaweedFS" for +operational guidance; conceptual rules ("tenant key-prefix isolation", +"swap to AWS S3 in SaaS through `IStorageProvider`") were never +backend-specific and stand verbatim. + +## Amendment 2 — Cache: Redis → Valkey; PostgreSQL: pin major to 18.x (2026-05-19) + +Two backend-row clarifications recorded together because they share the +trigger (do major-version + vendor calls while LearnStack is still +pre-implementation, so the migration drag is zero): + +1. **Cache + state backend Redis → Valkey** per + [ADR-0030](0030-redis-compatible-store-valkey.md). Redis 7.4 was the + last BSD-3-Clause Redis release; the 8.x line ships under a + triple-license (AGPLv3 / RSALv2 / SSPLv1). Valkey is the + Linux-Foundation-governed BSD-3-Clause fork, drop-in compatible on + the RESP protocol. The Dapr `state.redis` component name does not + change — it is the RESP-provider identifier, not a vendor brand. +2. **PostgreSQL major pinned to 18.x** per + [ADR-0031](0031-postgresql-major-version.md). 18 is the longest- + runway LTS available (EOL 2030-11), brings native `gen_uuid_v7()` + that the [ADR-0023 draft](README.md#open-adr-drafts) can adopt + without an extension, and async I/O for sequential scans helps the + partitioned `audit_log` + ([ADR-0016](0016-audit-log-subsystem.md)) operator queries. RLS + policy syntax + connection-string + role provisioning are unchanged + from 16 / 17, so the tenant-isolation defense-in-depth pattern + ([ADR-0003](0003-tenant-isolation-defense-in-depth.md)) transfers + verbatim. + +Every doc that previously said "Redis 7" or "PostgreSQL 16" should be +read as "Valkey 8" or "PostgreSQL 18". Library names + protocol +identifiers stay (`StackExchange.Redis`, `IConnectionMultiplexer`, +`state.redis` Dapr component, RESP) — those are protocol/library +identifiers, not vendor brands. diff --git a/docs/decisions/0003-tenant-isolation-defense-in-depth.md b/docs/decisions/0003-tenant-isolation-defense-in-depth.md index b115a6a..979b991 100644 --- a/docs/decisions/0003-tenant-isolation-defense-in-depth.md +++ b/docs/decisions/0003-tenant-isolation-defense-in-depth.md @@ -43,7 +43,7 @@ table without altering the tenant-level guarantees. | **Organization filter (new)** | `organization_id` column on org-scoped entities + EF query filter + RLS policy + architecture test | | Identity | Keycloak realm-per-tenant + `organization_id` JWT claim populated from active org | | Cache | Cache keys auto-prefixed `{tenant_id}:{organization_id}:{key}` when org context set; `{tenant_id}:{key}` otherwise | -| Files (MinIO) | Object key prefix `tenants/{tenant_id}/organizations/{org_id}/...` when org-scoped; `tenants/{tenant_id}/...` when tenant-wide | +| Files (SeaweedFS) | Object key prefix `tenants/{tenant_id}/organizations/{org_id}/...` when org-scoped; `tenants/{tenant_id}/...` when tenant-wide | | Search (Meilisearch) | Query filter `tenant_id = X AND (organization_id = Y OR organization_id IS NULL)` for org-context queries | | Jobs (Hangfire) | Job payload carries `TenantId` (mandatory) + `OrganizationId?` (when applicable) | | Audit | Every audit row carries `tenant_id` (mandatory) + `organization_id?` (when applicable) — ADR-0016 | diff --git a/docs/decisions/0010-cross-module-communication.md b/docs/decisions/0010-cross-module-communication.md index 4598041..48ab0d1 100644 --- a/docs/decisions/0010-cross-module-communication.md +++ b/docs/decisions/0010-cross-module-communication.md @@ -65,7 +65,7 @@ The following tests live in `LearnStack.Tests.Architecture` and run on every PR: ## Amendment 1 — Dapr pub/sub as outbox dispatch target (2026-05-18) Per [ADR-0014](0014-adopt-dapr.md), LearnStack adopts Dapr for cross-cutting infrastructure -(pub/sub Kafka, state Redis, secrets Vault). This amendment specifies how Mechanism #3 +(pub/sub Kafka, state Valkey, secrets Vault). This amendment specifies how Mechanism #3 (integration event via outbox) is dispatched. **Updated dispatch flow:** diff --git a/docs/decisions/0014-adopt-dapr.md b/docs/decisions/0014-adopt-dapr.md index 0773818..890ba41 100644 --- a/docs/decisions/0014-adopt-dapr.md +++ b/docs/decisions/0014-adopt-dapr.md @@ -14,9 +14,9 @@ LearnStack adopts **Dapr** (Distributed Application Runtime) for three building | Building block | Backend (production) | Component file | |----------------|---------------------|----------------| -| Pub/Sub | Apache Kafka | `dapr/components/pubsub.yaml` | -| State store | Redis | `dapr/components/statestore.yaml` | -| Secret store | HashiCorp Vault | `dapr/components/secretstore-vault.yaml` | +| Pub/Sub | Apache Kafka | `infra/dapr/components/pubsub-kafka.yaml` | +| State store | Valkey (RESP-protocol fork; see ADR-0030) | `infra/dapr/components/statestore-redis.yaml` (file name keeps the `-redis` suffix because `state.redis` is the Dapr provider-type identifier, not the vendor brand) | +| Secret store | HashiCorp Vault | `infra/dapr/components/secretstore-vault.yaml` | Application code interacts with Dapr **exclusively through wrapped abstractions** — `IEventBus`, `ICacheService`, `ISecretProvider` in `LearnStack.SharedKernel` — never via @@ -33,13 +33,13 @@ LearnStack is a modular monolith multi-tenant PaaS for education. It needs: - Cross-module integration events with at-least-once delivery, durable replay, multi-tenant fan-out. The outbox pattern (ADR-0010, ADR-0006) requires a dispatch target. - Distributed cache (L2) on top of in-process memory cache (L1) for tenant-scoped reads. -- Secret access for Keycloak admin credentials, MinIO access keys, LiveKit API secrets, +- Secret access for Keycloak admin credentials, SeaweedFS access keys, LiveKit API secrets, Stripe/Iyzico API keys, exchange rate API keys, etc. The platform plans triple deployment (SaaS / Dedicated / Self-Hosted; ADR-0020). All three must use the same backend abstractions; backend providers may differ per deployment -(Vault in production, file-based secret store in dev; managed Redis in SaaS, self-hosted -Redis on-prem). +(Vault in production, file-based secret store in dev; managed Valkey in SaaS, self-hosted +Valkey on-prem). Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, `Nexora/docs/decisions/0005-transactional-outbox.md`, @@ -50,7 +50,7 @@ Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, module-level coupling to Dapr** — the interfaces live in SharedKernel, the `DaprXxxService` implementation lives in Infrastructure, and modules never import `Dapr.Client`. -- Component swap (Kafka → another broker, Redis → another KV) is a configuration-only +- Component swap (Kafka → another broker, Valkey → another KV) is a configuration-only change with the same interface signatures. - Same sidecar pattern works in Docker Compose (dev), Kubernetes (production), and air-gapped installations (on-prem with bundled sidecar binary). @@ -58,7 +58,7 @@ Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, ## Decision drivers 1. **Provider portability.** A LearnStack deployment may need to replace Kafka with RabbitMQ, - Redis with KeyDB, Vault with AWS Secrets Manager. Dapr's component model treats this as a + Valkey with KeyDB, Vault with AWS Secrets Manager. Dapr's component model treats this as a YAML change, not a code change. 2. **Same abstraction across deployments.** SaaS uses managed Kafka; Self-Hosted uses bundled Kafka; both call the same `IEventBus.PublishAsync`. The differentiating layer is @@ -207,7 +207,7 @@ Three architecture tests enforce the abstraction boundary (added in Phase 02): ### Positive -- Provider portability: switching Kafka → RabbitMQ, Redis → KeyDB, Vault → AWS Secrets +- Provider portability: switching Kafka → RabbitMQ, Valkey → KeyDB, Vault → AWS Secrets Manager is a Dapr component YAML change. - Same code, same components across SaaS / Dedicated / Self-Hosted. - Outbox + at-least-once delivery + retry + DLQ provided by Dapr pub/sub; LearnStack's diff --git a/docs/decisions/0017-tenant-organization-hierarchy.md b/docs/decisions/0017-tenant-organization-hierarchy.md index cdf68a0..261ad0a 100644 --- a/docs/decisions/0017-tenant-organization-hierarchy.md +++ b/docs/decisions/0017-tenant-organization-hierarchy.md @@ -177,7 +177,7 @@ public OrganizationId? OrganizationId { get; private set; } // null = tenant-w | **Organization filter** | `organization_id` column + EF query filter + RLS policy + arch test | | Identity | Keycloak realm-per-tenant; `organization_id` JWT claim populated from active org | | Cache | Cache keys auto-prefixed `{tenant_id}:{organization_id}:{key}` when org context set | -| Files (MinIO) | Bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...` | +| Files (SeaweedFS) | Bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...` | | Search (Meilisearch) | Query filter `tenant_id = X AND (org_id = Y OR org_id = null)` (tenant-wide content visible to all orgs) | | Jobs (Hangfire) | Job payload carries both `TenantId` and `OrganizationId?` | | Audit | Every audit row carries `tenant_id` (mandatory) + `organization_id?` | diff --git a/docs/decisions/0018-tenant-driven-customization-model.md b/docs/decisions/0018-tenant-driven-customization-model.md index 9f78b97..da74290 100644 --- a/docs/decisions/0018-tenant-driven-customization-model.md +++ b/docs/decisions/0018-tenant-driven-customization-model.md @@ -387,7 +387,7 @@ primitive based on its JSON Schema type and format, composes the result. aggregate's lifecycle. - **Authentication and authorization plumbing** — Keycloak realm structure, JWT emission, permission policy evaluation. -- **Storage layout** — MinIO bucket prefixes, object key conventions. +- **Storage layout** — SeaweedFS bucket prefixes, object key conventions. - **Outbox / event-bus contract** — integration event shapes are LearnStack-defined. - **Generic primitive set** — adding a new primitive (e.g. `whiteboard`) is a LearnStack release, not a tenant action. Tenants compose existing primitives. diff --git a/docs/decisions/0022-custom-domain-tls.md b/docs/decisions/0022-custom-domain-tls.md index 416c776..6e7c477 100644 --- a/docs/decisions/0022-custom-domain-tls.md +++ b/docs/decisions/0022-custom-domain-tls.md @@ -345,7 +345,7 @@ public sealed class TenantMiddleware } ``` -`_hostToTenantResolver` is backed by `ICacheService` (Dapr State / Redis); cache key +`_hostToTenantResolver` is backed by `ICacheService` (Dapr State / Valkey); cache key `hub:host:{host}` invalidated on `CustomDomainActivatedEvent` / `CustomDomainRevokedEvent`. ### Public suffix list validation diff --git a/docs/decisions/0029-object-storage-seaweedfs.md b/docs/decisions/0029-object-storage-seaweedfs.md new file mode 100644 index 0000000..2dd7120 --- /dev/null +++ b/docs/decisions/0029-object-storage-seaweedfs.md @@ -0,0 +1,204 @@ +# ADR-0029: Object Storage — SeaweedFS + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "MinIO" choice on +the storage row only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **Original choice (MinIO) lost its free-tier path.** The MinIO Inc. policy + shift around the AGPL community edition and the archival of the official + `minio/minio` container repository in 2026-04 mean there is no supported + upstream stream of new tagged images for the self-hosted, no-license-server + posture LearnStack relies on. Continuing on the last free image + (`RELEASE.2025-09-07T16-13-09Z`) is a frozen-image dead end, not a + maintainable foundation. +- **Provider portability is non-negotiable.** Per + [ADR-0002](0002-initial-architecture.md) + [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md), + every external dependency sits behind a `LearnStack`-defined interface. + `IStorageProvider` already abstracts the storage adapter — the choice of + backend behind the interface should be an operational call, not a code + rewrite. +- **Self-Hosted + Air-Gapped must work.** The triple-deployment model + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires the + storage backend to ship as a single self-contained binary or container + with no phone-home, no license-key check, no SaaS dependency. +- **S3 API compatibility is the integration contract.** The `IStorageProvider` + adapter that the rest of the stack talks to is written against the S3 API + (signed URLs, multipart upload, key-prefix isolation per + [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md)). + Any backend that does not expose a high-fidelity S3 API forces a second + adapter layer. +- **Day-1 multi-region story is a Phase-11 concern, not Day-1.** The MVP + scope is single-region; the chosen backend must not paint us into a + corner if Phase 11 adds replication, but multi-region is not required + today. +- **Operational footprint must stay one-process-small.** The dev compose + is already 14 services; a storage backend that introduces three or four + more processes (or a dedicated control plane) is disproportionate cost. + +## Considered Options + +1. **SeaweedFS (chosen).** Self-hosted single binary, native S3-compatible + API, Apache 2.0 license, active upstream, optional volume + filer + S3 + gateway split-or-merged per deployment size. +2. **Stay on the last free MinIO image, indefinitely (rejected).** + `minio/minio:RELEASE.2025-09-07T16-13-09Z` works today but freezes the + stack to a no-longer-supported image — including security patches. This + makes ADR-0002's storage row a long-term tech debt magnet. +3. **Garage S3 (rejected for now).** Rust-based, Apache 2.0, smaller + feature set. Strong fit for edge / small deployments; weaker + production-scale story and a smaller operational community. Worth + revisiting if SeaweedFS develops a blocker. +4. **Ceph RGW (rejected).** Heavy-weight; designed for petabyte clusters + with a separate operator team. Disproportionate for LearnStack's + single-binary Self-Hosted posture; production multi-region story is + solid but is solving a problem we do not yet have. +5. **Cloud-managed S3 only (rejected — incompatible with deployment + model).** Amazon S3 / R2 / B2 are excellent for SaaS; they directly + violate Self-Hosted Air-Gapped (ADR-0020). A SaaS-only deployment + could swap behind `IStorageProvider`, but the *default* must be + self-hostable. + +## Decision + +LearnStack adopts **SeaweedFS** as the object-storage backend behind the +`IStorageProvider` adapter for all four deployment modes (Development / +SaaS / Dedicated / SelfHosted). The S3 gateway exposes the same S3 API +surface the MinIO-based adapter relied on, so adapter code remains +unchanged in signature. + +Image: `chrislusf/seaweedfs:3.94` in dev (the current pinned tag in +`infra/compose/dev.yml`); every environment carries an explicit tag per +[Standards 12 § Image Conventions](../standards/12-infrastructure.md) — +no environment uses `:latest`. + +This ADR **supersedes the storage choice in ADR-0002 only** — the rest of +ADR-0002 (Postgres, Valkey, modular monolith) is unchanged. + +## Context + +The repository accreted ~27 docs that name MinIO directly (Standards 11, +Standards 12, Standards 20, ADR-0002, ADR-0003, ADR-0014, ADR-0017, +ADR-0018, architecture 02 / 03 / 04 / 05 / 07 / 08 / 09 / 16 / 18 / 23 / +25 / 29 / 32, decisions/README). The Day-1 commitment was: provider +portability behind `IStorageProvider`, single-binary self-hostable +backend, S3 API as the integration contract. MinIO satisfied all three at +the time; SeaweedFS satisfies all three today **and** is not on a +license-shift trajectory. + +The bulk of the MinIO references describe **storage characteristics that +remain true** — tenant key-prefix isolation (`{tenant_id}/...`), +production swap-out to AWS S3, partition-friendly key layout. Those rules +do not change. The backend name does, and a tightly-scoped doc sweep +plus a runtime config swap close out the migration. + +### Why MinIO got us here and why we're moving off + +MinIO was the right choice in 2024–early 2026 because it shipped as a +self-hosted binary, exposed a high-fidelity S3 API, and stayed on a +permissive license. The 2026-04 archival of the `minio/minio` Docker Hub +repository plus MinIO Inc.'s pivot to a commercial-first posture removed +the *no-license-server, no phone-home* path the Self-Hosted Air-Gapped +deployment mode (ADR-0020) requires. The platform-level commitment is to +keep that mode first-class; the backend choice must follow. + +### Why SeaweedFS + +- **Single binary**, multi-process when scale needs it. The dev compose + runs one container; production can split master / volume / filer / S3 + gateway across replicas. +- **Native S3 gateway** (`weed s3` subcommand) exposing a near-complete + S3 API surface — signed URLs, multipart upload, bucket policies, key + versioning. Sufficient for what `IStorageProvider` consumes today. +- **Apache 2.0 license**, no telemetry phone-home, no license key + required. +- **Active upstream** (commits weekly, stable release cadence). +- **Operational primitives we already need**: tiered storage (hot → + warm), erasure coding (Phase 11 cost-optimization), built-in metrics + (Prometheus-compatible). + +### Why not Garage + +Garage's feature set is tighter (no native erasure coding; smaller +ecosystem). The trade-off is conscious and we re-consider Garage if +SeaweedFS produces a concrete blocker — at that point Garage is one +ADR + one adapter-config swap away. + +### Adapter-level impact: none + +`IStorageProvider` already abstracts the storage backend behind a S3- +shaped contract (signed URL issuance, multipart upload coordination, +bucket + key access, server-side encryption). The SeaweedFS S3 gateway +implements the subset `IStorageProvider` uses. The composition root in +each deployment mode picks the backend's endpoint + credentials; no +module-level code change is required. + +## Consequences + +### Positive + +- Self-Hosted Air-Gapped stays first-class — no phone-home, no license + key, no commercial-tier gating. +- The dev compose stays single-container for storage (no operator- + pattern overhead at dev time). +- Apache 2.0 license matches the rest of LearnStack's OSS posture + (Standards 20 § Self-Hosted Infrastructure Preferred). +- Production tiered storage + erasure coding give us a clear cost- + reduction lever in Phase 11 without a backend swap. +- The decision **also** answers "what happens if MinIO Inc. tightens + licensing further" — we're already off. + +### Negative + +- One-time doc + scaffold migration: ~27 doc references + the dev + compose service + the `IStorageProvider` adapter implementation + (Phase 02a). The adapter change is small because both backends speak + S3, but the doc sweep is mechanical work. +- SeaweedFS S3 gateway is not 100% S3 surface — a few edge endpoints + (CloudFront-specific extensions, S3 Object Lambda) do not exist. + None are on the LearnStack-side dependency list today; flagging so + Phase 02a's adapter-write does not assume them. +- Smaller operational community than MinIO at its peak. Mitigated by + active upstream and the architecture-level provider-portability + commitment — switching backends behind `IStorageProvider` remains a + composition-root edit. + +### Neutral + +- The `MINIO_` env-var names in the dev compose become `SEAWEEDFS_` (or + the equivalent SeaweedFS variables). Names move; semantics do not. +- Production AWS S3 / Cloudflare R2 / Backblaze B2 swap-in continues to + work through `IStorageProvider`; the dev backend choice is independent + of the production backend choice. + +## Implementation Notes + +- Phase 01 packet 6 cleanup (this ADR's commit): the dev compose `minio` + service is replaced by a `seaweedfs` service exposing the S3 gateway + on the same `localhost:9000` port (drop-in for any dev URL that + hardcoded the old endpoint), with the master / volume processes on + their own internal ports. The MinIO console (port 9001) becomes + SeaweedFS's filer UI. +- Phase 02a (Storage adapter): `LearnStack.Infrastructure.Storage.SeaweedFS` + ships the `IStorageProvider` implementation. The signed-URL, + multipart-upload, and bucket-prefix conventions in + [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md) + stay verbatim — the adapter speaks S3 underneath. +- Phase 11 (production): erasure coding policy + tiered storage tuning; + multi-region replication evaluation; production credential rotation + (today's dev credentials live in dev compose only). + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original storage row, now superseded for this slot. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) — Self-Hosted Air-Gapped requirement that motivated the move. +- [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md) +- [Standards 20 § Self-Hosted Infrastructure](../standards/20-infrastructure-stack.md) +- [architecture/16-media-pipeline.md](../architecture/16-media-pipeline.md) — media + recording storage paths. +- SeaweedFS upstream: . diff --git a/docs/decisions/0030-redis-compatible-store-valkey.md b/docs/decisions/0030-redis-compatible-store-valkey.md new file mode 100644 index 0000000..808c83b --- /dev/null +++ b/docs/decisions/0030-redis-compatible-store-valkey.md @@ -0,0 +1,180 @@ +# ADR-0030: Redis-compatible Store — Valkey + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "Redis" choice on +the cache + state-store row only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **Redis Inc. left BSD in 2024-03.** Redis 7.4 was the last BSD-3-Clause + release. From 7.4 onward the project ships under a triple-license + (AGPLv3 / RSALv2 / SSPLv1) where the user picks one. None of those are + permissive OSS in the BSD sense, and SSPL in particular has not been + tested in court for indirect-SaaS usage — a gray area we do not want a + Self-Hosted tenant's legal team to discover months into a deployment. +- **Valkey** is the Linux-Foundation-governed BSD-3-Clause fork of Redis + 7.2.4. AWS, Google, Oracle, Ericsson, Snap, Tencent and others are + first-tier sponsors; the governance is vendor-neutral by design. + Valkey 8.x reached protocol + command parity with Redis 7.4 in + 2024-11 and added meaningful CPU/RAM improvements on top + (multi-threaded I/O, smarter command pipelining). +- **Drop-in compatibility.** Valkey speaks the RESP protocol identically + to Redis. Every library the LearnStack stack uses — `StackExchange.Redis` + (consumed only inside `LearnStack.Infrastructure` per ADR-0014), + Dapr's `state.redis` component, `ICacheService` over Dapr — sees the + same wire protocol. Switching backends is one image tag and one + `redisHost` value. +- **Provider portability is non-negotiable.** Per + [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md) + the cache + state backend already sits behind `ICacheService` (Dapr + building block). The provider portability claim of ADR-0014 only + holds when the underlying provider is not on a license-shift + trajectory; Valkey honours that claim, Redis Inc. 8.x trajectory + weakens it. +- **Self-Hosted Air-Gapped must work.** Triple-deployment model + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires a + backend with no phone-home, no license-key check, no commercial-tier + gating. Valkey satisfies that; the SSPL/RSALv2 sides of Redis 8.x + introduce ambiguity at the Self-Hosted boundary. +- **Ecosystem signal.** Debian 13, Ubuntu 25.04, RHEL 10 made Valkey the + default `redis`-named package. AWS ElastiCache for Valkey is ~20% + cheaper than ElastiCache for Redis. The boring-choice principle + (Standards 00 § 9) now points at Valkey, not Redis. + +## Considered Options + +1. **Valkey** (chosen). BSD-3-Clause, Linux Foundation governance, + RESP-protocol drop-in for Redis 7.4. +2. **Stay on Redis 7.4 indefinitely** (rejected). The 7.4 line gets + security patches under the old license, but no new features arrive; + the rest of the world moves to either Valkey or Redis 8.x, and our + image freeze becomes an EOL clock. +3. **Adopt Redis 8.x (AGPLv3 selection)** (rejected). AGPL applied to a + network-reachable backend in a SaaS context is a copyleft surface a + proprietary platform like LearnStack should not adopt without legal + review; the gain over Valkey is small (vendor parity), the risk is + real (license interpretation). +4. **DragonflyDB** (rejected). High-performance RESP-compatible + in-memory store, BSL-licensed (also source-available, not OSS by OSI + definition). Solves the wrong problem — performance is not the + bottleneck, license clarity is. +5. **KeyDB** (rejected). Active development slowed substantially after + Snap acquisition; the project's future under EVA Information Security + is unclear. Not a stable bet. + +## Decision + +LearnStack adopts **Valkey** as the Redis-compatible cache + state +backend behind `ICacheService` (Dapr `state.redis` component) for all +four deployment modes. The Dapr component name `state.redis` is the +**Dapr provider-type identifier** (RESP-compatible store) — it does not +imply the Redis Inc. brand and does not change. + +Image (dev compose): `valkey/valkey:8.1-alpine`, pinned per +[Standards 12 § Image Conventions](../standards/12-infrastructure.md). +Production swaps to AWS ElastiCache for Valkey / equivalent managed +offering through the composition root. + +This ADR **supersedes the cache + state-store row of ADR-0002 only** — +the rest of ADR-0002 (.NET 10, EF Core, Next.js, modular monolith) +stays. Together with [ADR-0029 (SeaweedFS)](0029-object-storage-seaweedfs.md), +both backend rows of ADR-0002 now have explicit successor decisions. + +## Context + +The original choice (Redis) was correct in 2023 — BSD-3-Clause, widely +known, every library + cloud provider supported it. The 2024-03 license +shift removed the unambiguous BSD path; the BSD-3-Clause continuation +moved to the Valkey fork. + +The platform-level commitment to keep Self-Hosted Air-Gapped first-class +(ADR-0020) is what forces the choice now rather than later: a Self- +Hosted tenant inherits whatever license the backend ships with, and we +do not want their security/legal review to find SSPL-laden code paths +they then have to interpret. Valkey moves that conversation off the +table entirely. + +### Backward-compat: what does NOT change + +- **`ICacheService`** interface (`LearnStack.SharedKernel`) — unchanged. +- **Dapr `state.redis` component name** — unchanged. The component is the + RESP-protocol adapter; Valkey is RESP, so it consumes the same + component. +- **`StackExchange.Redis` library usage** — unchanged. The library + connects to anything speaking RESP. +- **`IConnectionMultiplexer` forbidden-in-modules rule** (Standards 20) + — unchanged. The rule is about not importing the library outside + `LearnStack.Infrastructure`; the rule does not care about the backend + vendor. +- **Architecture tests** `ICacheService_Is_OnlyCacheAbstraction`, + `Modules_DoNotReference_DaprPackage` — unchanged. + +### What DOES change + +- `infra/compose/dev.yml`: service `redis` → `valkey`; image + `redis:7.4-alpine` → `valkey/valkey:8.1-alpine`; volume `redis-data` + → `valkey-data`; `depends_on: redis:` callers → `valkey:`. +- `infra/dapr/components/statestore-redis.yaml`: `redisHost: redis:6379` + → `redisHost: valkey:6379` (the file name stays — `state.redis` is the + Dapr provider-type convention). +- The "boring choice" enumeration in standards / CLAUDE.md / README.md + / ADR-0002: "Redis 7" → "Valkey 8". +- `.gitignore`: `redis-data/` → `valkey-data/`. + +## Consequences + +### Positive + +- License surface stays BSD-3-Clause — no AGPL/SSPL/RSALv2 to interpret. +- Self-Hosted Air-Gapped stays first-class. +- Vendor-neutral governance (Linux Foundation under) — no single-vendor + policy-shift risk. +- AWS ElastiCache for Valkey ~20% cheaper than for Redis — direct SaaS + margin win. +- Drop-in: zero application-code change required; Dapr component swap is + one YAML edit. + +### Negative + +- One-time dev image swap + the doc/scaffold sweep. Both mechanical. +- Smaller commercial-support market than Redis Inc.'s, today. Mitigated + by Linux Foundation backing + the major cloud providers committing + managed offerings. +- A subset of Redis Inc.'s newest commercial features (Redis Search, + Redis JSON commercial extensions in 8.x) does not exist in Valkey. + None of these are on LearnStack's roadmap; flagging so a future + feature ADR that wants them knows the choice. + +### Neutral + +- The `MINIO_`-style env-var rename trap does not apply here — Valkey + honours the same `REDISCLI_AUTH` / connection-string conventions + Redis does. +- Production swap to AWS / GCP / Azure managed offerings remains a + composition-root edit through `ICacheService`. + +## Implementation Notes + +- **This commit** (Phase 01 packet 6 cleanup): dev compose service swap; + Dapr component `redisHost` update; doc + standards sweep; `.gitignore` + volume name update. +- **Phase 02a** (Cache adapter): `LearnStack.Infrastructure.Caching.Dapr` + consumes `state.redis` component against Valkey — no code-level + awareness of the backend vendor. +- **Phase 11** (production hardening): production sizing + Valkey HA + topology decision (Sentinel vs Cluster mode) lives in its own ADR if + it diverges from the default. + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original cache + state row, now partially superseded. +- [ADR-0014 Adopt Dapr](0014-adopt-dapr.md) — `ICacheService` over Dapr `state.redis`. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) — Self-Hosted Air-Gapped requirement that motivated the move. +- [Standards 12 § Local Infrastructure](../standards/12-infrastructure.md) +- [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md) +- Valkey upstream: . diff --git a/docs/decisions/0031-postgresql-major-version.md b/docs/decisions/0031-postgresql-major-version.md new file mode 100644 index 0000000..8c95a29 --- /dev/null +++ b/docs/decisions/0031-postgresql-major-version.md @@ -0,0 +1,184 @@ +# ADR-0031: PostgreSQL — Start on Major Version 18 + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "PostgreSQL" +major-version choice only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **LearnStack is pre-implementation.** No migrations exist yet, no + production database exists, no data exists. The migration-drag cost + of choosing the wrong major version is zero today and grows + exponentially after the first deployed schema. +- **Postgres 18 is the longest-runway LTS available.** EOL 2030-11 + versus 16 LTS at 2028-11. Starting on 18 buys an extra two years of + upstream patches before any forced major upgrade. +- **`gen_uuid_v7()` is native in 18.** LearnStack's + [ADR-0023 (Strongly-typed ID source generator)](README.md#open-adr-drafts) is a + pending draft considering UUIDv7 as the canonical id format + (time-ordered, index-friendly). Postgres 18 ships a built-in + `gen_uuid_v7()` SQL function — DB-side DEFAULT values become trivial, + the app side keeps the strongly-typed wrapping, no extension is + required. Postgres 16 / 17 force a choice between an extension + (`pg_uuidv7`) and app-side generation; Postgres 18 closes that gap + natively. +- **Async I/O for sequential scans (Postgres 18).** On NVMe-backed + production hardware the perf improvement is measurable and + particularly relevant for the partitioned `audit_log` (per + [ADR-0016](0016-audit-log-subsystem.md)) scans that operators run + during incident review. +- **OAuth authentication (Postgres 18).** Opens a future option to + shorten the Keycloak → Postgres auth path for diagnostic / + break-glass scenarios. Not adopted today (the + [ADR-0004](0004-authentication-strategy.md) realm-based posture + stays), but worth recording as a Phase-11 lever. +- **EF Core + Npgsql provider parity.** `Npgsql.EntityFrameworkCore.PostgreSQL` + 10.0.0 (already pinned in `Directory.Packages.props`) supports + Postgres 18 features. Switching majors does not require a provider + bump; the same csproj graph works. +- **RLS-specific defaults are unchanged.** Tenant + organization + isolation defense-in-depth ([ADR-0003 Amendment 1](0003-tenant-isolation-defense-in-depth.md)) + uses the `PERMISSIVE` / `RESTRICTIVE` RLS-policy primitives + the + `app.tenant_id` / `app.organization_id` session-var pattern. None of + this changed across Postgres 16 / 17 / 18; the policy shape we will + write in Phase 02a works identically on 18. + +## Considered Options + +1. **Start on PostgreSQL 18.x** (chosen). Newest LTS, longest runway, + native UUIDv7, async I/O perf, pre-implementation = zero migration + cost. +2. **Stay on 16 LTS until production** (rejected). Defers the major + upgrade to Phase 11 where every migration already exists; the + `pg_upgrade` exercise + extension recompile + app-side UUIDv7 swap + become a multi-day operational task instead of a zero-cost image + bump. +3. **Skip to 17, defer 18 until proven** (rejected). 17 is already + superseded by 18 LTS; adopting 17 buys nothing 18 doesn't, and + forces the same upgrade question in 18 months. +4. **Adopt 18 only for dev, keep 16 for production** (rejected). + Splits the deployment-mode portability promise (ADR-0020) — every + feature has to be tested on both major versions, every migration + has to be written for the lowest-common-denominator. The single- + binary, single-major-Postgres posture is simpler and safer. + +## Decision + +LearnStack's primary RDBMS is **PostgreSQL 18.x** across all four +deployment modes. Dev compose pins `postgres:18.4-alpine`. EF Core +provider (`Npgsql.EntityFrameworkCore.PostgreSQL` 10.0.0) targets 18. +Tenant + organization RLS policies are written for the 18 syntax (no +divergence from 16 / 17 here; the choice is forward-looking, not a +compatibility break). + +This ADR **supersedes the PostgreSQL major-version choice in ADR-0002 +only** — the rest of ADR-0002 (EF Core, modular monolith, …) stays. +Together with [ADR-0029 (SeaweedFS)](0029-object-storage-seaweedfs.md) +and [ADR-0030 (Valkey)](0030-redis-compatible-store-valkey.md), all +three backend rows of ADR-0002 now have explicit successor decisions. + +## Context + +### Why now + +Every major upgrade gets harder as the schema grows. A pre- +implementation codebase has no schema. The cost of choosing 18 today is +one image tag change + a `docker compose down -v`; the cost of doing it +in Phase 11 is `pg_upgrade` across every production cluster + extension +recompilation + a possible app-side UUIDv7 generator swap if the +extension we picked early diverges from the 18 native function. + +### What 18 brings that we directly benefit from + +| 18 feature | LearnStack benefit | +|------------|--------------------| +| `gen_uuid_v7()` built-in | ADR-0023 draft can pick "DB-side DEFAULT" without committing to an extension | +| Async I/O for sequential scans | `audit_log` partition scans (ADR-0016) — operator query latency | +| OAuth authentication | Optional shortcut for Phase 11 break-glass paths (not adopted today) | +| Virtual generated columns | Computed columns for `LocalizedMessage`-like derived data (Phase 02a+) | +| `EXPLAIN (ANALYZE)` improvements | Day-to-day query tuning | + +### What does NOT change + +- **EF Core provider** — `Npgsql.EntityFrameworkCore.PostgreSQL` 10.0.0 + already supports 18. +- **RLS policy syntax** — identical in 16 / 17 / 18. +- **Connection-string + role provisioning** — same. +- **Session-var pattern** for `app.tenant_id` / `app.organization_id` + in transaction-local config — unchanged. +- **`pg_partman` / `pg_stat_statements` / `pgcrypto` / `citext`** — + all 18-ready in recent releases. +- **Dapr state store + outbox dispatcher patterns** — RDBMS-agnostic. + +### What the upgrade costs in dev + +A single line in `infra/compose/dev.yml` (`postgres:16.14-alpine` → +`postgres:18.4-alpine`) plus a one-time `docker compose -f +infra/compose/dev.yml down -v && up -d` to wipe the incompatible +catalog. No code change, no migration change (there are no migrations +yet). + +### What the upgrade costs in production (deferred to Phase 11) + +The first production deployment lands in Phase 11 anyway. Choosing 18 +now means Phase 11 deploys *fresh* on 18; there is no `pg_upgrade` +exercise to schedule. If LearnStack ever runs an earlier production +preview, that preview already runs on 18 — no major upgrade needed. + +## Consequences + +### Positive + +- Longest support runway (EOL 2030-11) — five-year horizon before any + forced major upgrade. +- Native UUIDv7 — ADR-0023 design space widens. +- Async I/O perf — direct benefit to `audit_log` and any future + read-heavy partitioned table. +- Phase 11 production deployment ships on the modern LTS without a + separate "upgrade Postgres" mini-project on its critical path. + +### Negative + +- 18 is younger than 16 LTS — community knowledge base is thinner; a + rare edge-case query plan regression may take longer to resolve via + StackOverflow / mailing list. Mitigated by EF Core provider maturity + + the broader Postgres ecosystem's quick uptake of LTS releases. +- Some Postgres-as-a-service offerings (some smaller cloud regions, a + handful of niche providers) may lag 18 availability by 1–2 quarters. + Mitigated by ADR-0020 portability: the SaaS / Dedicated deployment + modes can pin to a specific managed offering and the Self-Hosted + modes ship with the container image. + +### Neutral + +- Backup tooling (`pg_basebackup`, `pg_dump`) shape is unchanged — + Phase 11 backup runbooks stay the same template. +- `Standards 12 § Database Operations` continues to apply verbatim + (daily logical backups for dev-grade restore, continuous WAL + archiving in production). + +## Implementation Notes + +- **This commit** (Phase 01 packet 6 cleanup): dev compose image bump; + ADR-0002 Amendment 2 references this decision; doc sweep across + Standards 12 / Architecture / Standards 20. +- **Phase 02a** (Platform kernel): first EF migration targets Postgres + 18; if ADR-0023 picks UUIDv7, evaluate DB-side `gen_uuid_v7()` as the + default-value generator. +- **Phase 11** (production hardening): production sizing, backup + cadence, replication topology — all written for 18. + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original PostgreSQL major-version row, now partially superseded. +- [ADR-0003 Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) — RLS pattern unchanged across 16/17/18. +- [ADR-0016 Audit Log Subsystem](0016-audit-log-subsystem.md) — partitioned `audit_log` benefits from async I/O. +- [ADR-0023 Strongly-typed ID source generator](README.md#open-adr-drafts) — draft (no dedicated file yet — listed in the decisions index); UUIDv7 native in 18 widens the design space. +- [Standards 05 — Database](../standards/05-database.md) +- [Standards 12 § Database Operations](../standards/12-infrastructure.md) +- PostgreSQL 18 release notes: . diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 0e9f743..113f3f0 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -16,7 +16,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | # | Title | Topic | |---|---|---| | 0001 | [Platform Name](0001-platform-name.md) | Name and naming conventions | -| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Redis + MinIO + Next.js; modular monolith | +| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Valkey + SeaweedFS + Next.js; modular monolith | | 0003 | [Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) | Query filters + RLS + audit + architecture tests (Amendment 1: Organization scope, 2026-05-18) | | 0004 | [Authentication Strategy](0004-authentication-strategy.md) | Off-the-shelf identity provider preferred over hand-rolled auth (Amendment 1: `learnstack-hub` realm, 2026-05-18) | | 0005 | [Live Classroom Media Stack](0005-live-classroom-media-stack.md) | LiveKit OSS self-hosted by default; LiveKit Cloud optional; no custom SFU | @@ -28,7 +28,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0011 | _Superseded — see below_ | Was: Vertical Extension Points | | 0012 | [Search Strategy](0012-search-strategy.md) | Meilisearch; one instance per env; index-per-(kind, locale); tenant_id as query filter | | 0013 | [Page Block Schema Versioning](0013-page-block-schema-versioning.md) | `(key, schemaVersion)` tuple; immutable schemas; lazy + bulk migration; placeholder on unknown version | -| 0014 | [Adopt Dapr](0014-adopt-dapr.md) | Dapr building blocks for pub/sub (Kafka), state (Redis), secrets (Vault); abstracted behind SharedKernel interfaces | +| 0014 | [Adopt Dapr](0014-adopt-dapr.md) | Dapr building blocks for pub/sub (Kafka), state (Valkey), secrets (Vault); abstracted behind SharedKernel interfaces | | 0015 | [API Gateway with APISIX](0015-api-gateway-apisix.md) | APISIX standalone mode; JWT + rate limit + CORS + correlation-id at the edge; defense-in-depth | | 0016 | [Audit Log Subsystem](0016-audit-log-subsystem.md) | `LearnStack.Modules.Audit`; EF interceptor + `IAuditStateCapture` + `AuditLogBehavior`; partitioned `audit_log` table; retention | | 0017 | [Tenant + Organization Hierarchy](0017-tenant-organization-hierarchy.md) | Two-level: Tenant → Organization; permission scope Platform / Tenant / Organization | @@ -37,6 +37,9 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0020 | [Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) | SaaS / Dedicated / Self-Hosted from one codebase; phone-home + RSA-signed key + 30-day grace | | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | +| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | +| 0030 | [Redis-compatible Store — Valkey](0030-redis-compatible-store-valkey.md) | Valkey (Linux Foundation, BSD-3-Clause) for the cache + Dapr state-store backend; RESP-protocol drop-in; partially supersedes ADR-0002's Redis row | +| 0031 | [PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) | Pin primary RDBMS major version to PostgreSQL 18; native `gen_uuid_v7()` + async I/O + longest LTS runway; partially supersedes ADR-0002's PostgreSQL row | ## Superseded ADRs diff --git a/docs/glossary.md b/docs/glossary.md index e47ae19..048f500 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -75,7 +75,7 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | **Live Attendance** | A computed or recorded record of who joined a Live Session, for how long, and in what role. | | **Live Session Material** | A file, link, or content entry attached to a Live Session and visible inside the classroom. | | **Live Session Event** | An append-only event emitted during a Live Session (join, leave, screen-share start, recording start, etc.). | -| **Live Recording** | Metadata for a recording produced by the provider's egress pipeline. The file lives in MinIO / S3; LearnStack stores the metadata and consent state. | +| **Live Recording** | Metadata for a recording produced by the provider's egress pipeline. The file lives in SeaweedFS / S3; LearnStack stores the metadata and consent state. | > **Cohort vs. Classroom vs. Live Session.** Cohort is a *group of people*. Live Session is a *scheduled event*. Live Room is the *runtime artifact* of a Live Session. Earlier drafts used `Classroom` for both group and runtime; the term `Classroom` is deprecated in favor of the explicit `Cohort` / `Live Session` / `Live Room` split. @@ -194,7 +194,7 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | **Right to Erasure** | The user's right to have their personal data deleted, subject to retention exceptions (legal hold, financial records). | | **Anonymisation** | Replacement of PII fields with pseudonymous values; row stays for analytics / audit integrity. Distinct from soft delete and hard delete. | | **Consent Record** | An append-only per-purpose record (terms of service, recording, marketing). "Changing one's mind" creates a new record, never edits an old one. | -| **Sub-processor** | A third-party service LearnStack uses to process tenant data (Keycloak, LiveKit, S3 / MinIO, email provider, ...). Changes require 30-day tenant notice. | +| **Sub-processor** | A third-party service LearnStack uses to process tenant data (Keycloak, LiveKit, S3 / SeaweedFS, email provider, ...). Changes require 30-day tenant notice. | ## Audit @@ -245,9 +245,9 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | Term | Definition | |------|------------| -| **Dapr Building Blocks** | The three Dapr abstractions LearnStack uses: pub/sub (Kafka), state (Redis), secrets (Vault) per [ADR-0014](decisions/0014-adopt-dapr.md). Service invocation, workflow, bindings, and actors are out of scope. | +| **Dapr Building Blocks** | The three Dapr abstractions LearnStack uses: pub/sub (Kafka), state (Valkey), secrets (Vault) per [ADR-0014](decisions/0014-adopt-dapr.md). Service invocation, workflow, bindings, and actors are out of scope. | | **`IEventBus`** | Interface for publishing integration events. Backed by `DaprEventBus` (production) or `InProcessEventBus` (development). The `OutboxProcessor` is the only sanctioned caller. | -| **`ICacheService`** | Interface for cache reads / writes. Backed by `DaprCacheService` (production, Redis-backed) or `InMemoryCacheService` (development). Cache keys carry `{tenant_id}` prefix. | +| **`ICacheService`** | Interface for cache reads / writes. Backed by `DaprCacheService` (production, Valkey-backed) or `InMemoryCacheService` (development). Cache keys carry `{tenant_id}` prefix. | | **`ISecretProvider`** | Interface for secret reads. Backed by `DaprSecretProvider` (production, Vault) or `EnvironmentSecretProvider` (development). Secret namespace `learnstack/{deployment}/{module}/{key}`. | | **`IEntitlementProvider`** | Interface for the entitlement source. Implementations: `NullEntitlementProvider` (dev), `HubEntitlementProvider` (SaaS / Dedicated), `SignedLicenseKeyEntitlementProvider` (Self-Hosted). | | **`IHostToTenantResolver`** | Interface for host → `(tenant_id, organization_id?)` resolution. Backed by `platform_host_to_tenant`. | diff --git a/docs/roadmap/phase-00-product-architecture.md b/docs/roadmap/phase-00-product-architecture.md index af72de9..0fa8772 100644 --- a/docs/roadmap/phase-00-product-architecture.md +++ b/docs/roadmap/phase-00-product-architecture.md @@ -70,7 +70,7 @@ later should be made explicit before implementation begins. - .NET 10 backend. - EF Core and PostgreSQL. -- Redis and MinIO. +- Valkey and SeaweedFS. - Next.js frontend. - Modular monolith. - Shared-database multi-tenancy for the initial implementation. diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 627b72d..0d07985 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -21,20 +21,32 @@ > `pnpm-lock.yaml` committed; `postinstall` hook stubs `.next/types/routes.d.ts`. > > **Packet 3 — Core dev compose ✅** -> `infra/compose/dev.yml` with PostgreSQL 16, Redis 7, MinIO + console, Mailpit +> `infra/compose/dev.yml` with PostgreSQL 18, Valkey 7, SeaweedFS + console, Mailpit > (binary `readyz` healthcheck), Meilisearch — pinned tags, healthchecks, > named volumes, dev-only credential banners. > -> **Packet 4 — Identity stack (pending)** -> Keycloak two realms (`learnstack` + `learnstack-hub`) in dev compose, seed -> realm config. +> **Packet 4 — Identity stack ✅** +> Keycloak 26 in dev compose with two realms imported on first boot +> (`learnstack` tenant-facing + `learnstack-hub` operator), each hard-isolated +> per ADR-0004 Amendment 1. Realm seeds at `infra/keycloak/realms/`; Postgres +> init script provisions the Keycloak DB on the first start of the +> `postgres-data` volume. > -> **Packet 5 — Live media (pending)** -> LiveKit OSS + Coturn in dev compose. +> **Packet 5 — Live media ✅** +> LiveKit OSS v1.8.0 + Coturn 4.6 in dev compose with the dev key/secret pair +> the eventual `ILiveClassProvider` adapter (Phase 08c) will sign tokens with. +> Configs at `infra/livekit/livekit.yaml` and `infra/coturn/turnserver.conf`. > -> **Packet 6 — Eventing + secrets + gateway (pending)** -> Kafka + kafka-ui, Vault (dev mode), Dapr sidecar + placement, APISIX -> standalone YAML-reload + dashboard, `infra/apisix/config.yaml`. +> **Packet 6 — Eventing + secrets + gateway ✅** +> Kafka 7.8 in KRaft mode (no ZooKeeper) + kafka-ui, Vault 1.18 in `-dev` mode, +> Dapr 1.14.4 sidecar (`learnstack-api` app id, `-app-channel-address +> host.docker.internal` so subscriptions reach the workstation) + placement +> with three components — `pubsub-kafka.yaml`, `statestore-redis.yaml` +> (`actorStateStore: false` per ADR-0014 non-goals), `secretstore-vault.yaml` +> — and APISIX 3.10 in file-driven standalone mode (`deployment.role: +> data_plane`, no Admin API, no companion dashboard) per ADR-0015. The +> `/api/internal/*` Phase-02c surface is documented as an SSL-object + +> ip-restriction stub (mTLS in APISIX is not a route-level plugin). > > **Packet 7 — Developer experience (pending)** > `Makefile` (`make dev` / `test` / `lint` / `seed`), `.env.example` per app, @@ -147,9 +159,9 @@ in the `learnstack-hub` repository* per Docker Compose under `infra/compose/`: -- PostgreSQL 16. -- Redis 7. -- MinIO + MinIO console. +- PostgreSQL 18. +- Valkey 7. +- SeaweedFS + SeaweedFS console. - Mailpit (outbound email). - Meilisearch. - LiveKit OSS + Coturn (for in-app classroom development). @@ -160,7 +172,7 @@ Docker Compose under `infra/compose/`: - **Kafka** (Dapr pub/sub backend) + kafka-ui. - **Vault** (Dapr secret store, dev mode). - **Dapr sidecar** + placement service. -- **APISIX** (standalone YAML-reload mode) + apisix-dashboard (dev only). +- **APISIX** (file-driven standalone `data_plane` mode per ADR-0015 — no etcd, no Admin API, no dashboard companion). - Optional Jaeger or Tempo (for trace inspection). - Optional `learnstack-hub` compose overlay for local Hub development (depends on the same Keycloak / Postgres / Kafka / Vault / APISIX stack). @@ -192,7 +204,7 @@ Two compose files: - Working backend solution scaffolded with modular layout. - Working frontend workspace with the single Next.js app. -- Local Docker Compose infrastructure with PostgreSQL, Redis, MinIO, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. +- Local Docker Compose infrastructure with PostgreSQL, Valkey, SeaweedFS, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. - Initial CI pipeline. - Local development documentation. - `make seed` populating two demo tenants + one platform admin user. @@ -201,7 +213,7 @@ Two compose files: - A new developer can clone the repository and start the local environment by following one document. - Backend API responds on `GET /healthz`. -- PostgreSQL, Redis, MinIO, LiveKit, Coturn, Keycloak all run locally via compose. +- PostgreSQL, Valkey, SeaweedFS, LiveKit, Coturn, Keycloak all run locally via compose. - Frontend builds and serves the three route segments. - CI passes on `main`. - The architecture-test project is set up and green even before domain features exist. diff --git a/docs/roadmap/phase-02a-kernel-tenancy.md b/docs/roadmap/phase-02a-kernel-tenancy.md index 5c3f61c..8374300 100644 --- a/docs/roadmap/phase-02a-kernel-tenancy.md +++ b/docs/roadmap/phase-02a-kernel-tenancy.md @@ -59,7 +59,7 @@ They are codified in: Per [ADR-0014](../decisions/0014-adopt-dapr.md): - Dapr sidecar runs in dev `docker-compose.yml`. Pub/sub component → Kafka. State - component → Redis. Secrets component → Vault (dev mode). + component → Valkey. Secrets component → Vault (dev mode). - `DaprEventBus`, `DaprCacheService`, `DaprSecretProvider` implementations ship in `LearnStack.Infrastructure`. - Topic naming convention enforced by `Dapr_PubSub_TopicNames_FollowConvention` @@ -74,8 +74,11 @@ Per [ADR-0015](../decisions/0015-api-gateway-apisix.md): - APISIX runs in standalone YAML-reload mode in dev `docker-compose.yml`. - `infra/apisix/config.yaml` ships with the plugin chain wired: `cors` → `jwt-auth` → `limit-req` → `proxy-rewrite` → `prometheus`. -- A second route set guarded by `mtls` for the future `/api/internal/*` endpoints - (the endpoints themselves arrive in 02c but the gateway slot is reserved Day 1). +- A second route set bound to a dedicated SSL object (mTLS in APISIX is SSL-object + config — `client.ca` / `client.depth` — not a route plugin) plus an `ip-restriction` + on the Hub egress range, reserved Day 1 for the future `/api/internal/*` endpoints + (the endpoints themselves arrive in 02c). The commented stub at the bottom of + `infra/apisix/apisix.yaml` documents the canonical shape. ### Tenancy Schema Foundations @@ -219,7 +222,7 @@ The architecture test project starts going green during this phase. Phase 02a co - `Dapr_PubSub_TopicNames_FollowConvention`. - `AuditEntry_Inherits_Entity_Not_AuditableEntity`. - `LearnStack_Modules_DoNotReference_Hub`. -- `Modules_Do_Not_Inject_Redis_Directly`, `Modules_Do_Not_Read_Entitlement_Cache_Directly`, +- `Modules_Do_Not_Inject_Valkey_Directly`, `Modules_Do_Not_Read_Entitlement_Cache_Directly`, `Modules_Do_Not_Write_AuditLog_Directly`. - `Modules_Do_Not_Reference_DeploymentMode` — modules never read `DeploymentMode` directly; the composition root selects provider implementations once. See diff --git a/docs/roadmap/phase-04-cms-media-pages.md b/docs/roadmap/phase-04-cms-media-pages.md index f9456c1..3f7a85b 100644 --- a/docs/roadmap/phase-04-cms-media-pages.md +++ b/docs/roadmap/phase-04-cms-media-pages.md @@ -94,7 +94,7 @@ The block system ships with: ### Media Library -- MinIO upload. +- SeaweedFS upload. - Asset metadata. - Folder and tag organization. - Image dimensions. diff --git a/docs/roadmap/phase-08c-classroom.md b/docs/roadmap/phase-08c-classroom.md index 493d45c..3afe532 100644 --- a/docs/roadmap/phase-08c-classroom.md +++ b/docs/roadmap/phase-08c-classroom.md @@ -62,7 +62,7 @@ Recording is **opt-in**, off by default per tenant. The consent flow is part of - Per-participant consent state captured before the room is joined; absence blocks the join. - Recording metadata persisted: storage key, duration, consent state, retention deadline. - Recording can be configured at policy / metadata level even when execution is disabled — for tenants that want the audit trail without the bandwidth. -- LiveKit Egress writes the recording to S3 / MinIO; LearnStack does not transcode in this phase. +- LiveKit Egress writes the recording to S3 / SeaweedFS; LearnStack does not transcode in this phase. ### Provider Webhooks @@ -104,7 +104,7 @@ Failures at any layer produce a Problem Details response with a specific `code` - Cross-tenant join attempt (identity `{tenantA}:{userId}` against a room owned by tenant B) is rejected. - Attendance reflects participant join/leave streams correctly across reconnections. - Recording cannot start without consent; consent state is recorded in `LiveRecording`. -- Recording, when enabled, writes to S3 / MinIO with the correct key prefix. +- Recording, when enabled, writes to S3 / SeaweedFS with the correct key prefix. - LiveKit provider webhook handler is signature-verified, idempotent, and tenant-scoped. ## Risks diff --git a/docs/standards/00-principles.md b/docs/standards/00-principles.md index bc98e56..6a767e6 100644 --- a/docs/standards/00-principles.md +++ b/docs/standards/00-principles.md @@ -39,8 +39,8 @@ The modular monolith works only because modules pretend to be services. Cross-mo Anything that crosses the LearnStack boundary — payments, email, SMS, search, storage, identity, live-class media, the **Hub**, **entitlement source**, **host→tenant resolution**, **event bus**, **cache**, **secret store** — lives behind an interface. -The domain code knows nothing about Stripe, Postmark, MinIO, Keycloak, LiveKit, Dapr, -Kafka, Redis, Vault, or the Hub. Provider-specific code lives in +The domain code knows nothing about Stripe, Postmark, SeaweedFS, Keycloak, LiveKit, Dapr, +Kafka, Valkey, Vault, or the Hub. Provider-specific code lives in `Infrastructure.` packages. Swapping a provider is a composition-root edit, not a code change. @@ -66,7 +66,7 @@ A failing test should tell a reader what behavior we promised. Test names read l ## 9. Default to Boring -We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Redis, MinIO, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. +We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Valkey, SeaweedFS, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. ## 10. Cost-Aware From Day One diff --git a/docs/standards/05-database.md b/docs/standards/05-database.md index caa74e6..70a97e8 100644 --- a/docs/standards/05-database.md +++ b/docs/standards/05-database.md @@ -1,19 +1,21 @@ # 05 — Database Standards **Status:** Active -**Derives from:** [ADR-0002 Initial Architecture](../decisions/0002-initial-architecture.md), +**Derives from:** [ADR-0002 Initial Architecture](../decisions/0002-initial-architecture.md) +(Amendments 1 + 2), [ADR-0003 Tenant Isolation Defense in Depth](../decisions/0003-tenant-isolation-defense-in-depth.md) (Amendment 1: Organization Scope), [ADR-0006 Events and Outbox](../decisions/0006-events-and-outbox.md) (Amendment 1: Dapr pub/sub dispatch transport), [ADR-0014 Adopt Dapr](../decisions/0014-adopt-dapr.md), -[ADR-0017 Tenant + Organization Hierarchy](../decisions/0017-tenant-organization-hierarchy.md). +[ADR-0017 Tenant + Organization Hierarchy](../decisions/0017-tenant-organization-hierarchy.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). PostgreSQL schema, EF Core, and migration conventions. ## Database -- **PostgreSQL 16+** in all environments. +- **PostgreSQL 18+** in all environments. - One database per environment; single schema (`public`). - One `DbContext` per module (not one global). - Schema migrations live with the owning module. diff --git a/docs/standards/06-testing.md b/docs/standards/06-testing.md index dbf34ea..fbe9895 100644 --- a/docs/standards/06-testing.md +++ b/docs/standards/06-testing.md @@ -8,16 +8,27 @@ Test pyramid, conventions, and what every change must cover. ## Test Pyramid ```mermaid +--- +title: LearnStack Test Pyramid +--- flowchart TB e2e[End-to-end / Playwright
handful of golden flows] contract[Contract & API tests
OpenAPI + provider fakes] - integration[Integration tests
Testcontainers Postgres / Redis / MinIO] + integration[Integration tests
Testcontainers Postgres / Valkey / SeaweedFS] arch[Architecture tests
module boundaries + tenant invariants] unit[Unit tests
domain + application + UI logic] unit --> arch --> integration --> contract --> e2e ``` +Text fallback (for renderers without Mermaid support — pyramid base → top): + +- **Unit tests** (base layer, widest) — domain + application + UI logic. +- **Architecture tests** — module boundaries + tenant invariants. +- **Integration tests** — Testcontainers Postgres / Valkey / SeaweedFS. +- **Contract & API tests** — OpenAPI + provider fakes. +- **End-to-end / Playwright** (top, narrowest) — handful of golden flows. + We invest most at **unit + integration**. Architecture tests are zero-flake. E2E covers only what cannot be proven below. ## Backend Test Types @@ -39,7 +50,7 @@ We invest most at **unit + integration**. Architecture tests are zero-flake. E2E ### Integration Tests -- Real Postgres + Redis + MinIO via Testcontainers. +- Real Postgres + Valkey + SeaweedFS via Testcontainers. - One Postgres database per test class (or Respawn between tests). - Real module configuration; no mocked repositories. - Cover happy path, edge cases, and **every tenant-isolation invariant**. diff --git a/docs/standards/09-error-handling.md b/docs/standards/09-error-handling.md index fd624e2..54243bb 100644 --- a/docs/standards/09-error-handling.md +++ b/docs/standards/09-error-handling.md @@ -63,7 +63,7 @@ Standard error codes (machine-readable, stable): ``` LearnStackException (base) ├── DomainException (domain invariant broken from inside, programmer error) -├── InfrastructureException (DB, Redis, MinIO transient) +├── InfrastructureException (DB, Valkey, SeaweedFS transient) ├── ProviderException (upstream provider error) │ ├── PaymentProviderException │ ├── LiveClassProviderException diff --git a/docs/standards/10-observability.md b/docs/standards/10-observability.md index ee16cd7..8289cd7 100644 --- a/docs/standards/10-observability.md +++ b/docs/standards/10-observability.md @@ -98,8 +98,8 @@ Auto-instrument: - MediatR commands and queries. - Hangfire job invocations. - Outbox dispatcher batches. -- Redis client calls. -- MinIO/S3 SDK calls. +- Valkey client calls. +- SeaweedFS/S3 SDK calls. - LiveKit provider calls. Manual spans: diff --git a/docs/standards/11-security.md b/docs/standards/11-security.md index dfdc7dd..da8f25b 100644 --- a/docs/standards/11-security.md +++ b/docs/standards/11-security.md @@ -127,8 +127,12 @@ layer, not the sole control — the API re-verifies everything. - `cors` plugin handles preflight; authenticated cross-origin traffic is allow-listed per environment. - `limit-req` / `limit-count` plugins enforce the rate-limit policy below. -- `mtls` plugin guards the `/api/internal/*` route set; the client certificate must be - signed by the LearnStack-internal CA. +- The `/api/internal/*` route set is gated by **mTLS configured on the APISIX SSL + object** (`client.ca` + `client.depth` per APISIX 3.x SSL-config; mTLS is not a + route plugin) plus an `ip-restriction` (and, when applicable, `consumer-restriction`) + route plugin that only admits the documented Hub egress; the client certificate + must be signed by the LearnStack-internal CA. The route-level pattern is shown in + the commented `/api/internal/*` stub in `infra/apisix/apisix.yaml`. - Gateway config lives in `infra/apisix/` as YAML, version-controlled. No live edits. Direct ingress to backend pods (bypassing APISIX) is blocked at the network policy @@ -201,7 +205,7 @@ for the full strategy. Standards-side: - Enforce per-content-type size limits (image: 10 MB, document: 50 MB, video: 5 GB). - Strip EXIF where appropriate. - Store in tenant-scoped object storage prefix. -- Never trust the original filename. Generate a server-side key under the canonical tenant prefix: `tenants/{tenantId}/{category}/{uuid}.{ext}` (with `organizations/{orgId}/` segment for org-scoped assets), per [09-tenant-isolation.md § Storage (MinIO)](../architecture/09-tenant-isolation.md) and [16-media-pipeline.md § Key Layout](../architecture/16-media-pipeline.md). +- Never trust the original filename. Generate a server-side key under the canonical tenant prefix: `tenants/{tenantId}/{category}/{uuid}.{ext}` (with `organizations/{orgId}/` segment for org-scoped assets), per [09-tenant-isolation.md § Storage (SeaweedFS)](../architecture/09-tenant-isolation.md) and [16-media-pipeline.md § Key Layout](../architecture/16-media-pipeline.md). - Virus scan hook (ClamAV or cloud equivalent) before files become accessible. - Signed URLs for private files; TTL ≤ 1 hour. diff --git a/docs/standards/12-infrastructure.md b/docs/standards/12-infrastructure.md index 0f7cfe4..efa5173 100644 --- a/docs/standards/12-infrastructure.md +++ b/docs/standards/12-infrastructure.md @@ -6,7 +6,10 @@ [ADR-0014 Adopt Dapr](../decisions/0014-adopt-dapr.md), [ADR-0015 API Gateway: APISIX](../decisions/0015-api-gateway-apisix.md), [ADR-0019 LearnStack Hub](../decisions/0019-learnstack-hub.md), -[ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md). +[ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md), +[ADR-0029 Object Storage — SeaweedFS](../decisions/0029-object-storage-seaweedfs.md), +[ADR-0030 Redis-Compatible Store — Valkey](../decisions/0030-redis-compatible-store-valkey.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). How LearnStack is built, packaged, deployed, configured, and observed at the infrastructure level. The application-code rules for the foundation building blocks @@ -49,24 +52,30 @@ adapter table. ## Local Infrastructure (Docker Compose) +Shipped in Phase 01 packets 1-6 (`infra/compose/dev.yml`): + ``` -postgres -redis -minio + minio-console +postgres # PostgreSQL 18.x per ADR-0031 +valkey # Linux-Foundation BSD-3 fork of Redis 7.2.4 per ADR-0030 +seaweedfs # single dev binary: master + volume + filer + S3 gateway per ADR-0029 meilisearch +mailpit keycloak # two realms: learnstack + learnstack-hub -livekit-server -livekit-egress -coturn -mailhog -otel-collector - +livekit # SFU (livekit-server image) +coturn # TURN/STUN +kafka # KRaft mode (no ZooKeeper) — Dapr pub/sub backend +kafka-ui # ghcr.io/kafbat fork (dev only) +vault # Dapr secrets backend (dev mode) dapr-placement # Dapr building blocks dapr-sidecar-api # one sidecar per backend service -kafka + kafka-ui # Dapr pub/sub backend -vault # Dapr secrets backend (dev mode) -apisix # gateway in standalone YAML-reload mode -apisix-dashboard # optional, dev only +apisix # gateway in file-driven standalone (data_plane) mode — no etcd, no Admin API, no dashboard companion +``` + +Deferred — added by a later phase, not in the Phase 01 stack: + +``` +livekit-egress # Phase 08c (recording / consent / cost model) +otel-collector # Phase 11 (Production hardening — observability stack) ``` - Application projects run **outside** containers during active development; the Dapr @@ -136,7 +145,7 @@ Rules: ## Object Storage Operations -- MinIO local; S3-compatible cloud storage in production. +- SeaweedFS local; S3-compatible cloud storage in production. - One bucket per environment; tenant isolation enforced by key prefix (`{tenant_id}/...`). Bucket-per-tenant is not used. See [Media Pipeline § Key Layout](../architecture/16-media-pipeline.md) and [Tenant Isolation](../architecture/09-tenant-isolation.md). - Lifecycle policies for recording retention. - Cross-region replication for production buckets (optional, behind ADR). @@ -174,10 +183,13 @@ See [10-observability.md](10-observability.md). - TLS everywhere; HTTP → HTTPS redirect at the edge. - **APISIX is the only tenant-facing ingress** ([ADR-0015](../decisions/0015-api-gateway-apisix.md)). Direct ingress to backend pods is blocked by network policy. -- The `/api/internal/*` route set is reachable only through a separate APISIX - route guarded by the `mtls` plugin, with the LearnStack-internal CA pinned. +- The `/api/internal/*` route set is reachable only through an APISIX route bound + to an SSL object that pins the LearnStack-internal CA via `client.ca` / + `client.depth` (mTLS in APISIX is SSL-object config, not a route plugin), plus a + route-level `ip-restriction` constraint on the Hub egress range. See the commented + `/api/internal/*` stub in `infra/apisix/apisix.yaml` for the canonical shape. - Strict ingress rules; only documented ports open. -- Private VPC for backend services; database, Redis, Kafka, Vault not on public +- Private VPC for backend services; database, Valkey, Kafka, Vault not on public internet. - Outbound calls allow-listed where feasible. Hub outbound traffic is allow-listed per environment. @@ -190,11 +202,11 @@ See [10-observability.md](10-observability.md). | Dapr sidecar (per API/worker pod) | 0.25 vCPU | 256 MB | runs alongside each pod | | Workers (per instance) | 1 vCPU | 2 GB | autoscale 1–4 | | Postgres | 4 vCPU | 16 GB | initial; scale as needed | -| Redis | 1 vCPU | 2 GB | initial | +| Valkey | 1 vCPU | 2 GB | initial | | Kafka (per broker) | 2 vCPU | 4 GB | 3-broker cluster baseline | | Vault | 1 vCPU | 1 GB | HA mode in production (3 nodes) | | APISIX | 1 vCPU | 1 GB | autoscale 2–4 | -| MinIO | 2 vCPU | 4 GB | scaled by storage tier | +| SeaweedFS | 2 vCPU | 4 GB | scaled by storage tier | | LiveKit SFU | 2 vCPU | 4 GB | per 250 concurrent participants | | LiveKit Egress | 2 vCPU | 4 GB | per ~1.5 concurrent recordings | | coturn | 1 vCPU | 1 GB | bandwidth-bound | diff --git a/docs/standards/15-performance.md b/docs/standards/15-performance.md index 672e95b..f0c5678 100644 --- a/docs/standards/15-performance.md +++ b/docs/standards/15-performance.md @@ -64,7 +64,7 @@ Budgets are reviewed quarterly against measured production metrics. ### Memory - Avoid loading whole result sets when streaming would do. -- Stream large file uploads to MinIO/S3; never buffer the whole file in memory. +- Stream large file uploads to SeaweedFS/S3; never buffer the whole file in memory. - Avoid string concatenation in tight loops; use `StringBuilder` or pooled buffers. ## Frontend Rules diff --git a/docs/standards/20-infrastructure-stack.md b/docs/standards/20-infrastructure-stack.md index 694e04c..3975515 100644 --- a/docs/standards/20-infrastructure-stack.md +++ b/docs/standards/20-infrastructure-stack.md @@ -5,7 +5,10 @@ [ADR-0015 API Gateway: APISIX](../decisions/0015-api-gateway-apisix.md), [ADR-0019 LearnStack Hub](../decisions/0019-learnstack-hub.md), [ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md), -[ADR-0021 Feature-Based Entitlement](../decisions/0021-feature-based-entitlement.md). +[ADR-0021 Feature-Based Entitlement](../decisions/0021-feature-based-entitlement.md), +[ADR-0029 Object Storage — SeaweedFS](../decisions/0029-object-storage-seaweedfs.md), +[ADR-0030 Redis-compatible Store — Valkey](../decisions/0030-redis-compatible-store-valkey.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). This standard defines how application code uses the foundation infrastructure introduced in the 2026-05-18 redesign: Dapr building blocks, the APISIX gateway, the Hub HTTPS @@ -49,7 +52,7 @@ Rules: | Concern | `Development` | `SaaS` | `Dedicated` | `SelfHostedOnline` | `SelfHostedAirGapped` | |---|---|---|---|---|---| | Event bus | `InProcessEventBus` (MediatR) | `DaprEventBus` → Kafka | `DaprEventBus` → Kafka | `DaprEventBus` → Kafka (single-broker OK) | `DaprEventBus` → Kafka (single-broker OK) | -| Cache | `InMemoryCacheService` | `DaprCacheService` → Redis | `DaprCacheService` → Redis | `DaprCacheService` → Redis | `DaprCacheService` → Redis | +| Cache | `InMemoryCacheService` | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | | Secrets | `EnvironmentSecretProvider` | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault or file | | Entitlement | `NullEntitlementProvider` | `HubEntitlementProvider` | `HubEntitlementProvider` | `HubEntitlementProvider` (phone-home) | `SignedLicenseKeyEntitlementProvider` | | Host → tenant | Config / single tenant | Hub-mirrored projection | Hub-mirrored projection | Hub-mirrored projection | Config / `.lic` claim | @@ -80,9 +83,9 @@ ADR-0014 non-goals; do not introduce them without a new ADR. ### `ICacheService` (state) -- All Redis access goes through `ICacheService`. Direct `IConnectionMultiplexer` / +- All Valkey access goes through `ICacheService`. Direct `IConnectionMultiplexer` / `IDistributedCache` injections are forbidden by the architecture test - `Modules_Do_Not_Inject_Redis_Directly`. + `Modules_Do_Not_Inject_Valkey_Directly`. - Cache keys are `{tenant_id}:{module}:{logical-name}`. The `tenant_id` prefix is **mandatory** even when a value is platform-wide — use the sentinel `"platform"` tenant id rather than omitting the prefix. @@ -99,7 +102,7 @@ The most-referenced read paths follow this layered policy; mismatches across doc (e.g. "60s cache" vs "15-min TTL") refer to different layers of the same cache, not different decisions: -| Key family | L1 (in-process `IMemoryCache`) | L2 (Dapr state → Redis) | Eager invalidation event | +| Key family | L1 (in-process `IMemoryCache`) | L2 (Dapr state → Valkey) | Eager invalidation event | |---|---|---|---| | `hub:host:{host}` (host → tenant) | 2 min | 15 min | `learnstack.hub.custom-domain.activated/.deactivated` | | `hub:entitlement:{tenant_id}` (plan projection) | 60 s | 15 min (upper bound; Hub-push refresh resets it) | `learnstack.hub.entitlement` | @@ -142,9 +145,12 @@ Rules: 4. `proxy-rewrite` / `request-id` (correlation id injection) 5. `prometheus` (metrics export) - Hub-facing internal routes (`/api/internal/*`) live under a **separate APISIX - instance** (or a separate `route` set with `mtls` plugin) and require the mTLS client - certificate signed by the LearnStack-internal CA per - [ADR-0019](../decisions/0019-learnstack-hub.md). + instance** (or a separate route set bound to a dedicated SSL object that pins + `client.ca` to the LearnStack-internal CA — mTLS in APISIX is SSL-object config, + not a route plugin) plus a route-level `ip-restriction` for the Hub egress range; + the client certificate must be signed by that CA per + [ADR-0019](../decisions/0019-learnstack-hub.md). The commented `/api/internal/*` + stub in `infra/apisix/apisix.yaml` documents the canonical shape. - Gateway config lives in `infra/apisix/` as version-controlled YAML. Hot-reload via `apisix reload` after a config change; no in-place edit of running configs. @@ -191,8 +197,10 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) - `IFeatureFlags.IsEnabledAsync(FeatureKey)` is the only sanctioned read path. Direct SQL against `platform_entitlement_cache` outside the Tenancy module's infrastructure is forbidden (architecture test `Modules_Do_Not_Read_Entitlement_Cache_Directly`). -- Cache TTL for the in-process / Redis layer is 60s. Eager invalidation flows from the - Dapr event; the TTL is the safety net, not the typical refresh window. +- Cache TTLs: **L1 (in-process `IMemoryCache`)** = 60s; **L2 (Dapr state → Valkey)** = + 15-minute upper bound. Eager invalidation flows from the Dapr event + (`learnstack.hub.entitlement` / `learnstack.cache.invalidation`); the TTLs are the + safety net, not the typical refresh window. - For air-gapped deployments, `SignedLicenseKeyEntitlementProvider` reads a signed `.lic` file and runs the same projection write path; the rest of the system is source-agnostic. @@ -234,11 +242,11 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) - LearnStack monolith → APISIX edge → backend pods (1+). - Dapr sidecar runs alongside every backend pod. -- Kafka, Redis, Vault are accessed only via the Dapr sidecar. No direct client +- Kafka, Valkey, Vault are accessed only via the Dapr sidecar. No direct client libraries for these three in application code. -- Postgres is accessed directly (EF Core); Dapr's state-store sits on Redis, not +- Postgres is accessed directly (EF Core); Dapr's state-store sits on Valkey, not Postgres. -- MinIO is accessed via the configured S3-compatible client (no Dapr binding). +- SeaweedFS is accessed via the configured S3-compatible client (no Dapr binding). ## Forbidden diff --git a/infra/apisix/README.md b/infra/apisix/README.md new file mode 100644 index 0000000..378f53a --- /dev/null +++ b/infra/apisix/README.md @@ -0,0 +1,100 @@ +# APISIX Gateway (Dev) + +LearnStack's only tenant-facing ingress, per +[ADR-0015 (API Gateway: APISIX)](../../docs/decisions/0015-api-gateway-apisix.md). +File-driven standalone mode — `apisix.yaml` is the only source of truth, +hot-reloaded on file change. No etcd, no Admin API, no dashboard. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| HTTP gateway | `http://localhost:9080` | Public + authenticated routes (per `apisix.yaml`) | +| HTTPS gateway | `https://localhost:9443` | TLS-terminated routes (no cert in dev) | +| Prometheus metrics | `http://localhost:9091` | Per-route + plugin metrics scraped by Prometheus (Phase 11) | + +### Why no Admin API and no dashboard? + +APISIX 3.2+ moved file-driven standalone behind `deployment.role: +data_plane`, which **does not expose an Admin API**. The companion +`apisix-dashboard` reads routes from etcd; without etcd it cannot show +anything. Both are intentionally out of this compose stack because +ADR-0015 commits to "standalone YAML hot-reload, no etcd dependency." + +`apisix.yaml` is therefore the only place a developer changes routes; +diff-review of that file replaces the dashboard for dev workflow. The +etcd-backed deployment is reconsidered when a real admin-UI requirement +appears, behind its own ADR. + +## Plugin chain + +`infra/apisix/config.yaml` declares the universe of plugins; routes pick +from it per-request order: + +```text +real-ip → cors → openid-connect → limit-req → request-id → proxy-rewrite → upstream + ↓ + prometheus (response) +``` + +**mTLS is NOT a route-level plugin** in APISIX — it is configured on the +SSL/SNI object (`client.ca` / `client.depth`). The Phase 02c +`/api/internal/*` surface enforces mTLS through an `ssls:` entry plus +route-level `ip-restriction`; see the commented stub at the bottom of +`apisix.yaml`. + +## Routes shipped today (`apisix.yaml`) + +| Priority | URI | Methods | Plugin chain | Upstream | +|----------|-----|---------|--------------|----------| +| 1 | `/healthz` | GET | `cors`, `limit-req` (10 r/s), `prometheus` | `host.docker.internal:5080` | +| 99 | `/api/v*/**` | OPTIONS | `cors` only (preflight bypass) | `host.docker.internal:5080` | +| 100 | `/api/v*/**` | GET/POST/PUT/PATCH/DELETE | `cors`, `limit-req` (100 r/s), `request-id`, `prometheus` | `host.docker.internal:5080` | + +`openid-connect` is **commented out** in route 100; Phase 03 wires the +plugin against the `learnstack` Keycloak realm discovery endpoint. The +backend re-validates the JWT (defence in depth — gateway compromise must +not bypass auth). + +The `/api/internal/*` route is a **disabled placeholder** documenting the +SSL-object + ip-restriction shape Phase 02c activates. + +## Upstream addressing — `host.docker.internal:5080` + +The .NET API host runs **outside the container network** during active +dev (`dotnet run` on the developer's workstation). Docker Desktop's +`host.docker.internal` alias resolves to the host loopback; the +`extra_hosts: host.docker.internal:host-gateway` entry on the `apisix` +service (via the `*host-gateway` YAML anchor in `dev.yml`) makes the +alias available on Linux too — **no manual override needed**. + +When the .NET host moves inside compose in a later environment profile, +the upstream nodes shift to `learnstack-api:5080` — same `apisix.yaml`, +single one-line change. + +## Hot-reload + +APISIX watches `apisix.yaml` and re-applies the route table on file change. +A malformed YAML push takes the gateway down, so changes go through CI YAML +validation before they reach the running gateway. + +## Dev posture + +No credentials live in this directory — file-driven standalone mode has +no Admin API to protect, no dashboard to log into. The only sensitive +surface is the SSL cert pair that Phase 11 introduces when TLS lands. + +## What does NOT live here + +- etcd-backed deployment shape — reconsidered behind its own ADR if / + when a UI-driven admin requirement appears. +- `apisix-dashboard` companion — see "Why no Admin API and no dashboard?" + above. +- Production TLS cert (Let's Encrypt via the same adapter family ADR-0022 + picks for custom domains) — Phase 11. +- Hub-side APISIX route block (`hub.learnstack.dev`) — Phase 02c, lives in + the separate `learnstack-hub` repo. +- The .NET `AddJwtBearer` middleware (the backend's defence-in-depth + JWT re-validation) — Phase 02b. +- Hangfire dashboard `/admin/hangfire*` gating — Phase 08a when Hangfire + itself lands. diff --git a/infra/apisix/apisix.yaml b/infra/apisix/apisix.yaml new file mode 100644 index 0000000..b9260a1 --- /dev/null +++ b/infra/apisix/apisix.yaml @@ -0,0 +1,111 @@ +# APISIX route table — hot-reloaded on file change. +# Per ADR-0015 the route table separates public from authenticated, and +# CORS preflight from the authenticated path so OPTIONS bypasses +# openid-connect. +# +# This packet ships the scaffold routes only: +# - `/healthz` (priority 1, public, GET only) — proxies to the .NET API +# - `/api/v*/**` OPTIONS (priority 99, CORS preflight) +# - `/api/v*/**` authenticated (priority 100, GET/POST/PUT/PATCH/DELETE) +# - `/api/internal/*` — Phase 02c surface, NOT a route plugin: mTLS is +# enforced on the SSL/SNI object (see the Phase-02c stub at the bottom) + +routes: + - id: 1 + uri: /healthz + methods: [GET] + priority: 1 + plugins: + cors: {} + limit-req: + rate: 10 + burst: 5 + key: remote_addr + rejected_code: 429 + prometheus: {} + upstream: + type: roundrobin + nodes: + # The .NET API host runs OUTSIDE compose during active dev + # (`dotnet run` on the workstation). `host.docker.internal` is + # Docker Desktop's gateway alias; Linux developers get the + # `host-gateway` mapping via `extra_hosts` on the apisix service + # in dev.yml. Phase 07 (DX packet) sorts the cross-platform + # case alongside `make dev`. + "host.docker.internal:5080": 1 + + - id: 99 + uri: /api/v*/** + methods: [OPTIONS] + priority: 99 + plugins: + cors: {} + upstream: + type: roundrobin + nodes: + "host.docker.internal:5080": 1 + + - id: 100 + uri: /api/v*/** + methods: [GET, POST, PUT, PATCH, DELETE] + priority: 100 + plugins: + cors: {} + # openid-connect realm discovery is wired in Phase 03 against the + # learnstack Keycloak realm. Placeholder kept so the route schema + # validates today. + # openid-connect: + # discovery: http://keycloak:8080/realms/learnstack/.well-known/openid-configuration + # bearer_only: true + # realm: learnstack + limit-req: + rate: 100 + burst: 50 + key: remote_addr + rejected_code: 429 + request-id: + include_in_response: true + prometheus: {} + upstream: + type: roundrobin + nodes: + "host.docker.internal:5080": 1 + + # ---- Phase 02c — Hub internal API surface (RESERVED) -------------------- + # mTLS in APISIX is NOT a route-level plugin. It is enforced on the + # SSL/SNI object via `client.ca` (CA bundle for verifying client certs) + # + `client.depth` (cert chain depth). The route below documents the + # Phase-02c shape; activate by: + # + # 1. Declaring an `ssls:` entry with the LearnStack-internal CA bundle + # attached to the host (or sni) that fronts `/api/internal/*`. + # 2. Restricting the route to that host (or to an internal IP range) + # via the `ip-restriction` plugin and / or a host match. + # 3. Routing only authenticated mTLS client certs through here; the + # upstream re-validates the signed JWT + HMAC body signature + # (ADR-0019 § Closed 4-endpoint internal API). + # + # - id: 50 + # uri: /api/internal/* + # host: internal.learnstack.dev + # priority: 50 + # plugins: + # ip-restriction: + # whitelist: + # - 10.0.0.0/8 + # prometheus: {} + # upstream: + # type: roundrobin + # nodes: + # "host.docker.internal:5080": 1 + # + # ssls: + # - id: 1 + # sni: internal.learnstack.dev + # cert: "" + # key: "" + # client: + # ca: "" + # depth: 2 + +#END diff --git a/infra/apisix/config.yaml b/infra/apisix/config.yaml new file mode 100644 index 0000000..b2247ec --- /dev/null +++ b/infra/apisix/config.yaml @@ -0,0 +1,54 @@ +# APISIX main config — file-driven standalone mode. +# Per ADR-0015 the standalone deployment shape (no etcd) is the default; +# etcd-backed mode is reconsidered when an admin UI / dynamic config +# requirement arrives. APISIX 3.2+ moved file-driven standalone behind +# `deployment.role: data_plane` + `role_data_plane.config_provider: yaml`; +# the older `traditional + yaml` shape is removed in 3.10. The Admin API +# is NOT available in this mode — `apisix.yaml` is the only source of +# truth, hot-reloaded on file change. + +deployment: + role: data_plane + role_data_plane: + config_provider: yaml + +apisix: + enable_ipv6: false + proxy_mode: http + enable_dev_mode: false + +nginx_config: + error_log_level: info + worker_processes: auto + worker_rlimit_nofile: 20480 + +# Plugin chain available to routes. The actual per-route chain is declared +# in apisix.yaml; this list is the universe routes may pick from. +# +# NOTE: mTLS is NOT a route-level plugin in APISIX — it is configured on +# the SSL/SNI object (`client.ca` / `client.depth`) per the upstream docs. +# The Phase-02c `/api/internal/*` surface enforces mTLS via the SSL object +# + `ip-restriction` on the route; the plugin universe below intentionally +# omits any `mtls` entry. +plugins: + - real-ip + - cors + - openid-connect # JWT validation (Phase 03 wires Keycloak realm) + - jwt-auth + - limit-req + - limit-count + - request-id + - proxy-rewrite + - response-rewrite + - prometheus + - ip-restriction + - gzip + +plugin_attr: + prometheus: + enable_export_server: true + export_addr: + ip: "0.0.0.0" + port: 9091 + request-id: + include_in_response: true diff --git a/infra/compose/README.md b/infra/compose/README.md index 4179bbd..2965e44 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -3,17 +3,96 @@ Compose stacks for local development. Operational rules live in [docs/standards/12-infrastructure.md](../../docs/standards/12-infrastructure.md). -## `dev.yml` (this packet) +## `dev.yml` -Core data-plane services every backend dev needs from day one: +Services in order they appear in `dev.yml` (data plane → identity → media → +eventing → secrets → Dapr sidecar → gateway). Packets 1-6 shipped; packets +7-8 (DX orchestrator + CI) remain. + +### Data plane (Phase 01 packet 3) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| PostgreSQL 18 | `postgres:18.4-alpine` | `localhost:5432` | `learnstack` / `learnstack` | +| Valkey 8 | `valkey/valkey:8.1-alpine` | `localhost:6379` | — | +| SeaweedFS | `chrislusf/seaweedfs:3.94` | `localhost:9000` (S3), `localhost:9001` (filer UI), `localhost:9333` (master) | S3 access `learnstack` / secret `learnstack-dev-secret` | +| Mailpit | `axllent/mailpit:v1.29.7` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | +| Meilisearch | `getmeili/meilisearch:v1.44.0` | `localhost:7700` | master key `learnstack-dev-master-key` | + +### Identity (Phase 01 packet 4) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| Keycloak | `quay.io/keycloak/keycloak:26.6.2` | `localhost:8080` | master admin `admin` / `admin-dev-secret` | + +Two realms imported on first boot from `infra/keycloak/realms/`: + +- `learnstack` (tenant users) — clients `learnstack-api` (confidential) + `learnstack-web` (public PKCE); demo users `demo-admin@tenant-a.test` and `demo-learner@tenant-a.test` (both `demo-dev-secret`). +- `learnstack-hub` (operators) — client `learnstack-hub-web` (public PKCE); demo user `demo-operator@learnstack.test` (`demo-dev-secret`). `CONFIGURE_TOTP` required-action present so the MFA flow surfaces in dev. + +See [../keycloak/README.md](../keycloak/README.md) for the realm-isolation +invariant, re-seed procedure, and the Phase 02b/03 wiring notes. + +The Postgres init script at `postgres-init/01-create-keycloak-db.sql` creates +the `keycloak` database on the first start of the `postgres-data` volume. +Re-seeding the realm structure requires either `down -v` (wipes all volumes) +or a manual `DROP DATABASE keycloak; CREATE DATABASE keycloak OWNER learnstack;`. + +### Live media (Phase 01 packet 5) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| LiveKit OSS | `livekit/livekit-server:v1.12.0` | `ws://localhost:7880` (signaling), `tcp/7881` (TCP fallback), `tcp/7882` (TURN/TLS), `udp/50000-50100` (media) | API key `devkey` / secret `devsecret-32-byte-min-length-padding-xyz` | +| Coturn | `coturn/coturn:4.11.0` | `udp+tcp/3478` (STUN/TURN), `tcp/5349` (TURN/TLS), `udp/49152-49200` (relay range) | TURN user `devuser` / password `devsecret` | + +LiveKit config at `infra/livekit/livekit.yaml`; Coturn config at +`infra/coturn/turnserver.conf`. See [../livekit/README.md](../livekit/README.md) +for the `ILiveClassProvider` integration plan (Phase 08c) + the +recording / consent / cost-tracking story. + +### Eventing + secrets + Dapr sidecar + gateway (Phase 01 packet 6) | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| PostgreSQL 16 | `postgres:16.6-alpine` | `localhost:5432` | `learnstack` / `learnstack` | -| Redis 7 | `redis:7.4-alpine` | `localhost:6379` | — | -| MinIO | `minio/minio:RELEASE.2025-01-20T14-49-07Z` | `localhost:9000` (S3), `localhost:9001` (console) | `learnstack` / `learnstack-dev-secret` | -| Mailpit | `axllent/mailpit:v1.21` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | -| Meilisearch | `getmeili/meilisearch:v1.11` | `localhost:7700` | master key `learnstack-dev-master-key` | +| Kafka (KRaft) | `confluentinc/cp-kafka:8.2.1` | `localhost:9092` (in-cluster only — see note below) | none (`PLAINTEXT`, `authType: none`) | +| kafka-ui | `ghcr.io/kafbat/kafka-ui:v1.5.0` | `localhost:8081` | open UI (dev only) | +| Vault | `hashicorp/vault:1.21.4` | `localhost:8200` | root token `learnstack-dev-root-token` | +| Dapr placement | `daprio/placement:1.17.7` | `localhost:50005` | — | +| Dapr sidecar (api) | `daprio/daprd:1.17.7` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | +| APISIX | `apache/apisix:3.16.0-debian` | `localhost:9080` (HTTP), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | none (file-driven standalone — no Admin API) | + +Configs: + +- **Kafka** runs in KRaft mode (no ZooKeeper); cluster id is pinned so the + log dir survives restarts without re-format. Only the in-cluster + `PLAINTEXT://kafka:9092` listener is advertised; host-side tools (kcat, + kafka-topics from the workstation) will resolve the bootstrap address as + `kafka:9092` and fail unless `127.0.0.1 kafka` is added to `/etc/hosts`. + Use `kafka-ui` (`localhost:8081`) for workstation-side browsing; the + Phase 07 DX packet ships either an EXTERNAL listener or documents the + `kafka-ui`-only workflow as canonical. +- **Vault** runs in `-dev` mode with the root token baked in — production + runs HA + auto-unseal + AppRole. +- **Dapr** components live under `infra/dapr/components/` + (`pubsub-kafka.yaml`, `statestore-redis.yaml`, `secretstore-vault.yaml`); + runtime config at `infra/dapr/config/dapr-config.yaml`. The sidecar is + wired to call back to `host.docker.internal:5080` via the + `-app-channel-address` flag (daprd's default `127.0.0.1` would resolve + inside the sidecar container and break subscription deliveries). See + [../dapr/README.md](../dapr/README.md) for the + `IEventBus` / `ICacheService` / `ISecretProvider` consumption pattern. +- **APISIX** runs in file-driven standalone mode (`deployment.role: + data_plane`) per ADR-0015 — no etcd, no Admin API, no companion + dashboard. Main config at `infra/apisix/config.yaml`; route table at + `infra/apisix/apisix.yaml`. See [../apisix/README.md](../apisix/README.md) + for the plugin chain, route table walk-through, and the + `/api/internal/*` mTLS-via-SSL-object placeholder reserved for Phase 02c. + +The .NET API host runs OUTSIDE the compose network during active dev; both +the Dapr sidecar and APISIX target `host.docker.internal:5080` so they can +reach the workstation-local `dotnet run` process. The +`host.docker.internal:host-gateway` alias is wired via a YAML anchor in +`dev.yml` so Linux developers don't need a manual override. ```bash docker compose -f infra/compose/dev.yml up -d @@ -29,27 +108,26 @@ The shared credentials above are checked into the repo intentionally — they ar from Vault via `ISecretProvider` (Standards 12 § Secrets Management; Standards 20). Do not reuse these strings anywhere except local Docker. -### Tenant isolation in MinIO +### Tenant isolation in SeaweedFS S3 Tenant isolation in object storage is enforced by **key prefix** (`{tenant_id}/...`), never bucket-per-tenant — see Standards 12 § Object Storage Operations and [docs/architecture/16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). -A single bucket per environment is created at first use. +A single bucket per environment is created at first use by the +`IStorageProvider` adapter. The rule is backend-independent — it +applied to MinIO, it applies to SeaweedFS, it will apply to any future +S3-compatible swap-in. -## What this file deliberately does NOT bring up +See [../seaweedfs/README.md](../seaweedfs/README.md) for the SeaweedFS- +specific dev access surface (filer UI, S3 identity config, re-seed). + +## What this file deliberately does NOT bring up yet Per the [Phase 01 plan](../../docs/roadmap/phase-01-repository-tooling.md), later packets land: -- **Keycloak** (two realms: `learnstack` + `learnstack-hub`) -- **LiveKit OSS** + **Coturn** -- **Kafka** + **kafka-ui** (Dapr pub/sub backend) -- **Vault** (Dapr secret store, dev mode) -- **Dapr sidecar** + **placement service** -- **APISIX** (standalone YAML-reload) + **apisix-dashboard** - -These add their own healthchecks and volumes when they ship; this file stays -small in the interim to keep `make dev` cold-start fast. - -A companion `e2e.yml` (same stack, tuned for end-to-end test runs) is also a -later Phase 01 deliverable. +- `Makefile` (`make dev` / `test` / `lint` / `seed`), `.env.example` per app, + pre-commit hook (dotnet-format + prettier), and `infra/compose/e2e.yml` + companion stack — Phase 01 packet 7. +- GitHub Actions CI workflow + `make seed` populating two demo tenants and a + platform admin — Phase 01 packet 8. diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index d49c60c..8b556b7 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,19 +1,33 @@ -# LearnStack — local dev infrastructure (core services). +# LearnStack — local dev infrastructure. # -# Scope of THIS file (Phase 01 — packet 1-3): the data-plane services every -# backend dev needs from day one. Keycloak, LiveKit OSS, Coturn, Kafka, Vault, -# the Dapr sidecar, and APISIX land in subsequent Phase-01 packets per the -# roadmap (docs/roadmap/phase-01-repository-tooling.md § Local Infrastructure). +# Scope of THIS file (Phase 01 packets 1-6): the data-plane services +# (PostgreSQL 18 / Valkey / SeaweedFS / Mailpit / Meilisearch), the self-hosted Keycloak +# identity provider (two realms), the live-media stack (LiveKit OSS + Coturn), +# Kafka (KRaft) + kafka-ui, HashiCorp Vault (-dev mode), the Dapr sidecar + +# placement (pub/sub + state + secrets building blocks per ADR-0014), and the +# APISIX gateway in file-driven standalone mode per ADR-0015. The DX +# orchestrator (`make` targets, `.env.example`, e2e overlay) and CI workflow +# arrive in Phase-01 packets 7-8. # # Operational rules: Standards 12 § Local Infrastructure + Standards 20. -# Tenant isolation in MinIO is enforced by key prefix (`{tenant_id}/...`) — -# never bucket-per-tenant (Standards 12 § Object Storage Operations). +# Tenant isolation in SeaweedFS S3 is enforced by key prefix +# (`{tenant_id}/...`) — never bucket-per-tenant (Standards 12 § Object +# Storage Operations). name: learnstack-dev +# YAML anchor for the host-gateway alias every container that has to reach +# the workstation-local `dotnet run` process needs. Docker Desktop maps +# `host.docker.internal` automatically; Linux requires this explicit +# `host-gateway` entry. Anchored here so adding it to a new service is a +# `<<: *host-gateway` one-liner. +x-host-gateway: &host-gateway + extra_hosts: + - "host.docker.internal:host-gateway" + services: postgres: - image: postgres:16.6-alpine + image: postgres:18.4-alpine container_name: learnstack-postgres restart: unless-stopped environment: @@ -24,53 +38,103 @@ services: ports: - "5432:5432" volumes: - - postgres-data:/var/lib/postgresql/data + # Postgres 18+ refuses any mount at `/var/lib/postgresql/data` — the + # entrypoint pre-check treats that path's existence (even when empty) + # as a legacy 16/17 layout and aborts (see docker-library/postgres + # PR #1259). The modern layout mounts the parent `/var/lib/postgresql` + # and lets the image place data at `/var/lib/postgresql//docker` + # under PGDATA's default, which is `pg_upgrade --link`-compatible. + - postgres-data:/var/lib/postgresql + # First-boot init scripts (idempotent SQL). Creates the `keycloak` + # database the Keycloak service uses; see the "Identity" section of + # ../compose/README.md for the re-seed procedure. + - ./postgres-init:/docker-entrypoint-initdb.d:ro healthcheck: + # `start_period: 20s` gives the first boot enough room for initdb + + # the `postgres-init/*.sql` scripts to complete before pg_isready + # starts counting failures against the retry budget. Re-boots + # (data already in the volume) skip this — Postgres is ready in <2s. test: ["CMD-SHELL", "pg_isready -U learnstack -d learnstack"] interval: 5s timeout: 5s retries: 10 + start_period: 20s - redis: - image: redis:7.4-alpine - container_name: learnstack-redis + # Valkey replaces Redis per ADR-0030. RESP-protocol drop-in (Dapr + # `state.redis` component, StackExchange.Redis library, ICacheService + # all continue to work unchanged — those names are the protocol / + # library / Dapr provider-type identifier, not the vendor brand). + # The service name is `valkey` so a new reader sees the right backend; + # `redis-cli` continues to ship inside the Valkey image as a symlink. + valkey: + image: valkey/valkey:8.1-alpine + container_name: learnstack-valkey restart: unless-stopped - command: ["redis-server", "--appendonly", "yes"] + command: ["valkey-server", "--appendonly", "yes"] ports: - "6379:6379" volumes: - - redis-data:/data + - valkey-data:/data healthcheck: - test: ["CMD", "redis-cli", "ping"] + test: ["CMD", "valkey-cli", "ping"] interval: 5s timeout: 3s retries: 10 - minio: - image: minio/minio:RELEASE.2025-01-20T14-49-07Z - container_name: learnstack-minio + # SeaweedFS replaces MinIO per ADR-0029. Single dev binary packs master + # + volume + filer + S3 gateway; production splits them per ADR-0029 + # § Implementation Notes. The S3 gateway is port-mapped to the same + # host port (9000) MinIO used so any local script / env var / test + # fixture that hard-coded the S3 endpoint keeps working. + seaweedfs: + image: chrislusf/seaweedfs:3.94 + container_name: learnstack-seaweedfs restart: unless-stopped - command: server /data --console-address ":9001" - environment: - # Dev-only. Production credentials come from Vault via ISecretProvider. - MINIO_ROOT_USER: learnstack - MINIO_ROOT_PASSWORD: learnstack-dev-secret - MINIO_BROWSER_REDIRECT_URL: http://localhost:9001 + labels: + # OrbStack hint: route `seaweedfs.learnstack-dev.orb.local` to the Filer + # UI (8888) rather than picking the lowest EXPOSE'd port (8080 = Volume + # HTTP API root, which returns 400). Use explicit `:9333` / `:8333` for + # master / S3 when needed. + dev.orbstack.http-port: "8888" + command: + - server + - -dir=/data + - -master + - -volume + - -filer + - -s3 + - -s3.port=8333 + - -s3.config=/etc/s3-identities.json + - -metricsPort=9091 ports: - - "9000:9000" # S3 API - - "9001:9001" # console + - "9000:8333" # S3 API gateway (host:9000 matches the prior MinIO endpoint) + - "9001:8888" # Filer UI (replaces the MinIO console) + - "9333:9333" # Master HTTP API + - "8084:8080" # Volume HTTP API (host 8084 to avoid Keycloak's 8080) volumes: - - minio-data:/data + - seaweedfs-data:/data + # Dev-only S3 identity (matches prior MinIO credentials so callers + # don't rebuild env vars). Production loads identities from Vault. + - ../seaweedfs/s3-identities.json:/etc/s3-identities.json:ro healthcheck: - test: ["CMD", "curl", "-fsS", "http://localhost:9000/minio/health/live"] + # `127.0.0.1` (not `localhost`): SeaweedFS image's wget resolves + # `localhost` to `::1` first and does not fall back to IPv4, so a + # `localhost` probe gets connection-refused even though the master + # listens on 127.0.0.1:9333 + the container IP. `/cluster/healthz` + # is the master HTTP API readiness endpoint (SeaweedFS 3.x). + test: ["CMD-SHELL", "wget --spider -q -T 3 -t 1 http://127.0.0.1:9333/cluster/healthz"] interval: 10s timeout: 5s retries: 10 mailpit: - image: axllent/mailpit:v1.21 + image: axllent/mailpit:v1.29.7 container_name: learnstack-mailpit restart: unless-stopped + labels: + # OrbStack hint: route `mailpit.learnstack-dev.orb.local` to the Web UI + # (8025) rather than the SMTP listener (1025), which is not HTTP. + dev.orbstack.http-port: "8025" ports: - "1025:1025" # SMTP - "8025:8025" # web UI @@ -90,7 +154,7 @@ services: retries: 10 meilisearch: - image: getmeili/meilisearch:v1.11 + image: getmeili/meilisearch:v1.44.0 container_name: learnstack-meilisearch restart: unless-stopped environment: @@ -103,13 +167,315 @@ services: volumes: - meilisearch-data:/meili_data healthcheck: - test: ["CMD", "wget", "-qO-", "http://localhost:7700/health"] + # `127.0.0.1` (not `localhost`): the Meilisearch image resolves `localhost` + # to `::1` while the server listens only on IPv4 0.0.0.0:7700, so a + # `localhost` probe gets connection-refused even when the service is up. + test: ["CMD", "wget", "-qO-", "http://127.0.0.1:7700/health"] + interval: 10s + timeout: 5s + retries: 10 + + # ---- Identity stack (Phase 01 packet 4) ---------------------------------- + # Self-hosted Keycloak, two realms (`learnstack` tenant-facing + `learnstack-hub` + # operator). Per ADR-0004 + ADR-0019 the realms are hard-isolated; tokens from + # one realm are never accepted on the other realm's endpoints. See + # ../keycloak/README.md. + keycloak: + image: quay.io/keycloak/keycloak:26.6.2 + container_name: learnstack-keycloak + restart: unless-stopped + command: ["start-dev", "--import-realm"] + environment: + # Dev-only. Production credentials come from Vault via ISecretProvider. + KEYCLOAK_ADMIN: admin + KEYCLOAK_ADMIN_PASSWORD: admin-dev-secret + KC_DB: postgres + KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak + KC_DB_USERNAME: learnstack + KC_DB_PASSWORD: learnstack + KC_HOSTNAME_STRICT: "false" + KC_HTTP_ENABLED: "true" + KC_HEALTH_ENABLED: "true" + KC_METRICS_ENABLED: "true" + KC_FEATURES: token-exchange,admin-fine-grained-authz + ports: + - "8080:8080" + volumes: + # Realm seed JSONs (learnstack + learnstack-hub) — Keycloak imports them + # at first start when the realms do not yet exist in the DB. + - ../keycloak/realms:/opt/keycloak/data/import:ro + depends_on: + postgres: + condition: service_healthy + # Realm SMTP server points at `mailpit:1025`. If Mailpit hasn't bound + # the SMTP listener yet when Keycloak sends its first email (password + # reset, verification), the send silently fails — TCP connect refused + # bubbles up as a generic SMTP timeout. Gating Keycloak start on + # mailpit-healthy closes that window. + mailpit: + condition: service_healthy + healthcheck: + # The Keycloak 26 image is UBI-9-minimal based + openjdk-21-runtime — + # neither curl nor wget is in the image. Bash IS available, so we use + # the upstream-recommended bash /dev/tcp probe against the management + # interface (KC_HEALTH_ENABLED=true exposes /health/ready on :9000). + test: + - CMD-SHELL + - | + exec 3<>/dev/tcp/localhost/9000 && \ + echo -e "GET /health/ready HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n" >&3 && \ + grep -q "200 OK" <&3 + interval: 15s + timeout: 10s + retries: 20 + start_period: 60s + + # ---- Live media stack (Phase 01 packet 5) -------------------------------- + # Self-hosted LiveKit OSS + Coturn for the in-app classroom (ADR-0005). + # The .NET app talks to LiveKit through the `ILiveClassProvider` abstraction + # (Phase 08c); the LiveKit SDK is never imported by any module — only by + # `LearnStack.Infrastructure.LiveClassroom.LiveKit`. + livekit: + image: livekit/livekit-server:v1.12.0 + container_name: learnstack-livekit + restart: unless-stopped + # `--node-ip 127.0.0.1` and `rtc.use_external_ip: false` in + # livekit.yaml are two ways of telling LiveKit not to auto-discover the + # host IP; either alone suffices in dev. Both are kept for explicitness + # — production sets `--node-ip ` + `use_external_ip: true`. + command: ["--config", "/etc/livekit.yaml", "--node-ip", "127.0.0.1", "--dev"] + volumes: + - ../livekit/livekit.yaml:/etc/livekit.yaml:ro + ports: + - "7880:7880" # WebSocket signaling + - "7881:7881" # TCP fallback + - "7882:7882/tcp" # TURN/TLS + - "50000-50100:50000-50100/udp" # WebRTC media plane + depends_on: + coturn: + condition: service_started + healthcheck: + # `--spider` issues a HEAD without downloading the body; tolerates + # LiveKit's WebSocket-upgrade contract changing across minor versions + # since we only care that the listener is bound. `127.0.0.1` (not + # `localhost`) for consistency with seaweedfs / meilisearch probes — + # the IPv6 `::1` fallback failure has bitten us on other Alpine-based + # minimal images, and the LiveKit image is built on the same family. + test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:7880"] + interval: 10s + timeout: 5s + retries: 10 + + coturn: + image: coturn/coturn:4.11.0 + container_name: learnstack-coturn + restart: unless-stopped + command: ["-c", "/etc/turnserver.conf"] + volumes: + - ../coturn/turnserver.conf:/etc/turnserver.conf:ro + ports: + - "3478:3478/udp" # STUN/TURN (UDP) + - "3478:3478/tcp" # STUN/TURN (TCP) + - "5349:5349/tcp" # TURN/TLS + - "49152-49200:49152-49200/udp" # TURN relay range (dev: narrow) + + # ---- Eventing (Phase 01 packet 6) ---------------------------------------- + # Kafka in KRaft mode (no ZooKeeper). Backs Dapr pub/sub per ADR-0014. + # Module code never imports `Confluent.Kafka` — only `IEventBus` via Dapr. + # + # Listener note: only the in-cluster PLAINTEXT://kafka:9092 listener is + # advertised. Host-side CLI tools (kcat, kafka-topics from the workstation) + # connect through the port mapping below but will receive `kafka:9092` as + # the bootstrap address and fail to resolve it unless the developer adds + # `127.0.0.1 kafka` to /etc/hosts (or routes them through the kafka-ui). + # Phase 07 (DX) ships either an explicit EXTERNAL listener or a + # documented kafka-ui-only workflow. + kafka: + image: confluentinc/cp-kafka:8.2.1 + container_name: learnstack-kafka + restart: unless-stopped + environment: + # KRaft mode — single-node broker + controller. Production uses an + # external HA cluster; this is dev only. + KAFKA_NODE_ID: 1 + KAFKA_PROCESS_ROLES: broker,controller + KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 + KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 + KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT + KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 + KAFKA_LOG_DIRS: /var/lib/kafka/data + # Stable cluster id so KRaft does not re-format the log dir on restart. + # Format: URL-safe Base64-encoded UUIDv4 (16-byte UUID → 22-char Base64 + # without padding) — the format `kafka-storage.sh random-uuid` emits. + # This value is checked in deliberately for dev reproducibility. + CLUSTER_ID: ofPH0p5rSlK52BxOX3qLPQ + ports: + - "9092:9092" + volumes: + - kafka-data:/var/lib/kafka/data + healthcheck: + test: ["CMD-SHELL", "kafka-topics --bootstrap-server localhost:9092 --list || exit 1"] + interval: 15s + timeout: 10s + retries: 20 + start_period: 30s + + # Kafka UI — using the active community fork `kafbat/kafka-ui` because the + # original `provectuslabs/kafka-ui` was abandoned (no release since + # 2024-04). The fork preserves the same env-var contract. Pinned per + # Standards 12 § Image Conventions (every dev image carries an explicit + # tag; bump deliberately, never let `:latest` drift). + kafka-ui: + image: ghcr.io/kafbat/kafka-ui:v1.5.0 + container_name: learnstack-kafka-ui + restart: unless-stopped + environment: + KAFKA_CLUSTERS_0_NAME: learnstack-dev + KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092 + DYNAMIC_CONFIG_ENABLED: "false" + ports: + - "8081:8080" + depends_on: + kafka: + condition: service_healthy + healthcheck: + # Spring Boot Actuator (`/actuator/health` returns 200 + `{"status":"UP"}` + # when the Kafka client has connected). Spring Boot startup with JDK 21 + # + Netty native init + Kafka client discovery is slow — `start_period` + # buys the boot 30s before the healthcheck starts counting failures. + test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:8080/actuator/health"] + interval: 10s + timeout: 5s + retries: 12 + start_period: 30s + + # ---- Secrets (Phase 01 packet 6) ----------------------------------------- + # HashiCorp Vault in -dev mode. Production runs Vault in HA mode with + # auto-unseal + AppRole / Kubernetes auth methods (Standards 12 § Secrets + # Management). The root token is intentionally embedded here for dev only. + # + # IMPORTANT: the literal `learnstack-dev-root-token` appears in TWO files: + # this service definition AND ../dapr/components/secretstore-vault.yaml. + # If you change one you MUST change the other. Phase 07 (DX) wires both + # to a single `.env.example` source so the duplication goes away. + vault: + image: hashicorp/vault:1.21.4 + container_name: learnstack-vault + restart: unless-stopped + command: ["server", "-dev", "-dev-root-token-id=learnstack-dev-root-token", "-dev-listen-address=0.0.0.0:8200"] + environment: + # Dev-only. Production loads tokens via Dapr `secretKeyRef` indirection + # so the literal never appears in component YAML. + VAULT_DEV_ROOT_TOKEN_ID: learnstack-dev-root-token + VAULT_ADDR: http://0.0.0.0:8200 + cap_add: + - IPC_LOCK + ports: + - "8200:8200" + healthcheck: + test: ["CMD", "vault", "status"] + interval: 10s + timeout: 5s + retries: 10 + + # ---- Dapr sidecar (Phase 01 packet 6) ------------------------------------ + # Placement service required by daprd even though actors are out of scope + # per ADR-0014 non-goals (daprd will not boot without it). + dapr-placement: + image: daprio/placement:1.17.7 + container_name: learnstack-dapr-placement + restart: unless-stopped + command: ["./placement", "-log-level", "info"] + ports: + - "50005:50005" + + # Single sidecar bound to the learnstack-api app id. The .NET host runs + # OUTSIDE compose during active dev (`dotnet run` on the workstation); + # `-app-channel-address host.docker.internal` is what makes inbound + # subscription deliveries reach the host (the daprd default of 127.0.0.1 + # would resolve to inside the sidecar container). The `extra_hosts` + # anchor at the top of the file maps `host.docker.internal` on Linux. + # See ../dapr/README.md. + dapr-sidecar-api: + image: daprio/daprd:1.17.7 + container_name: learnstack-dapr-sidecar-api + restart: unless-stopped + <<: *host-gateway + command: + - ./daprd + - -app-id + - learnstack-api + - -app-port + - "5080" + - -app-channel-address + - host.docker.internal + - -dapr-http-port + - "3500" + - -dapr-grpc-port + - "50001" + - -placement-host-address + - dapr-placement:50005 + - -resources-path + - /components + - -config + - /config/dapr-config.yaml + - -log-level + - info + volumes: + - ../dapr/components:/components:ro + - ../dapr/config:/config:ro + ports: + - "3500:3500" # HTTP API + - "50001:50001" # gRPC API + depends_on: + kafka: + condition: service_healthy + valkey: + condition: service_healthy + vault: + condition: service_healthy + dapr-placement: + condition: service_started + + # ---- Gateway (Phase 01 packet 6) ----------------------------------------- + # APISIX in file-driven standalone mode per ADR-0015 — no etcd, no Admin + # API. `apisix.yaml` is the only source of truth (hot-reloaded on file + # change). The `apisix-dashboard` companion is intentionally absent from + # this stack: the dashboard needs etcd to read routes, which contradicts + # the standalone-no-etcd commitment of ADR-0015. Diff-review of + # `apisix.yaml` replaces the dashboard for the dev workflow. + apisix: + image: apache/apisix:3.16.0-debian + container_name: learnstack-apisix + restart: unless-stopped + <<: *host-gateway + volumes: + - ../apisix/config.yaml:/usr/local/apisix/conf/config.yaml:ro + - ../apisix/apisix.yaml:/usr/local/apisix/conf/apisix.yaml:ro + ports: + - "9080:9080" # HTTP gateway + - "9443:9443" # HTTPS gateway (no cert in dev) + - "9091:9091" # Prometheus metrics + healthcheck: + # apache/apisix:3.16.0-debian ships NEITHER curl NOR wget (verified by + # inspecting the image). It does ship bash. The /dev/tcp pseudo-device + # is a bash builtin that opens a TCP connection — perfect lightweight + # listener probe. `127.0.0.1` (not `localhost`) avoids the IPv6 `::1` + # detour for the same reason as the seaweedfs / meilisearch probes. + test: ["CMD", "bash", "-c", "echo > /dev/tcp/127.0.0.1/9080"] interval: 10s timeout: 5s retries: 10 + start_period: 30s volumes: postgres-data: - redis-data: - minio-data: + valkey-data: + seaweedfs-data: meilisearch-data: + kafka-data: diff --git a/infra/compose/postgres-init/01-create-keycloak-db.sql b/infra/compose/postgres-init/01-create-keycloak-db.sql new file mode 100644 index 0000000..d28e80c --- /dev/null +++ b/infra/compose/postgres-init/01-create-keycloak-db.sql @@ -0,0 +1,13 @@ +-- Postgres init script — runs ONCE on a fresh `postgres-data` volume. +-- Creates the `keycloak` database the Keycloak service uses to store its +-- realm + user state. Keycloak shares the `learnstack` Postgres role + the +-- same Postgres instance (dev-only convenience; production isolates Keycloak +-- in its own Postgres cluster per Standards 12 § Database Operations). +-- +-- If `postgres-data` already exists, this script does NOT re-run; either +-- `docker compose -f infra/compose/dev.yml down -v` to wipe + reseed, or +-- manually `CREATE DATABASE keycloak OWNER learnstack;` once. + +SELECT 'CREATE DATABASE keycloak OWNER learnstack' +WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'keycloak') +\gexec diff --git a/infra/coturn/turnserver.conf b/infra/coturn/turnserver.conf new file mode 100644 index 0000000..cdeeeda --- /dev/null +++ b/infra/coturn/turnserver.conf @@ -0,0 +1,40 @@ +# Coturn — dev TURN/STUN server. +# Provides STUN reflexive-address lookup and TURN media relay for LiveKit +# clients behind symmetric NATs. Production uses a dedicated Coturn cluster +# with TLS certs, ephemeral credentials, and dedicated relay IP ranges; this +# file is dev-only. + +listening-port=3478 +tls-listening-port=5349 + +# Narrow relay port range for dev (production uses 49152-65535 or wider). +min-port=49152 +max-port=49200 + +# Dev realm. Production sets the actual hostname (e.g. turn.learnstack.dev) +# matched by the issued TLS cert. +realm=localhost + +# Long-term credential mechanism (LiveKit issues per-session credentials in +# production via the shared-secret pattern — but dev uses a static user/secret +# pair so a developer can poke at TURN with `turnutils_uclient` without +# bouncing through LiveKit). +lt-cred-mech + +# Dev-only static credentials. Production credentials come from Vault via +# ISecretProvider; relay-side use auth-secret + use-auth-secret for ephemeral +# tokens. +user=devuser:devsecret + +# Network hygiene. +no-multicast-peers +mobility +# Notes for Coturn 4.6+: +# - `no-cli` removed: CLI is off by default; enable explicitly with cli-port+cli-password. +# - `no-tlsv1` / `no-tlsv1_1` removed: TLS 1.0 / 1.1 are already disabled by default +# (and OpenSSL deprecates them platform-wide); listing them only triggers a config +# parse warning. Coturn 4.11 negotiates TLS 1.2 / 1.3 only. + +# Logging to stdout for `docker compose logs coturn`. +log-file=stdout +verbose diff --git a/infra/dapr/README.md b/infra/dapr/README.md new file mode 100644 index 0000000..7c4bdc6 --- /dev/null +++ b/infra/dapr/README.md @@ -0,0 +1,102 @@ +# Dapr Sidecar (Dev) + +Cross-cutting infrastructure runtime per +[ADR-0014 (Adopt Dapr)](../../docs/decisions/0014-adopt-dapr.md). Three +building blocks are adopted; everything else is **out of scope** per ADR-0014 +non-goals. + +| Building block | Backend (dev) | Component file | Application interface | +|----------------|---------------|----------------|-----------------------| +| Pub/Sub | Kafka (`kafka:9092`) | `components/pubsub-kafka.yaml` | `IEventBus` (Phase 02b) | +| State store | Valkey (`valkey:6379`, RESP protocol) | `components/statestore-redis.yaml` | `ICacheService` (Phase 02a) | +| Secret store | Vault (`http://vault:8200`, dev mode) | `components/secretstore-vault.yaml` | `ISecretProvider` (Phase 02a) | + +Service invocation, workflow, bindings, **actors**, configuration, and +distributed lock are **not adopted**; if a future need appears the gate is a +new ADR. The state store's `actorStateStore` flag is therefore pinned to +`"false"` — see the comment at the top of `components/statestore-redis.yaml`. + +## Sidecar topology + +Dev compose runs one sidecar bound to the `learnstack-api` app id: + +```text +┌──────────────────────────┐ ┌──────────────────────────────────────────┐ +│ dotnet run │ │ daprd │ +│ → host:5080 │ │ ./daprd -app-id learnstack-api \ │ +│ │ │ -app-port 5080 \ │ +│ │◄──┤ -app-channel-address \ │ +│ │ │ host.docker.internal \ │ +│ │ │ -dapr-http-port 3500 \ │ +│ │ │ -dapr-grpc-port 50001 \ │ +│ │ │ -placement-host-address \ │ +│ │ │ dapr-placement:50005 \ │ +│ │ │ -resources-path /components \ │ +│ │ │ -config /config/dapr-config.yaml +└──────────────────────────┘ └──────────────────────────────────────────┘ +``` + +The .NET host runs **outside the container network** during active dev +(developers `dotnet run` from their workstation). `-app-channel-address +host.docker.internal` is what makes inbound subscription deliveries reach +the host — daprd's default of `127.0.0.1` would resolve to inside the +sidecar container, and every subscription would silently fail to deliver. +The `extra_hosts: host.docker.internal:host-gateway` YAML anchor in +`infra/compose/dev.yml` maps the alias on Linux (Docker Desktop does it +automatically). + +The placement service (`dapr-placement`) is required even though actors are +out of scope — `daprd` won't start without it. + +## Application access pattern + +Per ADR-0014 + Standards 20, modules **never** import `Dapr.Client`. They +consume: + +```csharp +public interface IEventBus { Task PublishAsync(T @event, CancellationToken ct) where T : IIntegrationEvent; } +public interface ICacheService { Task GetAsync(string key, CancellationToken ct); /* … */ } +public interface ISecretProvider { Task GetSecretAsync(string key, CancellationToken ct); /* … */ } +``` + +`DaprEventBus`, `DaprCacheService`, `DaprSecretProvider` are the **only** +Dapr-aware types in the codebase; they live in `LearnStack.Infrastructure` +and ship in Phase 02b. Architecture tests +`Dapr_SDK_Types_NotImportedOutsideInfrastructure`, +`Modules_DoNotReference_DaprPackage`, and +`ICacheService_Is_OnlyCacheAbstraction` keep this honest. + +## Dev credentials + +| Surface | Credential | +|---------|------------| +| Vault root token | `learnstack-dev-root-token` | +| Kafka auth | none (`authType: none`, `disableTls: true`) | +| Valkey password | (empty) | + +All dev-only. Production wires Vault with AppRole / Kubernetes auth and +loads the token through Dapr's `secretKeyRef` indirection so the literal +token never appears in the component YAML. + +### Vault token duplication + +The literal `learnstack-dev-root-token` appears in **two files**: + +- `infra/compose/dev.yml` — `vault` service command + env var (the token + Vault `-dev` mode boots with). +- `infra/dapr/components/secretstore-vault.yaml` — `vaultToken` metadata + (the token Dapr authenticates to Vault with). + +These MUST stay in lockstep. Phase 07 (DX) wires both to a single +`.env.example` source so the duplication goes away; until then, change +both places together. + +## What does NOT live here + +- The `IEventBus` / `ICacheService` / `ISecretProvider` implementations — + Phase 02b (`LearnStack.Infrastructure`). +- Outbox dispatcher (`OutboxProcessor` polling + dispatch) — Phase 02b. +- Per-module `inbox_messages` table + `IInboxGuard` — Phase 02b. +- Production Vault setup (HA mode, auto-unseal, AppRole policies) — Phase 11. +- Additional sidecars for the Hub overlay (separate app id, separate + consumer group) — Phase 02c. diff --git a/infra/dapr/components/pubsub-kafka.yaml b/infra/dapr/components/pubsub-kafka.yaml new file mode 100644 index 0000000..93ca0ff --- /dev/null +++ b/infra/dapr/components/pubsub-kafka.yaml @@ -0,0 +1,31 @@ +# Dapr pub/sub component — Kafka backend. +# Per ADR-0014, application code talks to `IEventBus` exclusively; the +# `DaprEventBus` implementation (lands Phase 02b) publishes through this +# component. Topic naming follows the `learnstack.{module}.{aggregate}` +# convention enforced by `Dapr_PubSub_TopicNames_FollowConvention` +# architecture test. +# +# IMPORTANT: `consumerGroup` is pinned to the LearnStack core app id +# (`learnstack-api`). Adding a second consumer (a second app id, the Hub +# overlay in packet 7, a separate worker process) requires a DIFFERENT +# consumer group — copy this component, do not reuse the YAML with a +# different `app-id`. Sharing a consumer group across distinct apps +# silently splits the topic partitions between them and breaks +# at-least-once semantics for both. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: pubsub +spec: + type: pubsub.kafka + version: v1 + metadata: + - name: brokers + value: kafka:9092 + - name: consumerGroup + value: learnstack-api + - name: authType + value: none + - name: disableTls + value: "true" diff --git a/infra/dapr/components/secretstore-vault.yaml b/infra/dapr/components/secretstore-vault.yaml new file mode 100644 index 0000000..c72489a --- /dev/null +++ b/infra/dapr/components/secretstore-vault.yaml @@ -0,0 +1,31 @@ +# Dapr secret store — HashiCorp Vault backend. +# Backs `ISecretProvider`. Dev compose runs Vault in -dev mode with the +# root token baked in; production replaces this with an authenticated +# Vault cluster + AppRole / Kubernetes auth method. +# +# Dev-only token. NEVER deploy a config carrying `vaultToken` literally — +# production wires the auth method via Vault's Dapr component metadata +# patterns documented at https://docs.dapr.io/. +# +# KV ENGINE NOTE (Phase 02b): Vault -dev mounts `secret/` as KV v2; this +# component reads from that mount with `vaultKVPrefix: secret`. Some Dapr +# Vault-component versions need an explicit `engineType: kv-v2` metadata +# entry to detect v2's `/data/` indirection. Validate `GET +# /v1.0/secrets/secretstore/` against a known dev secret before the +# first ISecretProvider integration test in Phase 02b; if it returns 404 +# while the secret exists, add `engineType: kv-v2` here. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: secretstore +spec: + type: secretstores.hashicorp.vault + version: v1 + metadata: + - name: vaultAddr + value: http://vault:8200 + - name: vaultToken + value: learnstack-dev-root-token + - name: vaultKVPrefix + value: secret diff --git a/infra/dapr/components/statestore-redis.yaml b/infra/dapr/components/statestore-redis.yaml new file mode 100644 index 0000000..29e8a48 --- /dev/null +++ b/infra/dapr/components/statestore-redis.yaml @@ -0,0 +1,34 @@ +# Dapr state store — RESP-protocol backend (Valkey). +# Backs `ICacheService` (L2 cache). Module code never imports +# `IDistributedCache` / `StackExchange.Redis` directly per +# `ICacheService_Is_OnlyCacheAbstraction` architecture test. +# +# `spec.type: state.redis` is the Dapr PROVIDER-TYPE identifier — it +# names the RESP-protocol adapter, NOT the Redis Inc. brand. Per +# ADR-0030 the actual backend is Valkey 8.x (Linux Foundation, +# BSD-3-Clause); Valkey is RESP-protocol-compatible with Redis 7.4 so +# Dapr's `state.redis` adapter consumes it without change. The file +# name keeps the `-redis` suffix because that mirrors the Dapr +# provider-type convention; renaming it would just hide the wire +# protocol the component speaks. +# +# `actorStateStore: "false"` is INTENTIONAL — per ADR-0014 non-goals, +# Dapr actors are NOT adopted. Flipping this to "true" requires a new +# ADR + an architecture-test update. The Dapr docs are explicit that the +# flag must be "true" only when the application code consumes the actors +# building block; declaring it true unused signals false intent. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: statestore +spec: + type: state.redis + version: v1 + metadata: + - name: redisHost + value: valkey:6379 + - name: redisPassword + value: "" + - name: actorStateStore + value: "false" diff --git a/infra/dapr/config/dapr-config.yaml b/infra/dapr/config/dapr-config.yaml new file mode 100644 index 0000000..b9f7b8d --- /dev/null +++ b/infra/dapr/config/dapr-config.yaml @@ -0,0 +1,14 @@ +# Dapr runtime configuration. +# Tracing target intentionally empty in dev — Phase 11 wires Tempo/Zipkin. +# Module code never reads this file; it tunes the sidecar's own behaviour. + +apiVersion: dapr.io/v1alpha1 +kind: Configuration +metadata: + name: dapr-config +spec: + tracing: + samplingRate: "1" + zipkin: + endpointAddress: "" + features: [] diff --git a/infra/keycloak/README.md b/infra/keycloak/README.md new file mode 100644 index 0000000..63b01b4 --- /dev/null +++ b/infra/keycloak/README.md @@ -0,0 +1,125 @@ +# Keycloak — Two-Realm Identity (Dev) + +Self-hosted Keycloak in dev compose, per +[ADR-0004 (Authentication Strategy)](../../docs/decisions/0004-authentication-strategy.md) +and [ADR-0019 (LearnStack Hub)](../../docs/decisions/0019-learnstack-hub.md). Two +realms, **hard-isolated from each other** — neither realm trusts tokens issued by +the other. This is an architectural invariant, not a configurable policy. + +## Realms + +| Realm | Purpose | User population | Clients | Demo users | +|-------|---------|-----------------|---------|------------| +| `learnstack` | Tenant-facing: admins, instructors, learners | All customer users | `learnstack-api` (confidential, service-account) + `learnstack-web` (public PKCE) | `demo-admin@tenant-a.test`, `demo-learner@tenant-a.test` (both `demo-dev-secret`) | +| `learnstack-hub` | Operator-facing: platform admin, support, billing-viewer | LearnStack staff only | `learnstack-hub-web` (public PKCE) | `demo-operator@learnstack.test` (`demo-dev-secret`) | + +## Access + +- **Admin console** — http://localhost:8080 (Keycloak Admin UI) +- **Master admin** — `admin` / `admin-dev-secret` +- **Tenant realm OIDC discovery** — http://localhost:8080/realms/learnstack/.well-known/openid-configuration +- **Hub realm OIDC discovery** — http://localhost:8080/realms/learnstack-hub/.well-known/openid-configuration + +## Dev credentials are dev credentials + +Every secret in this directory (master admin password, client secret, demo user +passwords) is **dev-only**. Production deployments load these from Vault via +`ISecretProvider` per [Standards 12 § Secrets Management](../../docs/standards/12-infrastructure.md) +and [Standards 20](../../docs/standards/20-infrastructure-stack.md). Do not reuse +any of these strings outside local Docker. + +## Two realms, zero cross-trust + +The realm separation is a **hard architectural invariant** per +[ADR-0004 Amendment 1](../../docs/decisions/0004-authentication-strategy.md#amendment-1): + +- A `learnstack-hub` token MUST be rejected on every tenant-facing endpoint + (the gateway + the backend both check the `iss` claim against the realm URL). +- A `learnstack` token MUST be rejected on every `/api/internal/*` endpoint + (the Hub-internal contract gates on mTLS + signed JWT, not on user tokens). +- An operator account cannot also be a tenant user under the same identity, and + vice versa. + +The Phase 02b OIDC integration enforces this in code; the dev seed mirrors it +in data (no shared users, no shared client IDs). + +## Dev-only client posture + +The realm JSONs keep `fullScopeAllowed: true` on every client for dev +simplicity — service-account tokens (for `learnstack-api`) and user tokens +(for `learnstack-web` / `learnstack-hub-web`) carry every realm role +automatically, so demo flows work without per-client `defaultClientScopes` +provisioning. **Production realm provisioning** (Terraform / +keycloak-config-cli, not these JSONs) flips this: + +| Client | Type | Dev | Production target | +|--------|------|-----|-------------------| +| `learnstack-api` | confidential, service-account | `fullScopeAllowed: true`, `directAccessGrantsEnabled: false`, `webOrigins: []` | `fullScopeAllowed: false` + explicit `scopeMappings` (least-privilege) | +| `learnstack-web` | public, PKCE | `fullScopeAllowed: true` | `fullScopeAllowed: false` + explicit `defaultClientScopes` listing the tenant-* roles only | +| `learnstack-hub-web` | public, PKCE | `fullScopeAllowed: true` | `fullScopeAllowed: false` + explicit `defaultClientScopes` listing the hub-* roles only | + +These notes used to sit inline in each client's `description` field, but +Keycloak's `CLIENT.DESCRIPTION` column is `varchar(255)` and the long +inline notes broke realm import. The descriptions in the JSON now point +back to this section. + +## Dev-only realm posture (tenant realm) + +The `learnstack` realm JSON carries permissive defaults so the demo flows +work without operator intervention. Each of these defaults is **flipped** +by the production realm-provisioning flow (Terraform / keycloak-config-cli), +not by editing the dev JSON: + +| Attribute | Dev (`learnstack.json`) | Production target | Why dev value | +|-----------|------------------------|-------------------|---------------| +| `registrationAllowed` | `true` | `false` (invite-only) | Lets reviewers exercise the tenant sign-up flow against `localhost:3000` without seeding a user first. | +| `verifyEmail` | `false` | `true` | Dev SMTP is Mailpit; flipping verification on would gate every demo on visiting `localhost:8025`. | +| `accessTokenLifespan` | `1800` (matches `learnstack-hub`) | `900` recommended (≤ 15 min, refresh-token-driven) | Halves the round-trip churn of debug sessions; still short enough to surface refresh bugs. | +| `bruteForceProtected` | `true` (same as prod) | `true` | No dev relaxation. | + +The `learnstack-hub` realm is **already** at the stricter posture +(`registrationAllowed: false`, `verifyEmail: true`, `accessTokenLifespan: 1800`) +because operators are seeded out-of-band per +[ADR-0019](../../docs/decisions/0019-learnstack-hub.md). The tenant-realm +fields above are the only intentional dev/prod deltas. + +## MFA + +- `learnstack-hub` realm: `CONFIGURE_TOTP` declared as a required action so the + TOTP enrolment flow is visible in dev; production enforces MFA mandatory for + every operator per ADR-0004 Amendment 1. +- `learnstack` realm: MFA is **optional in dev**; tenant policy may make it + mandatory per-tenant in Phase 03. + +## How the realms are seeded + +Keycloak boots with `start-dev --import-realm`. The JSON files under +`realms/` are bind-mounted to `/opt/keycloak/data/import/` and consumed once +at first start. To re-seed: + +```bash +docker compose -f infra/compose/dev.yml down keycloak +docker compose -f infra/compose/dev.yml up -d keycloak +``` + +Re-import overwrites the realm only if the realm did NOT already exist (Keycloak +default behaviour). To force a clean re-seed of the realm itself, wipe the +Keycloak database first: + +```bash +docker compose -f infra/compose/dev.yml exec postgres \ + psql -U learnstack -d learnstack -c "DROP DATABASE keycloak;" +docker compose -f infra/compose/dev.yml exec postgres \ + psql -U learnstack -d learnstack -c "CREATE DATABASE keycloak OWNER learnstack;" +docker compose -f infra/compose/dev.yml restart keycloak +``` + +## What does NOT live here + +- Production realm configuration — managed via terraform / keycloak-config-cli + per environment; the dev JSONs are not the source of truth for production. +- LearnStack-side identity domain (`User`, `Membership`, `Role`, `Permission`, + `Invitation`) — that's Phase 03, owned by `LearnStack.Modules.Identity`. The + realms here only define authentication; authorisation is application-side. +- OIDC code integration in the .NET API — Phase 02b wires `AddJwtBearer` and + the BFF callback handler. diff --git a/infra/keycloak/realms/learnstack-hub.json b/infra/keycloak/realms/learnstack-hub.json new file mode 100644 index 0000000..6a2d4d2 --- /dev/null +++ b/infra/keycloak/realms/learnstack-hub.json @@ -0,0 +1,111 @@ +{ + "realm": "learnstack-hub", + "displayName": "LearnStack Hub — Operators", + "displayNameHtml": "LearnStack Hub — Operators", + "enabled": true, + "registrationAllowed": false, + "rememberMe": false, + "verifyEmail": true, + "loginWithEmailAllowed": true, + "duplicateEmailsAllowed": false, + "resetPasswordAllowed": true, + "editUsernameAllowed": false, + "bruteForceProtected": true, + "permanentLockout": false, + "maxFailureWaitSeconds": 900, + "minimumQuickLoginWaitSeconds": 60, + "waitIncrementSeconds": 60, + "failureFactor": 10, + "accessTokenLifespan": 1800, + "ssoSessionIdleTimeout": 1800, + "ssoSessionMaxLifespan": 14400, + "sslRequired": "external", + "internationalizationEnabled": true, + "supportedLocales": ["en"], + "defaultLocale": "en", + "requiredActions": [ + { + "alias": "CONFIGURE_TOTP", + "name": "Configure OTP", + "providerId": "CONFIGURE_TOTP", + "enabled": true, + "defaultAction": false, + "priority": 10, + "config": {} + } + ], + "roles": { + "realm": [ + { + "name": "hub-platform-admin", + "description": "Full Hub platform administration: plan CRUD, tenant lifecycle, custom-domain admin, license issuance.", + "composite": false, + "clientRole": false + }, + { + "name": "hub-operator", + "description": "Day-to-day Hub operator: tenant support, entitlement overrides, audit review.", + "composite": false, + "clientRole": false + }, + { + "name": "hub-billing-viewer", + "description": "Read-only access to billing + invoice surfaces in the operator portal.", + "composite": false, + "clientRole": false + } + ] + }, + "clients": [ + { + "clientId": "learnstack-hub-web", + "name": "LearnStack Hub Web (public PKCE)", + "description": "Hub operator portal. Public client, PKCE-enforced. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", + "enabled": true, + "publicClient": true, + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": false, + "serviceAccountsEnabled": false, + "redirectUris": [ + "http://localhost:3100/*" + ], + "webOrigins": ["http://localhost:3100"], + "protocol": "openid-connect", + "attributes": { + "pkce.code.challenge.method": "S256", + "post.logout.redirect.uris": "http://localhost:3100/*" + }, + "fullScopeAllowed": true + } + ], + "users": [ + { + "username": "demo-operator@learnstack.test", + "email": "demo-operator@learnstack.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Operator", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["hub-operator"], + "requiredActions": ["CONFIGURE_TOTP"] + } + ], + "smtpServer": { + "host": "mailpit", + "port": "1025", + "from": "no-reply@hub.learnstack.dev", + "fromDisplayName": "LearnStack Hub Dev", + "replyTo": "no-reply@hub.learnstack.dev", + "ssl": "false", + "starttls": "false", + "auth": "false" + } +} diff --git a/infra/keycloak/realms/learnstack.json b/infra/keycloak/realms/learnstack.json new file mode 100644 index 0000000..1a4f474 --- /dev/null +++ b/infra/keycloak/realms/learnstack.json @@ -0,0 +1,154 @@ +{ + "realm": "learnstack", + "displayName": "LearnStack — Tenant Users", + "displayNameHtml": "LearnStack — Tenant Users", + "enabled": true, + "registrationAllowed": true, + "registrationEmailAsUsername": true, + "rememberMe": true, + "verifyEmail": false, + "loginWithEmailAllowed": true, + "duplicateEmailsAllowed": false, + "resetPasswordAllowed": true, + "editUsernameAllowed": false, + "bruteForceProtected": true, + "permanentLockout": false, + "maxFailureWaitSeconds": 900, + "minimumQuickLoginWaitSeconds": 60, + "waitIncrementSeconds": 60, + "quickLoginCheckMilliSeconds": 1000, + "maxDeltaTimeSeconds": 43200, + "failureFactor": 30, + "accessTokenLifespan": 1800, + "accessTokenLifespanForImplicitFlow": 900, + "ssoSessionIdleTimeout": 1800, + "ssoSessionMaxLifespan": 36000, + "offlineSessionIdleTimeout": 2592000, + "accessCodeLifespan": 60, + "accessCodeLifespanUserAction": 300, + "accessCodeLifespanLogin": 1800, + "actionTokenGeneratedByAdminLifespan": 43200, + "actionTokenGeneratedByUserLifespan": 300, + "sslRequired": "external", + "internationalizationEnabled": true, + "supportedLocales": ["en", "tr"], + "defaultLocale": "en", + "roles": { + "realm": [ + { + "name": "tenant-admin", + "description": "Tenant administrator — manages users, content, billing within a tenant.", + "composite": false, + "clientRole": false + }, + { + "name": "tenant-instructor", + "description": "Instructor — authors content + runs live sessions within a tenant.", + "composite": false, + "clientRole": false + }, + { + "name": "tenant-learner", + "description": "Learner — enrols in courses + attends sessions within a tenant.", + "composite": false, + "clientRole": false + } + ] + }, + "clients": [ + { + "clientId": "learnstack-api", + "name": "LearnStack API (confidential service)", + "description": "Backend .NET API. Confidential client; standard flow + service-account. ROPC disabled, webOrigins empty. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", + "enabled": true, + "publicClient": false, + "secret": "learnstack-api-dev-secret", + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": false, + "serviceAccountsEnabled": true, + "redirectUris": [ + "http://localhost:5080/*", + "http://localhost:3000/*" + ], + "webOrigins": [], + "protocol": "openid-connect", + "attributes": { + "access.token.lifespan": "1800", + "pkce.code.challenge.method": "S256" + }, + "fullScopeAllowed": true + }, + { + "clientId": "learnstack-web", + "name": "LearnStack Web (public PKCE)", + "description": "Next.js apps/web. Public client, PKCE-enforced, no secret. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", + "enabled": true, + "publicClient": true, + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": false, + "serviceAccountsEnabled": false, + "redirectUris": [ + "http://localhost:3000/*" + ], + "webOrigins": ["http://localhost:3000"], + "protocol": "openid-connect", + "attributes": { + "pkce.code.challenge.method": "S256", + "post.logout.redirect.uris": "http://localhost:3000/*" + }, + "fullScopeAllowed": true + } + ], + "users": [ + { + "username": "demo-admin@tenant-a.test", + "email": "demo-admin@tenant-a.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Admin", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["tenant-admin"], + "attributes": { + "tenant_id": ["tenant-a"] + } + }, + { + "username": "demo-learner@tenant-a.test", + "email": "demo-learner@tenant-a.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Learner", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["tenant-learner"], + "attributes": { + "tenant_id": ["tenant-a"] + } + } + ], + "smtpServer": { + "host": "mailpit", + "port": "1025", + "from": "no-reply@learnstack.dev", + "fromDisplayName": "LearnStack Dev", + "replyTo": "no-reply@learnstack.dev", + "ssl": "false", + "starttls": "false", + "auth": "false" + } +} diff --git a/infra/livekit/README.md b/infra/livekit/README.md new file mode 100644 index 0000000..9bacf22 --- /dev/null +++ b/infra/livekit/README.md @@ -0,0 +1,69 @@ +# LiveKit OSS + Coturn (Dev) + +Self-hosted live-classroom media stack per +[ADR-0005 (Live Classroom Media Stack)](../../docs/decisions/0005-live-classroom-media-stack.md). +The .NET application talks to LiveKit through the `ILiveClassProvider` +abstraction (Phase 08c); the LiveKit server SDK is never imported by any +LearnStack module — only by `LearnStack.Infrastructure.LiveClassroom.LiveKit`. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| WebSocket signaling | `ws://localhost:7880` | LiveKit client connects here for room join + signaling | +| TCP fallback | `tcp://localhost:7881` | WebRTC TCP fallback when UDP is blocked | +| TURN/TLS | `tcp://localhost:7882` | Encrypted TURN listener (dev: no real cert) | +| WebRTC media | UDP `50000-50100` | LiveKit media plane | +| STUN/TURN (Coturn) | `udp/tcp://localhost:3478` | NAT reflexive-address + media relay | +| TURN/TLS (Coturn) | `tcp://localhost:5349` | Encrypted TURN listener | +| Relay range (Coturn) | UDP `49152-49200` | TURN relay port range (dev: narrowed from production) | + +## Dev API key + +LiveKit room tokens are signed with a single dev key/secret pair declared in +`infra/livekit/livekit.yaml`: + +| Key | Secret | +|-----|--------| +| `devkey` | `devsecret-32-byte-min-length-padding-xyz` | + +The secret padding satisfies LiveKit's >= 32-byte requirement. Production +key/secret pairs come from Vault via `ISecretProvider` per +[Standards 12 § Secrets Management](../../docs/standards/12-infrastructure.md); +do not reuse these strings. + +## Dev Coturn credentials + +| User | Password | +|------|----------| +| `devuser` | `devsecret` | + +These are **static long-term credentials** — fine for a developer poking at +TURN with `turnutils_uclient`, but production switches to the +`use-auth-secret` shared-secret pattern so LiveKit can issue per-session +ephemeral credentials. + +## How LearnStack uses LiveKit + +The full integration arrives in Phase 08c. Summary: + +- **`ILiveClassProvider`** abstraction in `LearnStack.Application.Contracts`; + LiveKit-specific implementation in `LearnStack.Infrastructure.LiveClassroom.LiveKit`. +- **Token issuance** happens server-side from `learnstack-api` using the dev + key above; tokens scoped per `(tenant_id, session_id, user_id)` with a + short TTL. +- **Recording** consumes LiveKit Egress (separate service, lands in + Phase 08c) writing to SeaweedFS via the existing storage provider abstraction. + Recording is **tenant-configurable** and **consent-aware** per ADR-0005 + + [16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). +- **Cost metrics** (participant minutes, bandwidth, recording minutes) + surface in the Phase 09 analytics pipeline. + +## What does NOT live here + +- LiveKit Egress (recording) — Phase 08c. +- Production TLS cert provisioning — ADR-0022 + the same Let's Encrypt + adapter family APISIX uses. +- Custom WebRTC SFU — explicitly out of scope per ADR-0005 + ADR-0018. +- Cloud LiveKit (managed) — supported as a swap-in path through the same + `ILiveClassProvider` adapter; dev defaults to self-hosted OSS. diff --git a/infra/livekit/livekit.yaml b/infra/livekit/livekit.yaml new file mode 100644 index 0000000..4227e53 --- /dev/null +++ b/infra/livekit/livekit.yaml @@ -0,0 +1,53 @@ +# LiveKit OSS — dev config. +# Per ADR-0005, LiveKit is the preferred SFU; self-hosted OSS is the default +# supported path. The .NET application talks to LiveKit through the +# `ILiveClassProvider` abstraction (Phase 08c) — never imports the LiveKit +# server SDK directly. +# +# Dev-only keys + secrets. Production replaces these with values fetched +# from Vault via ISecretProvider per Standards 12 § Secrets Management. + +port: 7880 + +rtc: + tcp_port: 7881 + # Dev: machine has no stable external IP; bind to loopback for now. + # Production sets `use_external_ip: true` so LiveKit auto-discovers the + # node's public IP for ICE candidates. + use_external_ip: false + port_range_start: 50000 + port_range_end: 50100 + +# Dev API key + secret. The secret MUST be >= 32 bytes (LiveKit hard rule); +# the value below is padded to satisfy that and is checked into git only +# because this file is the dev compose config. +keys: + devkey: devsecret-32-byte-min-length-padding-xyz + +turn: + # LiveKit's built-in TURN is OFF in dev — ADR-0005 uses standalone Coturn + # (see the `coturn` service in dev.yml) for STUN/TURN. Built-in TURN also + # rejects `domain: localhost` ("TURN domain is not correct"), which would + # otherwise restart-loop the container. Production keeps the standalone- + # Coturn split for the same reason: rotating credentials and TLS termination + # live on the TURN side, not in the SFU. + enabled: false + +# ICE servers (Phase 08c wiring): The `coturn` service runs at host:3478 but +# THIS file does not yet declare `rtc.turn_servers` — meaning LiveKit room +# tokens issued from a `Room.CreateAt`-flavoured call carry no TURN reference +# and browser clients fall back to direct ICE only. The wiring lands with the +# `ILiveClassProvider` LiveKit adapter in Phase 08c (per ADR-0005), which +# issues per-session credentials via the shared-secret pattern; until then, +# dev classroom flows that need symmetric-NAT relay must be tested with +# `--node-ip` set to the workstation's LAN IP, not 127.0.0.1. + +room: + auto_create: true + empty_timeout: 300 + departure_timeout: 20 + max_participants: 50 + +logging: + level: info + json: false diff --git a/infra/seaweedfs/README.md b/infra/seaweedfs/README.md new file mode 100644 index 0000000..43e25fc --- /dev/null +++ b/infra/seaweedfs/README.md @@ -0,0 +1,61 @@ +# SeaweedFS (Dev) + +Self-hosted S3-compatible object storage per +[ADR-0029](../../docs/decisions/0029-object-storage-seaweedfs.md). The .NET +app talks to SeaweedFS through `IStorageProvider`; the SeaweedFS SDK is +never imported by any module — only by +`LearnStack.Infrastructure.Storage.SeaweedFS` (ships Phase 02a). The dev +container packs master + volume + filer + S3 gateway in one binary; the +production topology splits them per ADR-0029 § Implementation Notes. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| S3 API gateway | `http://localhost:9000` | The endpoint `IStorageProvider` talks to (drop-in port-map for the previous MinIO S3 endpoint) | +| Filer / volume UI | `http://localhost:9001` | Bucket browser, replaces the MinIO console | +| Master HTTP API | `http://localhost:9333` | Cluster topology + health (`/cluster/healthz`) | +| Volume HTTP API | `http://localhost:8084` | Internal — read / write blob ops. Host port 8084 (container 8080) avoids Keycloak's 8080 binding. | + +## Dev credentials + +| Surface | Credential | +|---------|------------| +| S3 access key | `learnstack` | +| S3 secret key | `learnstack-dev-secret` | + +Both come from `infra/seaweedfs/s3-identities.json`, mounted read-only at +`/etc/s3.json`. Production loads identities from Vault via +`ISecretProvider` per Standards 12 § Secrets Management; the literals +above are dev-only. + +The credential pair intentionally matches the prior MinIO defaults so any +test fixture, env var, or local script that already hard-coded +`learnstack` / `learnstack-dev-secret` keeps working through the swap. + +## Tenant isolation pattern (unchanged) + +Tenant isolation in object storage is enforced by **key prefix** +(`{tenant_id}/...`), never bucket-per-tenant — see +[Standards 12 § Object Storage Operations](../../docs/standards/12-infrastructure.md) +and [docs/architecture/16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). +A single bucket per environment is created at first use by the +`IStorageProvider` adapter at startup. + +## Re-seed / wipe + +```bash +docker compose -f infra/compose/dev.yml down seaweedfs +docker volume rm learnstack-dev_seaweedfs-data +docker compose -f infra/compose/dev.yml up -d seaweedfs +``` + +## What does NOT live here + +- The `IStorageProvider` adapter implementation — + `LearnStack.Infrastructure.Storage.SeaweedFS` (Phase 02a). +- Production-mode topology (split master / volume / filer / S3 gateway + containers, erasure-coding policy, tiered storage) — Phase 11. +- Multi-region replication evaluation — Phase 11+. +- The MinIO-era image (`minio/minio:RELEASE.2025-01-20T...`) — removed + in this packet per ADR-0029. diff --git a/infra/seaweedfs/s3-identities.json b/infra/seaweedfs/s3-identities.json new file mode 100644 index 0000000..5ee9ea4 --- /dev/null +++ b/infra/seaweedfs/s3-identities.json @@ -0,0 +1,14 @@ +{ + "identities": [ + { + "name": "learnstack-dev", + "credentials": [ + { + "accessKey": "learnstack", + "secretKey": "learnstack-dev-secret" + } + ], + "actions": ["Admin", "Read", "Write", "List", "Tagging"] + } + ] +}