From 2680da94e3ad6f7075ff72a3f7c7b9a22841e2e4 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 17:23:00 +0300 Subject: [PATCH 01/11] =?UTF-8?q?feat(infra):=20Phase=2001=20packet=204=20?= =?UTF-8?q?=E2=80=94=20Keycloak=20two-realm=20identity=20stack?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stands up the self-hosted Keycloak identity provider in dev compose with the two-realm topology that ADR-0004 Amendment 1 + ADR-0019 mandate: `learnstack` for tenant users and `learnstack-hub` for LearnStack operators. The realms are hard-isolated — neither realm trusts tokens issued by the other — which is the load-bearing invariant the Phase-02b OIDC integration and the Hub internal API surface will both lean on. Adds - Keycloak 26 service in `infra/compose/dev.yml` (port 8080, management port 9000 for /health/ready, depends on healthy Postgres). - `infra/compose/postgres-init/01-create-keycloak-db.sql` — idempotent CREATE DATABASE bound to Postgres via docker-entrypoint-initdb.d on first boot of the `postgres-data` volume. - `infra/keycloak/realms/learnstack.json` — tenant realm, clients `learnstack-api` (confidential, service-account + standard + direct grants) and `learnstack-web` (public PKCE), roles tenant-{admin,instructor,learner}, two demo users wired to Mailpit SMTP. - `infra/keycloak/realms/learnstack-hub.json` — operator realm, client `learnstack-hub-web` (public PKCE), roles hub-{platform-admin,operator, billing-viewer}, one demo operator, CONFIGURE_TOTP required action so the MFA enrolment flow surfaces in dev (per ADR-0004 Amendment 1: MFA mandatory in production). - `infra/keycloak/README.md` — realm matrix, access URLs, cross-trust invariant, re-seed procedure, what does/does NOT live here. Docs - `infra/compose/README.md` regrouped data-plane → identity (packet 4) → remaining pending packets. - `docs/roadmap/phase-01-repository-tooling.md` packet-4 status flipped to ✅. Verification - Both realm JSONs parse cleanly. - `docker compose -f infra/compose/dev.yml config -q` exits 0. - Markdown link sweep on changed docs clean. - Compose-up smoke test not run in this commit (Keycloak first boot is ~60s); the packet 5/6 stack will exercise the full stack together. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/roadmap/phase-01-repository-tooling.md | 9 +- infra/compose/README.md | 45 +++-- infra/compose/dev.yml | 58 ++++++- .../postgres-init/01-create-keycloak-db.sql | 13 ++ infra/keycloak/README.md | 85 ++++++++++ infra/keycloak/realms/learnstack-hub.json | 110 +++++++++++++ infra/keycloak/realms/learnstack.json | 154 ++++++++++++++++++ 7 files changed, 453 insertions(+), 21 deletions(-) create mode 100644 infra/compose/postgres-init/01-create-keycloak-db.sql create mode 100644 infra/keycloak/README.md create mode 100644 infra/keycloak/realms/learnstack-hub.json create mode 100644 infra/keycloak/realms/learnstack.json diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 627b72d..7f90559 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -25,9 +25,12 @@ > (binary `readyz` healthcheck), Meilisearch — pinned tags, healthchecks, > named volumes, dev-only credential banners. > -> **Packet 4 — Identity stack (pending)** -> Keycloak two realms (`learnstack` + `learnstack-hub`) in dev compose, seed -> realm config. +> **Packet 4 — Identity stack ✅** +> Keycloak 26 in dev compose with two realms imported on first boot +> (`learnstack` tenant-facing + `learnstack-hub` operator), each hard-isolated +> per ADR-0004 Amendment 1. Realm seeds at `infra/keycloak/realms/`; Postgres +> init script provisions the Keycloak DB on the first start of the +> `postgres-data` volume. > > **Packet 5 — Live media (pending)** > LiveKit OSS + Coturn in dev compose. diff --git a/infra/compose/README.md b/infra/compose/README.md index 4179bbd..233d944 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -3,9 +3,13 @@ Compose stacks for local development. Operational rules live in [docs/standards/12-infrastructure.md](../../docs/standards/12-infrastructure.md). -## `dev.yml` (this packet) +## `dev.yml` -Core data-plane services every backend dev needs from day one: +Services in order they appear in `dev.yml` (data plane → identity → media → +eventing+secrets → gateway). Packets 1-4 shipped; 5-6 land in subsequent +Phase-01 packets. + +### Data plane (Phase 01 packet 3) | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| @@ -15,6 +19,25 @@ Core data-plane services every backend dev needs from day one: | Mailpit | `axllent/mailpit:v1.21` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | | Meilisearch | `getmeili/meilisearch:v1.11` | `localhost:7700` | master key `learnstack-dev-master-key` | +### Identity (Phase 01 packet 4) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| Keycloak | `quay.io/keycloak/keycloak:26.0` | `localhost:8080` | master admin `admin` / `admin-dev-secret` | + +Two realms imported on first boot from `infra/keycloak/realms/`: + +- `learnstack` (tenant users) — clients `learnstack-api` (confidential) + `learnstack-web` (public PKCE); demo users `demo-admin@tenant-a.test` and `demo-learner@tenant-a.test` (both `demo-dev-secret`). +- `learnstack-hub` (operators) — client `learnstack-hub-web` (public PKCE); demo user `demo-operator@learnstack.test` (`demo-dev-secret`). `CONFIGURE_TOTP` required-action present so the MFA flow surfaces in dev. + +See [../keycloak/README.md](../keycloak/README.md) for the realm-isolation +invariant, re-seed procedure, and the Phase 02b/03 wiring notes. + +The Postgres init script at `postgres-init/01-create-keycloak-db.sql` creates +the `keycloak` database on the first start of the `postgres-data` volume. +Re-seeding the realm structure requires either `down -v` (wipes all volumes) +or a manual `DROP DATABASE keycloak; CREATE DATABASE keycloak OWNER learnstack;`. + ```bash docker compose -f infra/compose/dev.yml up -d docker compose -f infra/compose/dev.yml ps # confirm healthchecks pass @@ -36,20 +59,16 @@ Tenant isolation in object storage is enforced by **key prefix** Storage Operations and [docs/architecture/16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). A single bucket per environment is created at first use. -## What this file deliberately does NOT bring up +## What this file deliberately does NOT bring up yet Per the [Phase 01 plan](../../docs/roadmap/phase-01-repository-tooling.md), later packets land: -- **Keycloak** (two realms: `learnstack` + `learnstack-hub`) -- **LiveKit OSS** + **Coturn** -- **Kafka** + **kafka-ui** (Dapr pub/sub backend) -- **Vault** (Dapr secret store, dev mode) -- **Dapr sidecar** + **placement service** -- **APISIX** (standalone YAML-reload) + **apisix-dashboard** - -These add their own healthchecks and volumes when they ship; this file stays -small in the interim to keep `make dev` cold-start fast. +- **LiveKit OSS** + **Coturn** (Phase 01 packet 5) +- **Kafka** + **kafka-ui** (Dapr pub/sub backend; Phase 01 packet 6) +- **Vault** (Dapr secret store, dev mode; Phase 01 packet 6) +- **Dapr sidecar** + **placement service** (Phase 01 packet 6) +- **APISIX** (standalone YAML-reload) + **apisix-dashboard** (Phase 01 packet 6) A companion `e2e.yml` (same stack, tuned for end-to-end test runs) is also a -later Phase 01 deliverable. +later Phase 01 deliverable (packet 7). diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index d49c60c..b0cace2 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,9 +1,10 @@ -# LearnStack — local dev infrastructure (core services). +# LearnStack — local dev infrastructure. # -# Scope of THIS file (Phase 01 — packet 1-3): the data-plane services every -# backend dev needs from day one. Keycloak, LiveKit OSS, Coturn, Kafka, Vault, -# the Dapr sidecar, and APISIX land in subsequent Phase-01 packets per the -# roadmap (docs/roadmap/phase-01-repository-tooling.md § Local Infrastructure). +# Scope of THIS file (Phase 01 packets 1-4): the data-plane services + the +# self-hosted Keycloak identity provider (two realms). LiveKit OSS, Coturn, +# Kafka, Vault, the Dapr sidecar, and APISIX land in subsequent Phase-01 +# packets per the roadmap (docs/roadmap/phase-01-repository-tooling.md +# § Local Infrastructure). # # Operational rules: Standards 12 § Local Infrastructure + Standards 20. # Tenant isolation in MinIO is enforced by key prefix (`{tenant_id}/...`) — @@ -25,6 +26,9 @@ services: - "5432:5432" volumes: - postgres-data:/var/lib/postgresql/data + # First-boot init scripts (idempotent SQL). Creates the `keycloak` + # database the Keycloak service uses; see ./postgres-init/README. + - ./postgres-init:/docker-entrypoint-initdb.d:ro healthcheck: test: ["CMD-SHELL", "pg_isready -U learnstack -d learnstack"] interval: 5s @@ -108,6 +112,50 @@ services: timeout: 5s retries: 10 + # ---- Identity stack (Phase 01 packet 4) ---------------------------------- + # Self-hosted Keycloak, two realms (`learnstack` tenant-facing + `learnstack-hub` + # operator). Per ADR-0004 + ADR-0019 the realms are hard-isolated; tokens from + # one realm are never accepted on the other realm's endpoints. See + # ../keycloak/README.md. + keycloak: + image: quay.io/keycloak/keycloak:26.0 + container_name: learnstack-keycloak + restart: unless-stopped + command: ["start-dev", "--import-realm"] + environment: + # Dev-only. Production credentials come from Vault via ISecretProvider. + KEYCLOAK_ADMIN: admin + KEYCLOAK_ADMIN_PASSWORD: admin-dev-secret + KC_DB: postgres + KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak + KC_DB_USERNAME: learnstack + KC_DB_PASSWORD: learnstack + KC_HOSTNAME_STRICT: "false" + KC_HOSTNAME_STRICT_HTTPS: "false" + KC_HTTP_ENABLED: "true" + KC_HEALTH_ENABLED: "true" + KC_METRICS_ENABLED: "true" + KC_FEATURES: token-exchange,admin-fine-grained-authz + ports: + - "8080:8080" + volumes: + # Realm seed JSONs (learnstack + learnstack-hub) — Keycloak imports them + # at first start when the realms do not yet exist in the DB. + - ../keycloak/realms:/opt/keycloak/data/import:ro + depends_on: + postgres: + condition: service_healthy + healthcheck: + # Keycloak 26 exposes /health/ready on the dedicated management + # interface (port 9000) when KC_HEALTH_ENABLED=true. The image is + # UBI-9-minimal based and ships curl; this is the upstream-recommended + # readiness pattern. + test: ["CMD-SHELL", "curl -fsS http://localhost:9000/health/ready || exit 1"] + interval: 15s + timeout: 10s + retries: 20 + start_period: 60s + volumes: postgres-data: redis-data: diff --git a/infra/compose/postgres-init/01-create-keycloak-db.sql b/infra/compose/postgres-init/01-create-keycloak-db.sql new file mode 100644 index 0000000..d28e80c --- /dev/null +++ b/infra/compose/postgres-init/01-create-keycloak-db.sql @@ -0,0 +1,13 @@ +-- Postgres init script — runs ONCE on a fresh `postgres-data` volume. +-- Creates the `keycloak` database the Keycloak service uses to store its +-- realm + user state. Keycloak shares the `learnstack` Postgres role + the +-- same Postgres instance (dev-only convenience; production isolates Keycloak +-- in its own Postgres cluster per Standards 12 § Database Operations). +-- +-- If `postgres-data` already exists, this script does NOT re-run; either +-- `docker compose -f infra/compose/dev.yml down -v` to wipe + reseed, or +-- manually `CREATE DATABASE keycloak OWNER learnstack;` once. + +SELECT 'CREATE DATABASE keycloak OWNER learnstack' +WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'keycloak') +\gexec diff --git a/infra/keycloak/README.md b/infra/keycloak/README.md new file mode 100644 index 0000000..d7014ef --- /dev/null +++ b/infra/keycloak/README.md @@ -0,0 +1,85 @@ +# Keycloak — Two-Realm Identity (Dev) + +Self-hosted Keycloak in dev compose, per +[ADR-0004 (Authentication Strategy)](../../docs/decisions/0004-authentication-strategy.md) +and [ADR-0019 (LearnStack Hub)](../../docs/decisions/0019-learnstack-hub.md). Two +realms, **hard-isolated from each other** — neither realm trusts tokens issued by +the other. This is an architectural invariant, not a configurable policy. + +## Realms + +| Realm | Purpose | User population | Clients | Demo users | +|-------|---------|-----------------|---------|------------| +| `learnstack` | Tenant-facing: admins, instructors, learners | All customer users | `learnstack-api` (confidential, service-account) + `learnstack-web` (public PKCE) | `demo-admin@tenant-a.test`, `demo-learner@tenant-a.test` (both `demo-dev-secret`) | +| `learnstack-hub` | Operator-facing: platform admin, support, billing-viewer | LearnStack staff only | `learnstack-hub-web` (public PKCE) | `demo-operator@learnstack.test` (`demo-dev-secret`) | + +## Access + +- **Admin console** — http://localhost:8080 (Keycloak Admin UI) +- **Master admin** — `admin` / `admin-dev-secret` +- **Tenant realm OIDC discovery** — http://localhost:8080/realms/learnstack/.well-known/openid-configuration +- **Hub realm OIDC discovery** — http://localhost:8080/realms/learnstack-hub/.well-known/openid-configuration + +## Dev credentials are dev credentials + +Every secret in this directory (master admin password, client secret, demo user +passwords) is **dev-only**. Production deployments load these from Vault via +`ISecretProvider` per [Standards 12 § Secrets Management](../../docs/standards/12-infrastructure.md) +and [Standards 20](../../docs/standards/20-infrastructure-stack.md). Do not reuse +any of these strings outside local Docker. + +## Two realms, zero cross-trust + +The realm separation is a **hard architectural invariant** per +[ADR-0004 Amendment 1](../../docs/decisions/0004-authentication-strategy.md#amendment-1): + +- A `learnstack-hub` token MUST be rejected on every tenant-facing endpoint + (the gateway + the backend both check the `iss` claim against the realm URL). +- A `learnstack` token MUST be rejected on every `/api/internal/*` endpoint + (the Hub-internal contract gates on mTLS + signed JWT, not on user tokens). +- An operator account cannot also be a tenant user under the same identity, and + vice versa. + +The Phase 02b OIDC integration enforces this in code; the dev seed mirrors it +in data (no shared users, no shared client IDs). + +## MFA + +- `learnstack-hub` realm: `CONFIGURE_TOTP` declared as a required action so the + TOTP enrolment flow is visible in dev; production enforces MFA mandatory for + every operator per ADR-0004 Amendment 1. +- `learnstack` realm: MFA is **optional in dev**; tenant policy may make it + mandatory per-tenant in Phase 03. + +## How the realms are seeded + +Keycloak boots with `start-dev --import-realm`. The JSON files under +`realms/` are bind-mounted to `/opt/keycloak/data/import/` and consumed once +at first start. To re-seed: + +```bash +docker compose -f infra/compose/dev.yml down keycloak +docker compose -f infra/compose/dev.yml up -d keycloak +``` + +Re-import overwrites the realm only if the realm did NOT already exist (Keycloak +default behaviour). To force a clean re-seed of the realm itself, wipe the +Keycloak database first: + +```bash +docker compose -f infra/compose/dev.yml exec postgres \ + psql -U learnstack -d learnstack -c "DROP DATABASE keycloak;" +docker compose -f infra/compose/dev.yml exec postgres \ + psql -U learnstack -d learnstack -c "CREATE DATABASE keycloak OWNER learnstack;" +docker compose -f infra/compose/dev.yml restart keycloak +``` + +## What does NOT live here + +- Production realm configuration — managed via terraform / keycloak-config-cli + per environment; the dev JSONs are not the source of truth for production. +- LearnStack-side identity domain (`User`, `Membership`, `Role`, `Permission`, + `Invitation`) — that's Phase 03, owned by `LearnStack.Modules.Identity`. The + realms here only define authentication; authorisation is application-side. +- OIDC code integration in the .NET API — Phase 02b wires `AddJwtBearer` and + the BFF callback handler. diff --git a/infra/keycloak/realms/learnstack-hub.json b/infra/keycloak/realms/learnstack-hub.json new file mode 100644 index 0000000..8f165c2 --- /dev/null +++ b/infra/keycloak/realms/learnstack-hub.json @@ -0,0 +1,110 @@ +{ + "realm": "learnstack-hub", + "displayName": "LearnStack Hub — Operators", + "displayNameHtml": "LearnStack Hub — Operators", + "enabled": true, + "registrationAllowed": false, + "rememberMe": false, + "verifyEmail": true, + "loginWithEmailAllowed": true, + "duplicateEmailsAllowed": false, + "resetPasswordAllowed": true, + "editUsernameAllowed": false, + "bruteForceProtected": true, + "permanentLockout": false, + "maxFailureWaitSeconds": 900, + "minimumQuickLoginWaitSeconds": 60, + "waitIncrementSeconds": 60, + "failureFactor": 10, + "accessTokenLifespan": 1800, + "ssoSessionIdleTimeout": 1800, + "ssoSessionMaxLifespan": 14400, + "sslRequired": "external", + "internationalizationEnabled": true, + "supportedLocales": ["en"], + "defaultLocale": "en", + "requiredActions": [ + { + "alias": "CONFIGURE_TOTP", + "name": "Configure OTP", + "providerId": "CONFIGURE_TOTP", + "enabled": true, + "defaultAction": false, + "priority": 10, + "config": {} + } + ], + "roles": { + "realm": [ + { + "name": "hub-platform-admin", + "description": "Full Hub platform administration: plan CRUD, tenant lifecycle, custom-domain admin, license issuance.", + "composite": false, + "clientRole": false + }, + { + "name": "hub-operator", + "description": "Day-to-day Hub operator: tenant support, entitlement overrides, audit review.", + "composite": false, + "clientRole": false + }, + { + "name": "hub-billing-viewer", + "description": "Read-only access to billing + invoice surfaces in the operator portal.", + "composite": false, + "clientRole": false + } + ] + }, + "clients": [ + { + "clientId": "learnstack-hub-web", + "name": "LearnStack Hub Web (public PKCE)", + "description": "learnstack-hub-web operator portal — public client, PKCE-enforced.", + "enabled": true, + "publicClient": true, + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": false, + "serviceAccountsEnabled": false, + "redirectUris": [ + "http://localhost:3100/*" + ], + "webOrigins": ["http://localhost:3100"], + "protocol": "openid-connect", + "attributes": { + "pkce.code.challenge.method": "S256", + "post.logout.redirect.uris": "http://localhost:3100/*" + }, + "fullScopeAllowed": true + } + ], + "users": [ + { + "username": "demo-operator@learnstack.test", + "email": "demo-operator@learnstack.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Operator", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["hub-operator"] + } + ], + "smtpServer": { + "host": "mailpit", + "port": "1025", + "from": "no-reply@hub.learnstack.dev", + "fromDisplayName": "LearnStack Hub Dev", + "replyTo": "no-reply@hub.learnstack.dev", + "ssl": "false", + "starttls": "false", + "auth": "false" + } +} diff --git a/infra/keycloak/realms/learnstack.json b/infra/keycloak/realms/learnstack.json new file mode 100644 index 0000000..220a5b2 --- /dev/null +++ b/infra/keycloak/realms/learnstack.json @@ -0,0 +1,154 @@ +{ + "realm": "learnstack", + "displayName": "LearnStack — Tenant Users", + "displayNameHtml": "LearnStack — Tenant Users", + "enabled": true, + "registrationAllowed": true, + "registrationEmailAsUsername": true, + "rememberMe": true, + "verifyEmail": false, + "loginWithEmailAllowed": true, + "duplicateEmailsAllowed": false, + "resetPasswordAllowed": true, + "editUsernameAllowed": false, + "bruteForceProtected": true, + "permanentLockout": false, + "maxFailureWaitSeconds": 900, + "minimumQuickLoginWaitSeconds": 60, + "waitIncrementSeconds": 60, + "quickLoginCheckMilliSeconds": 1000, + "maxDeltaTimeSeconds": 43200, + "failureFactor": 30, + "accessTokenLifespan": 3600, + "accessTokenLifespanForImplicitFlow": 900, + "ssoSessionIdleTimeout": 1800, + "ssoSessionMaxLifespan": 36000, + "offlineSessionIdleTimeout": 2592000, + "accessCodeLifespan": 60, + "accessCodeLifespanUserAction": 300, + "accessCodeLifespanLogin": 1800, + "actionTokenGeneratedByAdminLifespan": 43200, + "actionTokenGeneratedByUserLifespan": 300, + "sslRequired": "external", + "internationalizationEnabled": true, + "supportedLocales": ["en", "tr"], + "defaultLocale": "en", + "roles": { + "realm": [ + { + "name": "tenant-admin", + "description": "Tenant administrator — manages users, content, billing within a tenant.", + "composite": false, + "clientRole": false + }, + { + "name": "tenant-instructor", + "description": "Instructor — authors content + runs live sessions within a tenant.", + "composite": false, + "clientRole": false + }, + { + "name": "tenant-learner", + "description": "Learner — enrols in courses + attends sessions within a tenant.", + "composite": false, + "clientRole": false + } + ] + }, + "clients": [ + { + "clientId": "learnstack-api", + "name": "LearnStack API (confidential service)", + "description": "Backend .NET API — confidential client with service-account + standard + direct grants for dev.", + "enabled": true, + "publicClient": false, + "secret": "learnstack-api-dev-secret", + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": true, + "serviceAccountsEnabled": true, + "redirectUris": [ + "http://localhost:5080/*", + "http://localhost:3000/*" + ], + "webOrigins": ["+"], + "protocol": "openid-connect", + "attributes": { + "access.token.lifespan": "3600", + "pkce.code.challenge.method": "S256" + }, + "fullScopeAllowed": true + }, + { + "clientId": "learnstack-web", + "name": "LearnStack Web (public PKCE)", + "description": "Next.js apps/web — public client, PKCE-enforced, no client secret.", + "enabled": true, + "publicClient": true, + "standardFlowEnabled": true, + "implicitFlowEnabled": false, + "directAccessGrantsEnabled": false, + "serviceAccountsEnabled": false, + "redirectUris": [ + "http://localhost:3000/*" + ], + "webOrigins": ["http://localhost:3000"], + "protocol": "openid-connect", + "attributes": { + "pkce.code.challenge.method": "S256", + "post.logout.redirect.uris": "http://localhost:3000/*" + }, + "fullScopeAllowed": true + } + ], + "users": [ + { + "username": "demo-admin@tenant-a.test", + "email": "demo-admin@tenant-a.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Admin", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["tenant-admin"], + "attributes": { + "tenant_id": ["tenant-a"] + } + }, + { + "username": "demo-learner@tenant-a.test", + "email": "demo-learner@tenant-a.test", + "emailVerified": true, + "enabled": true, + "firstName": "Demo", + "lastName": "Learner", + "credentials": [ + { + "type": "password", + "value": "demo-dev-secret", + "temporary": false + } + ], + "realmRoles": ["tenant-learner"], + "attributes": { + "tenant_id": ["tenant-a"] + } + } + ], + "smtpServer": { + "host": "mailpit", + "port": "1025", + "from": "no-reply@learnstack.dev", + "fromDisplayName": "LearnStack Dev", + "replyTo": "no-reply@learnstack.dev", + "ssl": "false", + "starttls": "false", + "auth": "false" + } +} From be8a10ec529b96cefb2baf6014d5b1df73ed189b Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 17:27:05 +0300 Subject: [PATCH 02/11] =?UTF-8?q?feat(infra):=20Phase=2001=20packet=205=20?= =?UTF-8?q?=E2=80=94=20LiveKit=20OSS=20+=20Coturn=20live-media=20stack?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Stands up the self-hosted live-classroom media plane in dev compose, per ADR-0005 (self-hosted LiveKit OSS is the default supported path). The .NET app never imports the LiveKit server SDK directly — Phase 08c wires the `ILiveClassProvider` abstraction; this packet only ships the runtime LiveKit can hand back tokens against. Adds - `infra/compose/dev.yml` services `livekit` (port 7880 WS signaling, 7881 TCP fallback, 7882 TURN/TLS, 50000-50100/udp media plane) and `coturn` (3478 STUN/TURN, 5349 TURN/TLS, 49152-49200/udp relay range narrowed for dev). - `infra/livekit/livekit.yaml` with the single dev key/secret pair the Phase 08c token issuer will sign with (`devkey` / `devsecret-32-byte-min-length-padding-xyz` — padded to satisfy LiveKit's ≥ 32-byte hard requirement), TURN enabled against the Coturn ports, auto-create + 300s empty-timeout room policy. - `infra/coturn/turnserver.conf` with static long-term credentials (`devuser` / `devsecret`) — fine for a developer poking at TURN with `turnutils_uclient`, production switches to use-auth-secret ephemeral credentials. - `infra/livekit/README.md` documenting the access endpoints, dev credentials, the Phase 08c integration plan, recording + consent + cost-tracking story, and the "what does NOT live here" boundary. Docs - `infra/compose/README.md` gains a "Live media (Phase 01 packet 5)" section; the "does NOT bring up yet" list shrinks by two entries. - `docs/roadmap/phase-01-repository-tooling.md` packet-5 status flipped to ✅. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - YAML parsers accept dev.yml + livekit.yaml. - Markdown link sweep on changed docs clean. - Compose-up smoke test deferred to the packet 6 bundle (LiveKit + Coturn alone don't exercise the eventing/secrets/gateway dependencies yet). Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/roadmap/phase-01-repository-tooling.md | 6 +- infra/compose/README.md | 16 ++++- infra/compose/dev.yml | 51 +++++++++++++-- infra/coturn/turnserver.conf | 38 ++++++++++++ infra/livekit/README.md | 69 +++++++++++++++++++++ infra/livekit/livekit.yaml | 44 +++++++++++++ 6 files changed, 214 insertions(+), 10 deletions(-) create mode 100644 infra/coturn/turnserver.conf create mode 100644 infra/livekit/README.md create mode 100644 infra/livekit/livekit.yaml diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 7f90559..4112662 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -32,8 +32,10 @@ > init script provisions the Keycloak DB on the first start of the > `postgres-data` volume. > -> **Packet 5 — Live media (pending)** -> LiveKit OSS + Coturn in dev compose. +> **Packet 5 — Live media ✅** +> LiveKit OSS v1.8.0 + Coturn 4.6 in dev compose with the dev key/secret pair +> the eventual `ILiveClassProvider` adapter (Phase 08c) will sign tokens with. +> Configs at `infra/livekit/livekit.yaml` and `infra/coturn/turnserver.conf`. > > **Packet 6 — Eventing + secrets + gateway (pending)** > Kafka + kafka-ui, Vault (dev mode), Dapr sidecar + placement, APISIX diff --git a/infra/compose/README.md b/infra/compose/README.md index 233d944..8819c66 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -6,8 +6,7 @@ Compose stacks for local development. Operational rules live in ## `dev.yml` Services in order they appear in `dev.yml` (data plane → identity → media → -eventing+secrets → gateway). Packets 1-4 shipped; 5-6 land in subsequent -Phase-01 packets. +eventing+secrets → gateway). Packets 1-5 shipped; packet 6 lands next. ### Data plane (Phase 01 packet 3) @@ -38,6 +37,18 @@ the `keycloak` database on the first start of the `postgres-data` volume. Re-seeding the realm structure requires either `down -v` (wipes all volumes) or a manual `DROP DATABASE keycloak; CREATE DATABASE keycloak OWNER learnstack;`. +### Live media (Phase 01 packet 5) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| LiveKit OSS | `livekit/livekit-server:v1.8.0` | `ws://localhost:7880` (signaling), `tcp/7881` (TCP fallback), `tcp/7882` (TURN/TLS), `udp/50000-50100` (media) | API key `devkey` / secret `devsecret-32-byte-min-length-padding-xyz` | +| Coturn | `coturn/coturn:4.6` | `udp+tcp/3478` (STUN/TURN), `tcp/5349` (TURN/TLS), `udp/49152-49200` (relay range) | TURN user `devuser` / password `devsecret` | + +LiveKit config at `infra/livekit/livekit.yaml`; Coturn config at +`infra/coturn/turnserver.conf`. See [../livekit/README.md](../livekit/README.md) +for the `ILiveClassProvider` integration plan (Phase 08c) + the +recording / consent / cost-tracking story. + ```bash docker compose -f infra/compose/dev.yml up -d docker compose -f infra/compose/dev.yml ps # confirm healthchecks pass @@ -64,7 +75,6 @@ A single bucket per environment is created at first use. Per the [Phase 01 plan](../../docs/roadmap/phase-01-repository-tooling.md), later packets land: -- **LiveKit OSS** + **Coturn** (Phase 01 packet 5) - **Kafka** + **kafka-ui** (Dapr pub/sub backend; Phase 01 packet 6) - **Vault** (Dapr secret store, dev mode; Phase 01 packet 6) - **Dapr sidecar** + **placement service** (Phase 01 packet 6) diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index b0cace2..35c8f56 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,10 +1,10 @@ # LearnStack — local dev infrastructure. # -# Scope of THIS file (Phase 01 packets 1-4): the data-plane services + the -# self-hosted Keycloak identity provider (two realms). LiveKit OSS, Coturn, -# Kafka, Vault, the Dapr sidecar, and APISIX land in subsequent Phase-01 -# packets per the roadmap (docs/roadmap/phase-01-repository-tooling.md -# § Local Infrastructure). +# Scope of THIS file (Phase 01 packets 1-5): the data-plane services, the +# self-hosted Keycloak identity provider (two realms), and the live media +# stack (LiveKit OSS + Coturn). Kafka, Vault, the Dapr sidecar, and APISIX +# land in subsequent Phase-01 packets per the roadmap +# (docs/roadmap/phase-01-repository-tooling.md § Local Infrastructure). # # Operational rules: Standards 12 § Local Infrastructure + Standards 20. # Tenant isolation in MinIO is enforced by key prefix (`{tenant_id}/...`) — @@ -156,6 +156,47 @@ services: retries: 20 start_period: 60s + # ---- Live media stack (Phase 01 packet 5) -------------------------------- + # Self-hosted LiveKit OSS + Coturn for the in-app classroom (ADR-0005). + # The .NET app talks to LiveKit through the `ILiveClassProvider` abstraction + # (Phase 08c); the LiveKit SDK is never imported by any module — only by + # `LearnStack.Infrastructure.LiveClassroom.LiveKit`. + livekit: + image: livekit/livekit-server:v1.8.0 + container_name: learnstack-livekit + restart: unless-stopped + command: ["--config", "/etc/livekit.yaml", "--node-ip", "127.0.0.1", "--dev"] + volumes: + - ../livekit/livekit.yaml:/etc/livekit.yaml:ro + ports: + - "7880:7880" # WebSocket signaling + - "7881:7881" # TCP fallback + - "7882:7882/tcp" # TURN/TLS + - "50000-50100:50000-50100/udp" # WebRTC media plane + depends_on: + coturn: + condition: service_started + healthcheck: + # LiveKit serves a default 200 on `/` once the WebSocket listener is up. + # The image is alpine-based and ships wget. + test: ["CMD", "wget", "-qO-", "http://localhost:7880"] + interval: 10s + timeout: 5s + retries: 10 + + coturn: + image: coturn/coturn:4.6 + container_name: learnstack-coturn + restart: unless-stopped + command: ["-c", "/etc/turnserver.conf"] + volumes: + - ../coturn/turnserver.conf:/etc/turnserver.conf:ro + ports: + - "3478:3478/udp" # STUN/TURN (UDP) + - "3478:3478/tcp" # STUN/TURN (TCP) + - "5349:5349/tcp" # TURN/TLS + - "49152-49200:49152-49200/udp" # TURN relay range (dev: narrow) + volumes: postgres-data: redis-data: diff --git a/infra/coturn/turnserver.conf b/infra/coturn/turnserver.conf new file mode 100644 index 0000000..e18f7ea --- /dev/null +++ b/infra/coturn/turnserver.conf @@ -0,0 +1,38 @@ +# Coturn — dev TURN/STUN server. +# Provides STUN reflexive-address lookup and TURN media relay for LiveKit +# clients behind symmetric NATs. Production uses a dedicated Coturn cluster +# with TLS certs, ephemeral credentials, and dedicated relay IP ranges; this +# file is dev-only. + +listening-port=3478 +tls-listening-port=5349 + +# Narrow relay port range for dev (production uses 49152-65535 or wider). +min-port=49152 +max-port=49200 + +# Dev realm. Production sets the actual hostname (e.g. turn.learnstack.dev) +# matched by the issued TLS cert. +realm=localhost + +# Long-term credential mechanism (LiveKit issues per-session credentials in +# production via the shared-secret pattern — but dev uses a static user/secret +# pair so a developer can poke at TURN with `turnutils_uclient` without +# bouncing through LiveKit). +lt-cred-mech + +# Dev-only static credentials. Production credentials come from Vault via +# ISecretProvider; relay-side use auth-secret + use-auth-secret for ephemeral +# tokens. +user=devuser:devsecret + +# Network hygiene. +no-multicast-peers +mobility +no-cli +no-tlsv1 +no-tlsv1_1 + +# Logging to stdout for `docker compose logs coturn`. +log-file=stdout +verbose diff --git a/infra/livekit/README.md b/infra/livekit/README.md new file mode 100644 index 0000000..17954b9 --- /dev/null +++ b/infra/livekit/README.md @@ -0,0 +1,69 @@ +# LiveKit OSS + Coturn (Dev) + +Self-hosted live-classroom media stack per +[ADR-0005 (Live Classroom Media Stack)](../../docs/decisions/0005-live-classroom-media-stack.md). +The .NET application talks to LiveKit through the `ILiveClassProvider` +abstraction (Phase 08c); the LiveKit server SDK is never imported by any +LearnStack module — only by `LearnStack.Infrastructure.LiveClassroom.LiveKit`. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| WebSocket signaling | `ws://localhost:7880` | LiveKit client connects here for room join + signaling | +| TCP fallback | `tcp://localhost:7881` | WebRTC TCP fallback when UDP is blocked | +| TURN/TLS | `tcp://localhost:7882` | Encrypted TURN listener (dev: no real cert) | +| WebRTC media | UDP `50000-50100` | LiveKit media plane | +| STUN/TURN (Coturn) | `udp/tcp://localhost:3478` | NAT reflexive-address + media relay | +| TURN/TLS (Coturn) | `tcp://localhost:5349` | Encrypted TURN listener | +| Relay range (Coturn) | UDP `49152-49200` | TURN relay port range (dev: narrowed from production) | + +## Dev API key + +LiveKit room tokens are signed with a single dev key/secret pair declared in +`infra/livekit/livekit.yaml`: + +| Key | Secret | +|-----|--------| +| `devkey` | `devsecret-32-byte-min-length-padding-xyz` | + +The secret padding satisfies LiveKit's >= 32-byte requirement. Production +key/secret pairs come from Vault via `ISecretProvider` per +[Standards 12 § Secrets Management](../../docs/standards/12-infrastructure.md); +do not reuse these strings. + +## Dev Coturn credentials + +| User | Password | +|------|----------| +| `devuser` | `devsecret` | + +These are **static long-term credentials** — fine for a developer poking at +TURN with `turnutils_uclient`, but production switches to the +`use-auth-secret` shared-secret pattern so LiveKit can issue per-session +ephemeral credentials. + +## How LearnStack uses LiveKit + +The full integration arrives in Phase 08c. Summary: + +- **`ILiveClassProvider`** abstraction in `LearnStack.Application.Contracts`; + LiveKit-specific implementation in `LearnStack.Infrastructure.LiveClassroom.LiveKit`. +- **Token issuance** happens server-side from `learnstack-api` using the dev + key above; tokens scoped per `(tenant_id, session_id, user_id)` with a + short TTL. +- **Recording** consumes LiveKit Egress (separate service, lands in + Phase 08c) writing to MinIO via the existing storage provider abstraction. + Recording is **tenant-configurable** and **consent-aware** per ADR-0005 + + [16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). +- **Cost metrics** (participant minutes, bandwidth, recording minutes) + surface in the Phase 09 analytics pipeline. + +## What does NOT live here + +- LiveKit Egress (recording) — Phase 08c. +- Production TLS cert provisioning — ADR-0022 + the same Let's Encrypt + adapter family APISIX uses. +- Custom WebRTC SFU — explicitly out of scope per ADR-0005 + ADR-0018. +- Cloud LiveKit (managed) — supported as a swap-in path through the same + `ILiveClassProvider` adapter; dev defaults to self-hosted OSS. diff --git a/infra/livekit/livekit.yaml b/infra/livekit/livekit.yaml new file mode 100644 index 0000000..e4f22ed --- /dev/null +++ b/infra/livekit/livekit.yaml @@ -0,0 +1,44 @@ +# LiveKit OSS — dev config. +# Per ADR-0005, LiveKit is the preferred SFU; self-hosted OSS is the default +# supported path. The .NET application talks to LiveKit through the +# `ILiveClassProvider` abstraction (Phase 08c) — never imports the LiveKit +# server SDK directly. +# +# Dev-only keys + secrets. Production replaces these with values fetched +# from Vault via ISecretProvider per Standards 12 § Secrets Management. + +port: 7880 + +rtc: + tcp_port: 7881 + # Dev: machine has no stable external IP; bind to loopback for now. + # Production sets `use_external_ip: true` so LiveKit auto-discovers the + # node's public IP for ICE candidates. + use_external_ip: false + port_range_start: 50000 + port_range_end: 50100 + +# Dev API key + secret. The secret MUST be >= 32 bytes (LiveKit hard rule); +# the value below is padded to satisfy that and is checked into git only +# because this file is the dev compose config. +keys: + devkey: devsecret-32-byte-min-length-padding-xyz + +turn: + enabled: true + domain: localhost + tls_port: 7882 + udp_port: 3478 + # No TLS cert in dev — LiveKit OSS supports plain TURN listeners on the + # configured ports without a cert pair. Production wires Let's Encrypt + # through the same provider adapter family APISIX uses (ADR-0022). + +room: + auto_create: true + empty_timeout: 300 + departure_timeout: 20 + max_participants: 50 + +logging: + level: info + json: false From 4243f3a5d1035e994b5a2af4de2698c3e78471e6 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 17:32:29 +0300 Subject: [PATCH 03/11] =?UTF-8?q?feat(infra):=20Phase=2001=20packet=206=20?= =?UTF-8?q?=E2=80=94=20Kafka=20+=20Vault=20+=20Dapr=20+=20APISIX?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Brings up every cross-cutting runtime ADR-0014 and ADR-0015 commit the platform to, in one packet so the Phase-02a sockets (`IEventBus`, `ICacheService`, `ISecretProvider`, `IHostToTenantResolver`) can be wired against a real backend the moment Phase 02a starts. Adds (compose services) - `kafka` (KRaft mode, no ZooKeeper) — `confluentinc/cp-kafka:7.8.0`, single-node broker+controller, stable cluster id so the log dir survives restarts, healthchecked via `kafka-topics --list`. - `kafka-ui` (`provectuslabs/kafka-ui:v0.7.2`) on `localhost:8081`. - `vault` (`hashicorp/vault:1.18`) in `-dev` mode with root token `learnstack-dev-root-token`, `IPC_LOCK` cap. - `dapr-placement` + `dapr-sidecar-api` (`daprio/...:1.14.4`) — sidecar app id `learnstack-api`, targets `host.docker.internal:5080` since the .NET host runs OUTSIDE the compose network during active dev. - `apisix` (`apache/apisix:3.10.0-debian`) standalone YAML-reload mode, ports 9080 (HTTP), 9180 (admin), 9443 (HTTPS), 9091 (metrics); `extra_hosts: host.docker.internal:host-gateway` for Linux developers. - `apisix-dashboard` (`apache/apisix-dashboard:3.0.1-alpine`). - `kafka-data` named volume. Adds (configs) - `infra/dapr/components/pubsub-kafka.yaml` — Dapr pub/sub → Kafka. - `infra/dapr/components/statestore-redis.yaml` — Dapr state → Redis. - `infra/dapr/components/secretstore-vault.yaml` — Dapr secrets → Vault. - `infra/dapr/config/dapr-config.yaml` — sampling rate 1, tracing endpoint empty (Phase 11 wires Tempo). - `infra/dapr/README.md` — sidecar topology, application access pattern, what does/does NOT live here. - `infra/apisix/config.yaml` — standalone mode declaration, plugin universe (cors / openid-connect / limit-req / request-id / prometheus / mtls reserved for Phase 02c). - `infra/apisix/apisix.yaml` — three live routes (`/healthz`, `/api/v*/**` OPTIONS preflight, `/api/v*/**` authenticated) all upstreaming to `host.docker.internal:5080`; `/api/internal/*` mTLS route stubbed as a comment block for Phase 02c. - `infra/apisix/dashboard.yaml` — dev-only dashboard auth + allowlist. - `infra/apisix/README.md` — plugin chain walkthrough, route table, upstream addressing rationale, dev credentials, what does/does NOT live here. Docs - `infra/compose/README.md` regrouped to data plane → identity → media → eventing+secrets+Dapr+gateway; "does NOT bring up yet" list shrinks to packets 7-8 only. - `docs/roadmap/phase-01-repository-tooling.md` packet-6 status flipped to ✅. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - Every YAML in the diff (compose, 3 Dapr components, Dapr config, 3 APISIX configs, LiveKit config) parses through PyYAML. - Markdown link sweep on changed docs clean. - Full compose-up smoke test not run in this commit (Keycloak + Kafka cold start alone is ~90s; the DX packet's `make dev` orchestrator will surface a polished startup story). Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/roadmap/phase-01-repository-tooling.md | 10 +- infra/apisix/README.md | 91 ++++++++++ infra/apisix/apisix.yaml | 93 ++++++++++ infra/apisix/config.yaml | 52 ++++++ infra/apisix/dashboard.yaml | 32 ++++ infra/compose/README.md | 49 ++++- infra/compose/dev.yml | 177 ++++++++++++++++++- infra/dapr/README.md | 80 +++++++++ infra/dapr/components/pubsub-kafka.yaml | 23 +++ infra/dapr/components/secretstore-vault.yaml | 23 +++ infra/dapr/components/statestore-redis.yaml | 21 +++ infra/dapr/config/dapr-config.yaml | 14 ++ 12 files changed, 649 insertions(+), 16 deletions(-) create mode 100644 infra/apisix/README.md create mode 100644 infra/apisix/apisix.yaml create mode 100644 infra/apisix/config.yaml create mode 100644 infra/apisix/dashboard.yaml create mode 100644 infra/dapr/README.md create mode 100644 infra/dapr/components/pubsub-kafka.yaml create mode 100644 infra/dapr/components/secretstore-vault.yaml create mode 100644 infra/dapr/components/statestore-redis.yaml create mode 100644 infra/dapr/config/dapr-config.yaml diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 4112662..46ce22f 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -37,9 +37,13 @@ > the eventual `ILiveClassProvider` adapter (Phase 08c) will sign tokens with. > Configs at `infra/livekit/livekit.yaml` and `infra/coturn/turnserver.conf`. > -> **Packet 6 — Eventing + secrets + gateway (pending)** -> Kafka + kafka-ui, Vault (dev mode), Dapr sidecar + placement, APISIX -> standalone YAML-reload + dashboard, `infra/apisix/config.yaml`. +> **Packet 6 — Eventing + secrets + gateway ✅** +> Kafka 7.8 in KRaft mode (no ZooKeeper) + kafka-ui, Vault 1.18 in `-dev` mode, +> Dapr 1.14.4 sidecar (`learnstack-api` app id) + placement with three +> components — `pubsub-kafka.yaml`, `statestore-redis.yaml`, +> `secretstore-vault.yaml` per ADR-0014 — and APISIX 3.10 in standalone +> YAML-reload mode + dashboard per ADR-0015 (`infra/apisix/{config,apisix,dashboard}.yaml`, +> mTLS plugin slot reserved for the Phase-02c Hub internal API surface). > > **Packet 7 — Developer experience (pending)** > `Makefile` (`make dev` / `test` / `lint` / `seed`), `.env.example` per app, diff --git a/infra/apisix/README.md b/infra/apisix/README.md new file mode 100644 index 0000000..22274fa --- /dev/null +++ b/infra/apisix/README.md @@ -0,0 +1,91 @@ +# APISIX Gateway (Dev) + +LearnStack's only tenant-facing ingress, per +[ADR-0015 (API Gateway: APISIX)](../../docs/decisions/0015-api-gateway-apisix.md). +Standalone YAML-reload mode — no etcd, no admin-UI-driven config drift; the +files in this directory are the source of truth. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| HTTP gateway | `http://localhost:9080` | Public + authenticated routes (per `apisix.yaml`) | +| HTTPS gateway | `https://localhost:9443` | TLS-terminated routes (no cert in dev) | +| Admin API | `http://localhost:9180` | Route + plugin admin (dev only; `admin` key `learnstack-dev-admin-key`) | +| Prometheus metrics | `http://localhost:9091` | Per-route + plugin metrics scraped by Prometheus (Phase 11) | +| Dashboard | `http://localhost:9000` | Browser UI (dev only; user `admin` / pass `learnstack-dev-dashboard-pass`) | + +## Plugin chain + +`infra/apisix/config.yaml` declares the universe of plugins; routes pick +from it per-request order: + +``` +real-ip → cors → openid-connect → limit-req → request-id → proxy-rewrite → upstream + ↓ + prometheus (response) +``` + +`mtls` is **reserved** — Phase 02c (LearnStack Hub) activates it against the +LearnStack-internal CA for the `/api/internal/*` surface. + +## Routes shipped today (`apisix.yaml`) + +| Priority | URI | Methods | Plugin chain | Upstream | +|----------|-----|---------|--------------|----------| +| 1 | `/healthz` | GET | `cors`, `limit-req` (10 r/s), `prometheus` | `host.docker.internal:5080` | +| 99 | `/api/v*/**` | OPTIONS | `cors` only (preflight bypass) | `host.docker.internal:5080` | +| 100 | `/api/v*/**` | GET/POST/PUT/PATCH/DELETE | `cors`, `limit-req` (100 r/s), `request-id`, `prometheus` | `host.docker.internal:5080` | + +`openid-connect` is **commented out** in route 100; Phase 03 wires the +plugin against the `learnstack` Keycloak realm discovery endpoint. The +backend re-validates the JWT (defence in depth — gateway compromise must +not bypass auth). + +The `/api/internal/*` route is a **disabled placeholder**; Phase 02c +activates it with `mtls` + `ip-restriction` against the Hub-issued client +cert. + +## Upstream addressing — why `host.docker.internal:5080` + +The .NET API host runs **outside the container network** during active +dev (`dotnet run` on the developer's workstation). Docker Desktop's +`host.docker.internal` alias resolves to the host loopback; Linux developers +without Docker Desktop need to add: + +```yaml +apisix: + extra_hosts: + - "host.docker.internal:host-gateway" +``` + +…in their local override (the DX packet (07) ships `make dev` with the +cross-platform case handled). + +When the .NET host moves inside compose in a later environment profile, +the upstream nodes shift to `learnstack-api:5080` — same `apisix.yaml`, +single one-line change. + +## Hot-reload + +APISIX watches `apisix.yaml` and re-applies the route table on file change. +A malformed YAML push takes the gateway down, so changes go through CI YAML +validation before they reach the running gateway. + +## Dev credentials are dev credentials + +Every credential in this directory (admin key, dashboard JWT secret, +dashboard user) is dev-only. Production loads them from Vault via the +component metadata pattern + restricts the admin API to an +internal-only listener. + +## What does NOT live here + +- Production TLS cert (Let's Encrypt via the same adapter family ADR-0022 + picks for custom domains) — Phase 11. +- Hub-side APISIX route block (`hub.learnstack.dev`) — Phase 02c, lives in + the separate `learnstack-hub` repo. +- The .NET `AddJwtBearer` middleware (the backend's defence-in-depth + JWT re-validation) — Phase 02b. +- Hangfire dashboard `/admin/hangfire*` gating — Phase 08a when Hangfire + itself lands. diff --git a/infra/apisix/apisix.yaml b/infra/apisix/apisix.yaml new file mode 100644 index 0000000..6ab505a --- /dev/null +++ b/infra/apisix/apisix.yaml @@ -0,0 +1,93 @@ +# APISIX route table — hot-reloaded on file change. +# Per ADR-0015 the route table separates public from authenticated, and +# CORS preflight from the authenticated path so OPTIONS bypasses +# openid-connect. +# +# This packet ships the scaffold routes only: +# - `/healthz` (priority 1, public) — proxies to the .NET API host +# - `/api/v*/**` OPTIONS (priority 99, CORS preflight) — plugin: cors only +# - `/api/v*/**` authenticated (priority 100, all other verbs) — plugin +# chain with `openid-connect` placeholder (Keycloak realm discovery +# gets wired in Phase 03) +# - `/api/internal/*` (priority 50, mTLS) — placeholder route disabled +# (the `mtls` plugin entry is commented out; Phase 02c activates it +# once the Hub HTTPS contract surface lands) + +routes: + - id: 1 + uri: /healthz + priority: 1 + plugins: + cors: {} + limit-req: + rate: 10 + burst: 5 + key: remote_addr + rejected_code: 429 + prometheus: {} + upstream: + type: roundrobin + nodes: + # The .NET API host runs OUTSIDE compose during active dev + # (`dotnet run` on the workstation). `host.docker.internal` is + # Docker Desktop's gateway alias; Linux developers may need to + # add `--add-host=host.docker.internal:host-gateway` to the + # apisix service. Phase 07 (DX packet) sorts the cross-platform + # case alongside `make dev`. + "host.docker.internal:5080": 1 + + - id: 99 + uri: /api/v*/** + methods: [OPTIONS] + priority: 99 + plugins: + cors: {} + upstream: + type: roundrobin + nodes: + "host.docker.internal:5080": 1 + + - id: 100 + uri: /api/v*/** + methods: [GET, POST, PUT, PATCH, DELETE] + priority: 100 + plugins: + cors: {} + # openid-connect realm discovery is wired in Phase 03 against the + # learnstack Keycloak realm. Placeholder kept so the route schema + # validates today. + # openid-connect: + # discovery: http://keycloak:8080/realms/learnstack/.well-known/openid-configuration + # bearer_only: true + # realm: learnstack + limit-req: + rate: 100 + burst: 50 + key: remote_addr + rejected_code: 429 + request-id: + include_in_response: true + prometheus: {} + upstream: + type: roundrobin + nodes: + "host.docker.internal:5080": 1 + + # /api/internal/* route is RESERVED here so the plugin slot is + # documented; Phase 02c lands the Hub HTTPS contract surface and + # enables the mtls plugin against the LearnStack-internal CA. + # - id: 50 + # uri: /api/internal/* + # priority: 50 + # plugins: + # mtls: + # ca_uri: /path/to/learnstack-internal-ca.crt + # ip-restriction: + # whitelist: + # - 10.0.0.0/8 + # upstream: + # type: roundrobin + # nodes: + # "host.docker.internal:5080": 1 + +#END diff --git a/infra/apisix/config.yaml b/infra/apisix/config.yaml new file mode 100644 index 0000000..04cc342 --- /dev/null +++ b/infra/apisix/config.yaml @@ -0,0 +1,52 @@ +# APISIX main config — standalone YAML-reload mode. +# Per ADR-0015 the standalone deployment shape (no etcd) is the default; +# etcd-backed mode is reconsidered when an admin UI / dynamic config +# requirement arrives. Hot-reload watches `apisix.yaml`. + +deployment: + role: traditional + role_traditional: + config_provider: yaml + +apisix: + # Dev-only admin key. Production replaces this with a Vault-sourced value + # and only exposes the admin API on an internal-only listener. + enable_admin: true + admin_key: + - name: admin + key: learnstack-dev-admin-key + role: admin + enable_ipv6: false + proxy_mode: http + enable_dev_mode: false + +nginx_config: + error_log_level: info + worker_processes: auto + worker_rlimit_nofile: 20480 + +# Plugin chain available to routes. The actual per-route chain is declared +# in apisix.yaml; this list is the universe routes may pick from. +plugins: + - real-ip + - cors + - openid-connect # JWT validation (Phase 03 wires Keycloak realm) + - jwt-auth + - limit-req + - limit-count + - request-id + - proxy-rewrite + - response-rewrite + - prometheus + - ip-restriction + - mtls # Reserved for /api/internal/* (Phase 02c) + - gzip + +plugin_attr: + prometheus: + enable_export_server: true + export_addr: + ip: "0.0.0.0" + port: 9091 + request-id: + include_in_response: true diff --git a/infra/apisix/dashboard.yaml b/infra/apisix/dashboard.yaml new file mode 100644 index 0000000..93438b3 --- /dev/null +++ b/infra/apisix/dashboard.yaml @@ -0,0 +1,32 @@ +# APISIX Dashboard config (dev only). +# The dashboard is a developer convenience for browsing routes + plugins +# in dev mode. Production deployments either run without the dashboard +# (standalone YAML is the source of truth) or pin it behind mTLS on an +# internal-only listener. + +conf: + listen: + host: 0.0.0.0 + port: 9000 + allow_list: + - 0.0.0.0/0 # Dev-only. Production tightens to operator subnet. + etcd: + # APISIX runs in standalone mode (no etcd); the dashboard cannot + # introspect routes without it. The endpoints below are placeholders + # so the dashboard boots without erroring on missing config — the + # route list view will be empty. + endpoints: + - http://127.0.0.1:2379 + log: + error_log: + level: info + file_path: logs/error.log + access_log: + file_path: logs/access.log + +authentication: + secret: learnstack-dev-dashboard-jwt-secret + expire_time: 3600 + users: + - username: admin + password: learnstack-dev-dashboard-pass diff --git a/infra/compose/README.md b/infra/compose/README.md index 8819c66..3921302 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -6,7 +6,8 @@ Compose stacks for local development. Operational rules live in ## `dev.yml` Services in order they appear in `dev.yml` (data plane → identity → media → -eventing+secrets → gateway). Packets 1-5 shipped; packet 6 lands next. +eventing → secrets → Dapr sidecar → gateway). Packets 1-6 shipped; packets +7-8 (DX orchestrator + CI) remain. ### Data plane (Phase 01 packet 3) @@ -49,6 +50,40 @@ LiveKit config at `infra/livekit/livekit.yaml`; Coturn config at for the `ILiveClassProvider` integration plan (Phase 08c) + the recording / consent / cost-tracking story. +### Eventing + secrets + Dapr sidecar + gateway (Phase 01 packet 6) + +| Service | Image | Local endpoint | Default credentials | +|---------|-------|----------------|---------------------| +| Kafka (KRaft) | `confluentinc/cp-kafka:7.8.0` | `localhost:9092` | none (`PLAINTEXT`, `authType: none`) | +| kafka-ui | `provectuslabs/kafka-ui:v0.7.2` | `localhost:8081` | open UI (dev only) | +| Vault | `hashicorp/vault:1.18` | `localhost:8200` | root token `learnstack-dev-root-token` | +| Dapr placement | `daprio/placement:1.14.4` | `localhost:50005` | — | +| Dapr sidecar (api) | `daprio/daprd:1.14.4` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | +| APISIX | `apache/apisix:3.10.0-debian` | `localhost:9080` (HTTP), `localhost:9180` (admin), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | admin key `learnstack-dev-admin-key` | +| APISIX dashboard | `apache/apisix-dashboard:3.0.1-alpine` | `localhost:9000` | `admin` / `learnstack-dev-dashboard-pass` | + +Configs: + +- Kafka runs in KRaft mode (no ZooKeeper); cluster id is pinned so the log + dir survives restarts without re-format. +- Vault runs in `-dev` mode with the root token baked in — production runs + HA + auto-unseal + AppRole. +- Dapr components live under `infra/dapr/components/` (`pubsub-kafka.yaml`, + `statestore-redis.yaml`, `secretstore-vault.yaml`). Dapr runtime config at + `infra/dapr/config/dapr-config.yaml`. See [../dapr/README.md](../dapr/README.md) + for the `IEventBus` / `ICacheService` / `ISecretProvider` consumption + pattern. +- APISIX is in standalone YAML-reload mode (no etcd). Main config at + `infra/apisix/config.yaml`; route table at `infra/apisix/apisix.yaml`; + dashboard config at `infra/apisix/dashboard.yaml`. See + [../apisix/README.md](../apisix/README.md) for the plugin chain, route + table walk-through, and the `/api/internal/*` mTLS placeholder reserved + for Phase 02c. + +The .NET API host runs OUTSIDE the compose network during active dev; the +Dapr sidecar and APISIX both target `host.docker.internal:5080` so the +sidecar and gateway can reach the workstation-local `dotnet run` process. + ```bash docker compose -f infra/compose/dev.yml up -d docker compose -f infra/compose/dev.yml ps # confirm healthchecks pass @@ -75,10 +110,8 @@ A single bucket per environment is created at first use. Per the [Phase 01 plan](../../docs/roadmap/phase-01-repository-tooling.md), later packets land: -- **Kafka** + **kafka-ui** (Dapr pub/sub backend; Phase 01 packet 6) -- **Vault** (Dapr secret store, dev mode; Phase 01 packet 6) -- **Dapr sidecar** + **placement service** (Phase 01 packet 6) -- **APISIX** (standalone YAML-reload) + **apisix-dashboard** (Phase 01 packet 6) - -A companion `e2e.yml` (same stack, tuned for end-to-end test runs) is also a -later Phase 01 deliverable (packet 7). +- `Makefile` (`make dev` / `test` / `lint` / `seed`), `.env.example` per app, + pre-commit hook (dotnet-format + prettier), and `infra/compose/e2e.yml` + companion stack — Phase 01 packet 7. +- GitHub Actions CI workflow + `make seed` populating two demo tenants and a + platform admin — Phase 01 packet 8. diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index 35c8f56..7b7c00b 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,10 +1,13 @@ # LearnStack — local dev infrastructure. # -# Scope of THIS file (Phase 01 packets 1-5): the data-plane services, the -# self-hosted Keycloak identity provider (two realms), and the live media -# stack (LiveKit OSS + Coturn). Kafka, Vault, the Dapr sidecar, and APISIX -# land in subsequent Phase-01 packets per the roadmap -# (docs/roadmap/phase-01-repository-tooling.md § Local Infrastructure). +# Scope of THIS file (Phase 01 packets 1-6): the data-plane services +# (Postgres / Redis / MinIO / Mailpit / Meilisearch), the self-hosted Keycloak +# identity provider (two realms), the live-media stack (LiveKit OSS + Coturn), +# Kafka (KRaft) + kafka-ui, HashiCorp Vault (-dev mode), the Dapr sidecar + +# placement (pub/sub + state + secrets building blocks per ADR-0014), and the +# APISIX gateway + dashboard (standalone YAML-reload per ADR-0015). The DX +# orchestrator (`make` targets, `.env.example`, e2e overlay) and CI workflow +# arrive in Phase-01 packets 7-8. # # Operational rules: Standards 12 § Local Infrastructure + Standards 20. # Tenant isolation in MinIO is enforced by key prefix (`{tenant_id}/...`) — @@ -197,8 +200,172 @@ services: - "5349:5349/tcp" # TURN/TLS - "49152-49200:49152-49200/udp" # TURN relay range (dev: narrow) + # ---- Eventing (Phase 01 packet 6) ---------------------------------------- + # Kafka in KRaft mode (no ZooKeeper). Backs Dapr pub/sub per ADR-0014. + # Module code never imports `Confluent.Kafka` — only `IEventBus` via Dapr. + kafka: + image: confluentinc/cp-kafka:7.8.0 + container_name: learnstack-kafka + restart: unless-stopped + environment: + # KRaft mode — single-node broker + controller. Production uses an + # external HA cluster; this is dev only. + KAFKA_NODE_ID: 1 + KAFKA_PROCESS_ROLES: broker,controller + KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 + KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 + KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT + KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 + KAFKA_LOG_DIRS: /var/lib/kafka/data + # Stable cluster id so KRaft does not re-format the log dir on restart. + CLUSTER_ID: learnstack-dev-cluster-id-1 + ports: + - "9092:9092" + volumes: + - kafka-data:/var/lib/kafka/data + healthcheck: + test: ["CMD-SHELL", "kafka-topics --bootstrap-server localhost:9092 --list || exit 1"] + interval: 15s + timeout: 10s + retries: 20 + start_period: 30s + + kafka-ui: + image: provectuslabs/kafka-ui:v0.7.2 + container_name: learnstack-kafka-ui + restart: unless-stopped + environment: + KAFKA_CLUSTERS_0_NAME: learnstack-dev + KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092 + DYNAMIC_CONFIG_ENABLED: "false" + ports: + - "8081:8080" + depends_on: + kafka: + condition: service_healthy + + # ---- Secrets (Phase 01 packet 6) ----------------------------------------- + # HashiCorp Vault in -dev mode. Production runs Vault in HA mode with + # auto-unseal + AppRole / Kubernetes auth methods (Standards 12 § Secrets + # Management). The root token is intentionally embedded here for dev only. + vault: + image: hashicorp/vault:1.18 + container_name: learnstack-vault + restart: unless-stopped + command: ["server", "-dev", "-dev-root-token-id=learnstack-dev-root-token", "-dev-listen-address=0.0.0.0:8200"] + environment: + # Dev-only. Production loads tokens via Dapr `secretKeyRef` indirection + # so the literal never appears in component YAML. + VAULT_DEV_ROOT_TOKEN_ID: learnstack-dev-root-token + VAULT_ADDR: http://0.0.0.0:8200 + cap_add: + - IPC_LOCK + ports: + - "8200:8200" + healthcheck: + test: ["CMD", "vault", "status"] + interval: 10s + timeout: 5s + retries: 10 + + # ---- Dapr sidecar (Phase 01 packet 6) ------------------------------------ + # Placement service required by daprd even though actors are out of scope + # per ADR-0014 non-goals (daprd will not boot without it). + dapr-placement: + image: daprio/placement:1.14.4 + container_name: learnstack-dapr-placement + restart: unless-stopped + command: ["./placement", "-log-level", "info"] + ports: + - "50005:50005" + + # Single sidecar bound to the learnstack-api app id. The .NET host runs + # OUTSIDE compose during active dev (`dotnet run` on the workstation); the + # sidecar calls back to host.docker.internal:5080. See ../dapr/README.md. + dapr-sidecar-api: + image: daprio/daprd:1.14.4 + container_name: learnstack-dapr-sidecar-api + restart: unless-stopped + command: + - ./daprd + - -app-id + - learnstack-api + - -app-port + - "5080" + - -dapr-http-port + - "3500" + - -dapr-grpc-port + - "50001" + - -placement-host-address + - dapr-placement:50005 + - -resources-path + - /components + - -config + - /config/dapr-config.yaml + - -log-level + - info + volumes: + - ../dapr/components:/components:ro + - ../dapr/config:/config:ro + ports: + - "3500:3500" # HTTP API + - "50001:50001" # gRPC API + depends_on: + kafka: + condition: service_healthy + redis: + condition: service_healthy + vault: + condition: service_healthy + dapr-placement: + condition: service_started + + # ---- Gateway (Phase 01 packet 6) ----------------------------------------- + # APISIX in standalone YAML-reload mode per ADR-0015 (no etcd). + apisix: + image: apache/apisix:3.10.0-debian + container_name: learnstack-apisix + restart: unless-stopped + volumes: + - ../apisix/config.yaml:/usr/local/apisix/conf/config.yaml:ro + - ../apisix/apisix.yaml:/usr/local/apisix/conf/apisix.yaml:ro + ports: + - "9080:9080" # HTTP gateway + - "9180:9180" # Admin API (dev only) + - "9443:9443" # HTTPS gateway (no cert in dev) + - "9091:9091" # Prometheus metrics + extra_hosts: + # Lets APISIX reach the .NET host running on the developer's machine + # (outside the container network). Docker Desktop already maps this + # alias; Linux needs the explicit host-gateway entry. + - "host.docker.internal:host-gateway" + healthcheck: + test: ["CMD-SHELL", "curl -fsS http://localhost:9080 || curl -fsS http://localhost:9180/apisix/admin/routes -H 'X-API-KEY: learnstack-dev-admin-key' || exit 1"] + interval: 10s + timeout: 5s + retries: 10 + start_period: 15s + + apisix-dashboard: + image: apache/apisix-dashboard:3.0.1-alpine + container_name: learnstack-apisix-dashboard + restart: unless-stopped + volumes: + - ../apisix/dashboard.yaml:/usr/local/apisix-dashboard/conf/conf.yaml:ro + ports: + - "9000:9000" + depends_on: + apisix: + condition: service_started + volumes: postgres-data: redis-data: minio-data: meilisearch-data: + kafka-data: diff --git a/infra/dapr/README.md b/infra/dapr/README.md new file mode 100644 index 0000000..b362d06 --- /dev/null +++ b/infra/dapr/README.md @@ -0,0 +1,80 @@ +# Dapr Sidecar (Dev) + +Cross-cutting infrastructure runtime per +[ADR-0014 (Adopt Dapr)](../../docs/decisions/0014-adopt-dapr.md). Three +building blocks are adopted; everything else is **out of scope** per ADR-0014 +non-goals. + +| Building block | Backend (dev) | Component file | Application interface | +|----------------|---------------|----------------|-----------------------| +| Pub/Sub | Kafka (`kafka:9092`) | `components/pubsub-kafka.yaml` | `IEventBus` (Phase 02b) | +| State store | Redis (`redis:6379`) | `components/statestore-redis.yaml` | `ICacheService` (Phase 02a) | +| Secret store | Vault (`http://vault:8200`, dev mode) | `components/secretstore-vault.yaml` | `ISecretProvider` (Phase 02a) | + +Service invocation, workflow, bindings, actors, configuration, and distributed +lock are **not adopted**; if a future need appears the gate is a new ADR. + +## Sidecar topology + +Dev compose runs one sidecar bound to the `learnstack-api` app id: + +``` +┌──────────────────────────┐ ┌─────────────────────────────────────┐ +│ dotnet run │ │ daprd │ +│ → http://localhost:5080 │ │ ./daprd -app-id learnstack-api \ │ +│ │ │ -app-port 5080 \ │ +│ │◄──┤ -dapr-http-port 3500 \ │ +│ │ │ -dapr-grpc-port 50001 \ │ +│ │ │ -placement-host-address \ │ +│ │ │ dapr-placement:50005 \ │ +│ │ │ -components-path /comp \ │ +│ │ │ -config /config/... │ +└──────────────────────────┘ └─────────────────────────────────────┘ +``` + +The .NET host runs **outside the container network** during active dev +(developers `dotnet run` from their workstation). The Dapr sidecar inside the +compose network calls back to `host.docker.internal:5080` in production-shape +deployments; the dev compose default targets `learnstack-api:5080` and Phase 02b +documents how to switch when the .NET host moves inside compose. + +The placement service (`dapr-placement`) is required even though actors are +out of scope — `daprd` won't start without it. + +## Application access pattern + +Per ADR-0014 + Standards 20, modules **never** import `Dapr.Client`. They +consume: + +```csharp +public interface IEventBus { Task PublishAsync(T @event, CancellationToken ct) where T : IIntegrationEvent; } +public interface ICacheService { Task GetAsync(string key, CancellationToken ct); /* … */ } +public interface ISecretProvider { Task GetSecretAsync(string key, CancellationToken ct); /* … */ } +``` + +`DaprEventBus`, `DaprCacheService`, `DaprSecretProvider` are the **only** +Dapr-aware types in the codebase; they live in `LearnStack.Infrastructure` +and ship in Phase 02b. Architecture tests +`Dapr_SDK_Types_NotImportedOutsideInfrastructure`, +`Modules_DoNotReference_DaprPackage`, and +`ICacheService_Is_OnlyCacheAbstraction` keep this honest. + +## Dev credentials + +| Surface | Credential | +|---------|------------| +| Vault root token | `learnstack-dev-root-token` | +| Kafka auth | none (`authType: none`, `disableTls: true`) | +| Redis password | (empty) | + +All dev-only. Production wires Vault with AppRole / Kubernetes auth and +loads the token through Dapr's `secretKeyRef` indirection so the literal +token never appears in the component YAML. + +## What does NOT live here + +- The `IEventBus` / `ICacheService` / `ISecretProvider` implementations — + Phase 02b (`LearnStack.Infrastructure`). +- Outbox dispatcher (`OutboxProcessor` polling + dispatch) — Phase 02b. +- Per-module `inbox_messages` table + `IInboxGuard` — Phase 02b. +- Production Vault setup (HA mode, auto-unseal, AppRole policies) — Phase 11. diff --git a/infra/dapr/components/pubsub-kafka.yaml b/infra/dapr/components/pubsub-kafka.yaml new file mode 100644 index 0000000..315d320 --- /dev/null +++ b/infra/dapr/components/pubsub-kafka.yaml @@ -0,0 +1,23 @@ +# Dapr pub/sub component — Kafka backend. +# Per ADR-0014, application code talks to `IEventBus` exclusively; the +# `DaprEventBus` implementation (lands Phase 02b) publishes through this +# component. Topic naming follows the `learnstack.{module}.{aggregate}` +# convention enforced by `Dapr_PubSub_TopicNames_FollowConvention` +# architecture test. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: pubsub +spec: + type: pubsub.kafka + version: v1 + metadata: + - name: brokers + value: kafka:9092 + - name: consumerGroup + value: learnstack-api + - name: authType + value: none + - name: disableTls + value: "true" diff --git a/infra/dapr/components/secretstore-vault.yaml b/infra/dapr/components/secretstore-vault.yaml new file mode 100644 index 0000000..e7bb549 --- /dev/null +++ b/infra/dapr/components/secretstore-vault.yaml @@ -0,0 +1,23 @@ +# Dapr secret store — HashiCorp Vault backend. +# Backs `ISecretProvider`. Dev compose runs Vault in -dev mode with the +# root token baked in; production replaces this with an authenticated +# Vault cluster + AppRole / Kubernetes auth method. +# +# Dev-only token. NEVER deploy a config carrying `vaultToken` literally — +# production wires the auth method via Vault's Dapr component metadata +# patterns documented at https://docs.dapr.io/. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: secretstore +spec: + type: secretstores.hashicorp.vault + version: v1 + metadata: + - name: vaultAddr + value: http://vault:8200 + - name: vaultToken + value: learnstack-dev-root-token + - name: vaultKVPrefix + value: secret diff --git a/infra/dapr/components/statestore-redis.yaml b/infra/dapr/components/statestore-redis.yaml new file mode 100644 index 0000000..51b24be --- /dev/null +++ b/infra/dapr/components/statestore-redis.yaml @@ -0,0 +1,21 @@ +# Dapr state store — Redis backend. +# Backs `ICacheService` (L2 cache) and any Dapr actor state (not adopted +# per ADR-0014 non-goals, but the metadata is left actor-capable for +# future Dapr feature expansion behind its own ADR). Module code never +# imports `IDistributedCache` / `StackExchange.Redis` directly per +# `ICacheService_Is_OnlyCacheAbstraction` architecture test. + +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: statestore +spec: + type: state.redis + version: v1 + metadata: + - name: redisHost + value: redis:6379 + - name: redisPassword + value: "" + - name: actorStateStore + value: "true" diff --git a/infra/dapr/config/dapr-config.yaml b/infra/dapr/config/dapr-config.yaml new file mode 100644 index 0000000..b9f7b8d --- /dev/null +++ b/infra/dapr/config/dapr-config.yaml @@ -0,0 +1,14 @@ +# Dapr runtime configuration. +# Tracing target intentionally empty in dev — Phase 11 wires Tempo/Zipkin. +# Module code never reads this file; it tunes the sidecar's own behaviour. + +apiVersion: dapr.io/v1alpha1 +kind: Configuration +metadata: + name: dapr-config +spec: + tracing: + samplingRate: "1" + zipkin: + endpointAddress: "" + features: [] From 07175ba138331da7436cb1248341dfdf456bd10e Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 18:06:26 +0300 Subject: [PATCH 04/11] fix(infra): address Phase 01 packets 4-6 review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Acts on two consolidated reviews of commits 2680da9 + be8a10e + 4243f3a. Triaged 3 Blockers + 6 Majors + 9 Minors + 4 Suggestions; took action on every verified-correct finding. Cascade resolutions reduced the change surface: switching APISIX to file-driven `data_plane` mode drops the Admin API + dashboard companion, which in turn eliminates the port-9000 collision with MinIO and the dashboard's etcd connect-retry loop. Blocker - APISIX 3.10 rejects `deployment.role: traditional + config_provider: yaml`; the modern shape is `role: data_plane + role_data_plane. config_provider: yaml`. Switched accordingly. Admin API not exposed in this mode → removed port 9180, removed `enable_admin` + `admin_key`, removed the `apisix-dashboard` service entirely (etcd-less standalone + no Admin API means the dashboard has nothing to read), and deleted `infra/apisix/dashboard.yaml`. - Port 9000 collision (MinIO S3 vs apisix-dashboard) resolved as a side-effect of the dashboard removal above. - Keycloak healthcheck used `curl`, which is NOT in the `quay.io/keycloak/keycloak:26.0` image (UBI-9-minimal + openjdk-21-runtime ship neither curl nor wget). Replaced with the upstream-recommended bash `/dev/tcp` probe against `:9000/health/ready`. Major - Dapr sidecar's documented `host.docker.internal:5080` callback was not realised — `daprd`'s default `-app-channel-address` is `127.0.0.1`, which resolves to inside the sidecar container. Added `-app-channel-address host.docker.internal` to the daprd command, plus the `extra_hosts: host.docker.internal:host-gateway` mapping (Linux developers). Inbound subscription deliveries now reach the workstation `dotnet run` process. - `mtls` was listed in the APISIX plugin universe + referenced as a route-level plugin on the `/api/internal/*` placeholder. APISIX does not implement mTLS as a route plugin; it is enforced on the SSL/SNI object via `client.ca` / `client.depth`. Removed `mtls` from `config.yaml`'s plugin list; rewrote the Phase-02c placeholder in `apisix.yaml` as an `ssls:` entry + `ip-restriction` plugin shape. - `learnstack-hub` realm's demo operator now carries `requiredActions: ["CONFIGURE_TOTP"]` so the MFA enrolment flow actually surfaces on first login (registering the required-action at realm level was insufficient when the user record didn't request it). - `statestore-redis.yaml` declared `actorStateStore: "true"` despite ADR-0014 keeping actors out of scope. Flipped to `"false"` with an inline comment + a callout in the Dapr README. Minor + cleanup - Dapr README ASCII diagram updated (`-resources-path /components`, the current flag name) and gained an explicit Vault-token-duplication callout naming both files that hard-code `learnstack-dev-root-token`. - APISIX README rewritten to reflect data_plane mode + dashboard removal + the SSL-object pattern for mTLS; the "Linux developers must add extra_hosts manually" note is gone (it's wired via the YAML anchor). - LiveKit `--node-ip 127.0.0.1` flag + `rtc.use_external_ip: false` in livekit.yaml now has an inline comment noting the intentional belt-and-braces. - LiveKit healthcheck switched to `wget --spider -q` so a change in the WebSocket-upgrade response doesn't break the probe. - APISIX route 1 (`/healthz`) now restricts to `methods: [GET]` to match the README. - APISIX healthcheck tolerates either 200 or 404 (route absent ⇒ nginx is still listening, which is what the probe should confirm). - Postgres init-script comment now points at the compose README (no imaginary postgres-init/README). - `extra_hosts: host.docker.internal:host-gateway` extracted as a `*host-gateway` YAML anchor and applied to both `apisix` and `dapr-sidecar-api` via `<<: *host-gateway` so future services adding the same need become a one-liner. - `pubsub-kafka.yaml` gained a consumer-group-pinning warning so a future Hub-overlay sidecar doesn't silently split topic partitions by copy-pasting this component. - Compose README's eventing+gateway table re-grouped (admin port + dev credentials lines removed; Kafka in-cluster-only listener note added; Dapr `-app-channel-address` mechanism documented). - Phase 01 roadmap packet-6 status block reflects the data_plane mode + the SSL-object mTLS stub. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - YAML anchor expansion confirmed via `docker compose ... config`: `extra_hosts` materialises on both apisix and dapr-sidecar-api; `-app-channel-address host.docker.internal` appears in the daprd argument list. - 0 YAML / 0 JSON parse failures across all infra files. - Port table: 9000 is now ONLY bound by MinIO; no other service collision. - Markdown link sweep on every changed doc clean. - Live `docker compose up` smoke test still pending (Docker daemon unreachable on this workstation); ride along with the Phase 07 (DX) `make dev` smoke. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/roadmap/phase-01-repository-tooling.md | 13 +-- infra/apisix/README.md | 61 +++++++------ infra/apisix/apisix.yaml | 50 +++++++---- infra/apisix/config.yaml | 26 +++--- infra/apisix/dashboard.yaml | 32 ------- infra/compose/README.md | 54 +++++++----- infra/compose/dev.yml | 96 ++++++++++++++------- infra/dapr/README.md | 56 ++++++++---- infra/dapr/components/pubsub-kafka.yaml | 8 ++ infra/dapr/components/statestore-redis.yaml | 14 +-- infra/keycloak/realms/learnstack-hub.json | 3 +- 11 files changed, 245 insertions(+), 168 deletions(-) delete mode 100644 infra/apisix/dashboard.yaml diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 46ce22f..7f9a40e 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -39,11 +39,14 @@ > > **Packet 6 — Eventing + secrets + gateway ✅** > Kafka 7.8 in KRaft mode (no ZooKeeper) + kafka-ui, Vault 1.18 in `-dev` mode, -> Dapr 1.14.4 sidecar (`learnstack-api` app id) + placement with three -> components — `pubsub-kafka.yaml`, `statestore-redis.yaml`, -> `secretstore-vault.yaml` per ADR-0014 — and APISIX 3.10 in standalone -> YAML-reload mode + dashboard per ADR-0015 (`infra/apisix/{config,apisix,dashboard}.yaml`, -> mTLS plugin slot reserved for the Phase-02c Hub internal API surface). +> Dapr 1.14.4 sidecar (`learnstack-api` app id, `-app-channel-address +> host.docker.internal` so subscriptions reach the workstation) + placement +> with three components — `pubsub-kafka.yaml`, `statestore-redis.yaml` +> (`actorStateStore: false` per ADR-0014 non-goals), `secretstore-vault.yaml` +> — and APISIX 3.10 in file-driven standalone mode (`deployment.role: +> data_plane`, no Admin API, no companion dashboard) per ADR-0015. The +> `/api/internal/*` Phase-02c surface is documented as an SSL-object + +> ip-restriction stub (mTLS in APISIX is not a route-level plugin). > > **Packet 7 — Developer experience (pending)** > `Makefile` (`make dev` / `test` / `lint` / `seed`), `.env.example` per app, diff --git a/infra/apisix/README.md b/infra/apisix/README.md index 22274fa..4c4ab35 100644 --- a/infra/apisix/README.md +++ b/infra/apisix/README.md @@ -2,8 +2,8 @@ LearnStack's only tenant-facing ingress, per [ADR-0015 (API Gateway: APISIX)](../../docs/decisions/0015-api-gateway-apisix.md). -Standalone YAML-reload mode — no etcd, no admin-UI-driven config drift; the -files in this directory are the source of truth. +File-driven standalone mode — `apisix.yaml` is the only source of truth, +hot-reloaded on file change. No etcd, no Admin API, no dashboard. ## Access @@ -11,9 +11,20 @@ files in this directory are the source of truth. |----------|---------|---------| | HTTP gateway | `http://localhost:9080` | Public + authenticated routes (per `apisix.yaml`) | | HTTPS gateway | `https://localhost:9443` | TLS-terminated routes (no cert in dev) | -| Admin API | `http://localhost:9180` | Route + plugin admin (dev only; `admin` key `learnstack-dev-admin-key`) | | Prometheus metrics | `http://localhost:9091` | Per-route + plugin metrics scraped by Prometheus (Phase 11) | -| Dashboard | `http://localhost:9000` | Browser UI (dev only; user `admin` / pass `learnstack-dev-dashboard-pass`) | + +### Why no Admin API and no dashboard? + +APISIX 3.2+ moved file-driven standalone behind `deployment.role: +data_plane`, which **does not expose an Admin API**. The companion +`apisix-dashboard` reads routes from etcd; without etcd it cannot show +anything. Both are intentionally out of this compose stack because +ADR-0015 commits to "standalone YAML hot-reload, no etcd dependency." + +`apisix.yaml` is therefore the only place a developer changes routes; +diff-review of that file replaces the dashboard for dev workflow. The +etcd-backed deployment is reconsidered when a real admin-UI requirement +appears, behind its own ADR. ## Plugin chain @@ -26,8 +37,11 @@ real-ip → cors → openid-connect → limit-req → request-id → proxy-rewri prometheus (response) ``` -`mtls` is **reserved** — Phase 02c (LearnStack Hub) activates it against the -LearnStack-internal CA for the `/api/internal/*` surface. +**mTLS is NOT a route-level plugin** in APISIX — it is configured on the +SSL/SNI object (`client.ca` / `client.depth`). The Phase 02c +`/api/internal/*` surface enforces mTLS through an `ssls:` entry plus +route-level `ip-restriction`; see the commented stub at the bottom of +`apisix.yaml`. ## Routes shipped today (`apisix.yaml`) @@ -42,25 +56,17 @@ plugin against the `learnstack` Keycloak realm discovery endpoint. The backend re-validates the JWT (defence in depth — gateway compromise must not bypass auth). -The `/api/internal/*` route is a **disabled placeholder**; Phase 02c -activates it with `mtls` + `ip-restriction` against the Hub-issued client -cert. +The `/api/internal/*` route is a **disabled placeholder** documenting the +SSL-object + ip-restriction shape Phase 02c activates. -## Upstream addressing — why `host.docker.internal:5080` +## Upstream addressing — `host.docker.internal:5080` The .NET API host runs **outside the container network** during active dev (`dotnet run` on the developer's workstation). Docker Desktop's -`host.docker.internal` alias resolves to the host loopback; Linux developers -without Docker Desktop need to add: - -```yaml -apisix: - extra_hosts: - - "host.docker.internal:host-gateway" -``` - -…in their local override (the DX packet (07) ships `make dev` with the -cross-platform case handled). +`host.docker.internal` alias resolves to the host loopback; the +`extra_hosts: host.docker.internal:host-gateway` entry on the `apisix` +service (via the `*host-gateway` YAML anchor in `dev.yml`) makes the +alias available on Linux too — **no manual override needed**. When the .NET host moves inside compose in a later environment profile, the upstream nodes shift to `learnstack-api:5080` — same `apisix.yaml`, @@ -72,15 +78,18 @@ APISIX watches `apisix.yaml` and re-applies the route table on file change. A malformed YAML push takes the gateway down, so changes go through CI YAML validation before they reach the running gateway. -## Dev credentials are dev credentials +## Dev posture -Every credential in this directory (admin key, dashboard JWT secret, -dashboard user) is dev-only. Production loads them from Vault via the -component metadata pattern + restricts the admin API to an -internal-only listener. +No credentials live in this directory — file-driven standalone mode has +no Admin API to protect, no dashboard to log into. The only sensitive +surface is the SSL cert pair that Phase 11 introduces when TLS lands. ## What does NOT live here +- etcd-backed deployment shape — reconsidered behind its own ADR if / + when a UI-driven admin requirement appears. +- `apisix-dashboard` companion — see "Why no Admin API and no dashboard?" + above. - Production TLS cert (Let's Encrypt via the same adapter family ADR-0022 picks for custom domains) — Phase 11. - Hub-side APISIX route block (`hub.learnstack.dev`) — Phase 02c, lives in diff --git a/infra/apisix/apisix.yaml b/infra/apisix/apisix.yaml index 6ab505a..b9260a1 100644 --- a/infra/apisix/apisix.yaml +++ b/infra/apisix/apisix.yaml @@ -4,18 +4,16 @@ # openid-connect. # # This packet ships the scaffold routes only: -# - `/healthz` (priority 1, public) — proxies to the .NET API host -# - `/api/v*/**` OPTIONS (priority 99, CORS preflight) — plugin: cors only -# - `/api/v*/**` authenticated (priority 100, all other verbs) — plugin -# chain with `openid-connect` placeholder (Keycloak realm discovery -# gets wired in Phase 03) -# - `/api/internal/*` (priority 50, mTLS) — placeholder route disabled -# (the `mtls` plugin entry is commented out; Phase 02c activates it -# once the Hub HTTPS contract surface lands) +# - `/healthz` (priority 1, public, GET only) — proxies to the .NET API +# - `/api/v*/**` OPTIONS (priority 99, CORS preflight) +# - `/api/v*/**` authenticated (priority 100, GET/POST/PUT/PATCH/DELETE) +# - `/api/internal/*` — Phase 02c surface, NOT a route plugin: mTLS is +# enforced on the SSL/SNI object (see the Phase-02c stub at the bottom) routes: - id: 1 uri: /healthz + methods: [GET] priority: 1 plugins: cors: {} @@ -30,9 +28,9 @@ routes: nodes: # The .NET API host runs OUTSIDE compose during active dev # (`dotnet run` on the workstation). `host.docker.internal` is - # Docker Desktop's gateway alias; Linux developers may need to - # add `--add-host=host.docker.internal:host-gateway` to the - # apisix service. Phase 07 (DX packet) sorts the cross-platform + # Docker Desktop's gateway alias; Linux developers get the + # `host-gateway` mapping via `extra_hosts` on the apisix service + # in dev.yml. Phase 07 (DX packet) sorts the cross-platform # case alongside `make dev`. "host.docker.internal:5080": 1 @@ -73,21 +71,41 @@ routes: nodes: "host.docker.internal:5080": 1 - # /api/internal/* route is RESERVED here so the plugin slot is - # documented; Phase 02c lands the Hub HTTPS contract surface and - # enables the mtls plugin against the LearnStack-internal CA. + # ---- Phase 02c — Hub internal API surface (RESERVED) -------------------- + # mTLS in APISIX is NOT a route-level plugin. It is enforced on the + # SSL/SNI object via `client.ca` (CA bundle for verifying client certs) + # + `client.depth` (cert chain depth). The route below documents the + # Phase-02c shape; activate by: + # + # 1. Declaring an `ssls:` entry with the LearnStack-internal CA bundle + # attached to the host (or sni) that fronts `/api/internal/*`. + # 2. Restricting the route to that host (or to an internal IP range) + # via the `ip-restriction` plugin and / or a host match. + # 3. Routing only authenticated mTLS client certs through here; the + # upstream re-validates the signed JWT + HMAC body signature + # (ADR-0019 § Closed 4-endpoint internal API). + # # - id: 50 # uri: /api/internal/* + # host: internal.learnstack.dev # priority: 50 # plugins: - # mtls: - # ca_uri: /path/to/learnstack-internal-ca.crt # ip-restriction: # whitelist: # - 10.0.0.0/8 + # prometheus: {} # upstream: # type: roundrobin # nodes: # "host.docker.internal:5080": 1 + # + # ssls: + # - id: 1 + # sni: internal.learnstack.dev + # cert: "" + # key: "" + # client: + # ca: "" + # depth: 2 #END diff --git a/infra/apisix/config.yaml b/infra/apisix/config.yaml index 04cc342..b2247ec 100644 --- a/infra/apisix/config.yaml +++ b/infra/apisix/config.yaml @@ -1,21 +1,18 @@ -# APISIX main config — standalone YAML-reload mode. +# APISIX main config — file-driven standalone mode. # Per ADR-0015 the standalone deployment shape (no etcd) is the default; # etcd-backed mode is reconsidered when an admin UI / dynamic config -# requirement arrives. Hot-reload watches `apisix.yaml`. +# requirement arrives. APISIX 3.2+ moved file-driven standalone behind +# `deployment.role: data_plane` + `role_data_plane.config_provider: yaml`; +# the older `traditional + yaml` shape is removed in 3.10. The Admin API +# is NOT available in this mode — `apisix.yaml` is the only source of +# truth, hot-reloaded on file change. deployment: - role: traditional - role_traditional: + role: data_plane + role_data_plane: config_provider: yaml apisix: - # Dev-only admin key. Production replaces this with a Vault-sourced value - # and only exposes the admin API on an internal-only listener. - enable_admin: true - admin_key: - - name: admin - key: learnstack-dev-admin-key - role: admin enable_ipv6: false proxy_mode: http enable_dev_mode: false @@ -27,6 +24,12 @@ nginx_config: # Plugin chain available to routes. The actual per-route chain is declared # in apisix.yaml; this list is the universe routes may pick from. +# +# NOTE: mTLS is NOT a route-level plugin in APISIX — it is configured on +# the SSL/SNI object (`client.ca` / `client.depth`) per the upstream docs. +# The Phase-02c `/api/internal/*` surface enforces mTLS via the SSL object +# + `ip-restriction` on the route; the plugin universe below intentionally +# omits any `mtls` entry. plugins: - real-ip - cors @@ -39,7 +42,6 @@ plugins: - response-rewrite - prometheus - ip-restriction - - mtls # Reserved for /api/internal/* (Phase 02c) - gzip plugin_attr: diff --git a/infra/apisix/dashboard.yaml b/infra/apisix/dashboard.yaml deleted file mode 100644 index 93438b3..0000000 --- a/infra/apisix/dashboard.yaml +++ /dev/null @@ -1,32 +0,0 @@ -# APISIX Dashboard config (dev only). -# The dashboard is a developer convenience for browsing routes + plugins -# in dev mode. Production deployments either run without the dashboard -# (standalone YAML is the source of truth) or pin it behind mTLS on an -# internal-only listener. - -conf: - listen: - host: 0.0.0.0 - port: 9000 - allow_list: - - 0.0.0.0/0 # Dev-only. Production tightens to operator subnet. - etcd: - # APISIX runs in standalone mode (no etcd); the dashboard cannot - # introspect routes without it. The endpoints below are placeholders - # so the dashboard boots without erroring on missing config — the - # route list view will be empty. - endpoints: - - http://127.0.0.1:2379 - log: - error_log: - level: info - file_path: logs/error.log - access_log: - file_path: logs/access.log - -authentication: - secret: learnstack-dev-dashboard-jwt-secret - expire_time: 3600 - users: - - username: admin - password: learnstack-dev-dashboard-pass diff --git a/infra/compose/README.md b/infra/compose/README.md index 3921302..ee8ce8a 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -54,35 +54,45 @@ recording / consent / cost-tracking story. | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| Kafka (KRaft) | `confluentinc/cp-kafka:7.8.0` | `localhost:9092` | none (`PLAINTEXT`, `authType: none`) | +| Kafka (KRaft) | `confluentinc/cp-kafka:7.8.0` | `localhost:9092` (in-cluster only — see note below) | none (`PLAINTEXT`, `authType: none`) | | kafka-ui | `provectuslabs/kafka-ui:v0.7.2` | `localhost:8081` | open UI (dev only) | | Vault | `hashicorp/vault:1.18` | `localhost:8200` | root token `learnstack-dev-root-token` | | Dapr placement | `daprio/placement:1.14.4` | `localhost:50005` | — | | Dapr sidecar (api) | `daprio/daprd:1.14.4` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | -| APISIX | `apache/apisix:3.10.0-debian` | `localhost:9080` (HTTP), `localhost:9180` (admin), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | admin key `learnstack-dev-admin-key` | -| APISIX dashboard | `apache/apisix-dashboard:3.0.1-alpine` | `localhost:9000` | `admin` / `learnstack-dev-dashboard-pass` | +| APISIX | `apache/apisix:3.10.0-debian` | `localhost:9080` (HTTP), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | none (file-driven standalone — no Admin API) | Configs: -- Kafka runs in KRaft mode (no ZooKeeper); cluster id is pinned so the log - dir survives restarts without re-format. -- Vault runs in `-dev` mode with the root token baked in — production runs - HA + auto-unseal + AppRole. -- Dapr components live under `infra/dapr/components/` (`pubsub-kafka.yaml`, - `statestore-redis.yaml`, `secretstore-vault.yaml`). Dapr runtime config at - `infra/dapr/config/dapr-config.yaml`. See [../dapr/README.md](../dapr/README.md) - for the `IEventBus` / `ICacheService` / `ISecretProvider` consumption - pattern. -- APISIX is in standalone YAML-reload mode (no etcd). Main config at - `infra/apisix/config.yaml`; route table at `infra/apisix/apisix.yaml`; - dashboard config at `infra/apisix/dashboard.yaml`. See - [../apisix/README.md](../apisix/README.md) for the plugin chain, route - table walk-through, and the `/api/internal/*` mTLS placeholder reserved - for Phase 02c. - -The .NET API host runs OUTSIDE the compose network during active dev; the -Dapr sidecar and APISIX both target `host.docker.internal:5080` so the -sidecar and gateway can reach the workstation-local `dotnet run` process. +- **Kafka** runs in KRaft mode (no ZooKeeper); cluster id is pinned so the + log dir survives restarts without re-format. Only the in-cluster + `PLAINTEXT://kafka:9092` listener is advertised; host-side tools (kcat, + kafka-topics from the workstation) will resolve the bootstrap address as + `kafka:9092` and fail unless `127.0.0.1 kafka` is added to `/etc/hosts`. + Use `kafka-ui` (`localhost:8081`) for workstation-side browsing; the + Phase 07 DX packet ships either an EXTERNAL listener or documents the + `kafka-ui`-only workflow as canonical. +- **Vault** runs in `-dev` mode with the root token baked in — production + runs HA + auto-unseal + AppRole. +- **Dapr** components live under `infra/dapr/components/` + (`pubsub-kafka.yaml`, `statestore-redis.yaml`, `secretstore-vault.yaml`); + runtime config at `infra/dapr/config/dapr-config.yaml`. The sidecar is + wired to call back to `host.docker.internal:5080` via the + `-app-channel-address` flag (daprd's default `127.0.0.1` would resolve + inside the sidecar container and break subscription deliveries). See + [../dapr/README.md](../dapr/README.md) for the + `IEventBus` / `ICacheService` / `ISecretProvider` consumption pattern. +- **APISIX** runs in file-driven standalone mode (`deployment.role: + data_plane`) per ADR-0015 — no etcd, no Admin API, no companion + dashboard. Main config at `infra/apisix/config.yaml`; route table at + `infra/apisix/apisix.yaml`. See [../apisix/README.md](../apisix/README.md) + for the plugin chain, route table walk-through, and the + `/api/internal/*` mTLS-via-SSL-object placeholder reserved for Phase 02c. + +The .NET API host runs OUTSIDE the compose network during active dev; both +the Dapr sidecar and APISIX target `host.docker.internal:5080` so they can +reach the workstation-local `dotnet run` process. The +`host.docker.internal:host-gateway` alias is wired via a YAML anchor in +`dev.yml` so Linux developers don't need a manual override. ```bash docker compose -f infra/compose/dev.yml up -d diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index 7b7c00b..c9bb88d 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -5,7 +5,7 @@ # identity provider (two realms), the live-media stack (LiveKit OSS + Coturn), # Kafka (KRaft) + kafka-ui, HashiCorp Vault (-dev mode), the Dapr sidecar + # placement (pub/sub + state + secrets building blocks per ADR-0014), and the -# APISIX gateway + dashboard (standalone YAML-reload per ADR-0015). The DX +# APISIX gateway in file-driven standalone mode per ADR-0015. The DX # orchestrator (`make` targets, `.env.example`, e2e overlay) and CI workflow # arrive in Phase-01 packets 7-8. # @@ -15,6 +15,15 @@ name: learnstack-dev +# YAML anchor for the host-gateway alias every container that has to reach +# the workstation-local `dotnet run` process needs. Docker Desktop maps +# `host.docker.internal` automatically; Linux requires this explicit +# `host-gateway` entry. Anchored here so adding it to a new service is a +# `<<: *host-gateway` one-liner. +x-host-gateway: &host-gateway + extra_hosts: + - "host.docker.internal:host-gateway" + services: postgres: image: postgres:16.6-alpine @@ -30,7 +39,8 @@ services: volumes: - postgres-data:/var/lib/postgresql/data # First-boot init scripts (idempotent SQL). Creates the `keycloak` - # database the Keycloak service uses; see ./postgres-init/README. + # database the Keycloak service uses; see the "Identity" section of + # ../compose/README.md for the re-seed procedure. - ./postgres-init:/docker-entrypoint-initdb.d:ro healthcheck: test: ["CMD-SHELL", "pg_isready -U learnstack -d learnstack"] @@ -149,11 +159,16 @@ services: postgres: condition: service_healthy healthcheck: - # Keycloak 26 exposes /health/ready on the dedicated management - # interface (port 9000) when KC_HEALTH_ENABLED=true. The image is - # UBI-9-minimal based and ships curl; this is the upstream-recommended - # readiness pattern. - test: ["CMD-SHELL", "curl -fsS http://localhost:9000/health/ready || exit 1"] + # The Keycloak 26 image is UBI-9-minimal based + openjdk-21-runtime — + # neither curl nor wget is in the image. Bash IS available, so we use + # the upstream-recommended bash /dev/tcp probe against the management + # interface (KC_HEALTH_ENABLED=true exposes /health/ready on :9000). + test: + - CMD-SHELL + - | + exec 3<>/dev/tcp/localhost/9000 && \ + echo -e "GET /health/ready HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n" >&3 && \ + grep -q "200 OK" <&3 interval: 15s timeout: 10s retries: 20 @@ -168,6 +183,10 @@ services: image: livekit/livekit-server:v1.8.0 container_name: learnstack-livekit restart: unless-stopped + # `--node-ip 127.0.0.1` and `rtc.use_external_ip: false` in + # livekit.yaml are two ways of telling LiveKit not to auto-discover the + # host IP; either alone suffices in dev. Both are kept for explicitness + # — production sets `--node-ip ` + `use_external_ip: true`. command: ["--config", "/etc/livekit.yaml", "--node-ip", "127.0.0.1", "--dev"] volumes: - ../livekit/livekit.yaml:/etc/livekit.yaml:ro @@ -180,9 +199,10 @@ services: coturn: condition: service_started healthcheck: - # LiveKit serves a default 200 on `/` once the WebSocket listener is up. - # The image is alpine-based and ships wget. - test: ["CMD", "wget", "-qO-", "http://localhost:7880"] + # `--spider` issues a HEAD without downloading the body; tolerates + # LiveKit's WebSocket-upgrade contract changing across minor versions + # since we only care that the listener is bound. + test: ["CMD", "wget", "--spider", "-q", "http://localhost:7880"] interval: 10s timeout: 5s retries: 10 @@ -203,6 +223,14 @@ services: # ---- Eventing (Phase 01 packet 6) ---------------------------------------- # Kafka in KRaft mode (no ZooKeeper). Backs Dapr pub/sub per ADR-0014. # Module code never imports `Confluent.Kafka` — only `IEventBus` via Dapr. + # + # Listener note: only the in-cluster PLAINTEXT://kafka:9092 listener is + # advertised. Host-side CLI tools (kcat, kafka-topics from the workstation) + # connect through the port mapping below but will receive `kafka:9092` as + # the bootstrap address and fail to resolve it unless the developer adds + # `127.0.0.1 kafka` to /etc/hosts (or routes them through the kafka-ui). + # Phase 07 (DX) ships either an explicit EXTERNAL listener or a + # documented kafka-ui-only workflow. kafka: image: confluentinc/cp-kafka:7.8.0 container_name: learnstack-kafka @@ -253,6 +281,11 @@ services: # HashiCorp Vault in -dev mode. Production runs Vault in HA mode with # auto-unseal + AppRole / Kubernetes auth methods (Standards 12 § Secrets # Management). The root token is intentionally embedded here for dev only. + # + # IMPORTANT: the literal `learnstack-dev-root-token` appears in TWO files: + # this service definition AND ../dapr/components/secretstore-vault.yaml. + # If you change one you MUST change the other. Phase 07 (DX) wires both + # to a single `.env.example` source so the duplication goes away. vault: image: hashicorp/vault:1.18 container_name: learnstack-vault @@ -285,18 +318,25 @@ services: - "50005:50005" # Single sidecar bound to the learnstack-api app id. The .NET host runs - # OUTSIDE compose during active dev (`dotnet run` on the workstation); the - # sidecar calls back to host.docker.internal:5080. See ../dapr/README.md. + # OUTSIDE compose during active dev (`dotnet run` on the workstation); + # `-app-channel-address host.docker.internal` is what makes inbound + # subscription deliveries reach the host (the daprd default of 127.0.0.1 + # would resolve to inside the sidecar container). The `extra_hosts` + # anchor at the top of the file maps `host.docker.internal` on Linux. + # See ../dapr/README.md. dapr-sidecar-api: image: daprio/daprd:1.14.4 container_name: learnstack-dapr-sidecar-api restart: unless-stopped + <<: *host-gateway command: - ./daprd - -app-id - learnstack-api - -app-port - "5080" + - -app-channel-address + - host.docker.internal - -dapr-http-port - "3500" - -dapr-grpc-port @@ -326,43 +366,35 @@ services: condition: service_started # ---- Gateway (Phase 01 packet 6) ----------------------------------------- - # APISIX in standalone YAML-reload mode per ADR-0015 (no etcd). + # APISIX in file-driven standalone mode per ADR-0015 — no etcd, no Admin + # API. `apisix.yaml` is the only source of truth (hot-reloaded on file + # change). The `apisix-dashboard` companion is intentionally absent from + # this stack: the dashboard needs etcd to read routes, which contradicts + # the standalone-no-etcd commitment of ADR-0015. Diff-review of + # `apisix.yaml` replaces the dashboard for the dev workflow. apisix: image: apache/apisix:3.10.0-debian container_name: learnstack-apisix restart: unless-stopped + <<: *host-gateway volumes: - ../apisix/config.yaml:/usr/local/apisix/conf/config.yaml:ro - ../apisix/apisix.yaml:/usr/local/apisix/conf/apisix.yaml:ro ports: - "9080:9080" # HTTP gateway - - "9180:9180" # Admin API (dev only) - "9443:9443" # HTTPS gateway (no cert in dev) - "9091:9091" # Prometheus metrics - extra_hosts: - # Lets APISIX reach the .NET host running on the developer's machine - # (outside the container network). Docker Desktop already maps this - # alias; Linux needs the explicit host-gateway entry. - - "host.docker.internal:host-gateway" healthcheck: - test: ["CMD-SHELL", "curl -fsS http://localhost:9080 || curl -fsS http://localhost:9180/apisix/admin/routes -H 'X-API-KEY: learnstack-dev-admin-key' || exit 1"] + # The debian-based APISIX image ships curl; we hit the gateway port + # (a 404 is fine — it confirms nginx is listening). A 200 from a real + # route would require the .NET API host to be up, which is out of + # scope for this compose file. + test: ["CMD-SHELL", "curl -fsS -o /dev/null -w '%{http_code}' http://localhost:9080 | grep -E '^(200|404)$' || exit 1"] interval: 10s timeout: 5s retries: 10 start_period: 15s - apisix-dashboard: - image: apache/apisix-dashboard:3.0.1-alpine - container_name: learnstack-apisix-dashboard - restart: unless-stopped - volumes: - - ../apisix/dashboard.yaml:/usr/local/apisix-dashboard/conf/conf.yaml:ro - ports: - - "9000:9000" - depends_on: - apisix: - condition: service_started - volumes: postgres-data: redis-data: diff --git a/infra/dapr/README.md b/infra/dapr/README.md index b362d06..3126f1b 100644 --- a/infra/dapr/README.md +++ b/infra/dapr/README.md @@ -11,32 +11,39 @@ non-goals. | State store | Redis (`redis:6379`) | `components/statestore-redis.yaml` | `ICacheService` (Phase 02a) | | Secret store | Vault (`http://vault:8200`, dev mode) | `components/secretstore-vault.yaml` | `ISecretProvider` (Phase 02a) | -Service invocation, workflow, bindings, actors, configuration, and distributed -lock are **not adopted**; if a future need appears the gate is a new ADR. +Service invocation, workflow, bindings, **actors**, configuration, and +distributed lock are **not adopted**; if a future need appears the gate is a +new ADR. The state store's `actorStateStore` flag is therefore pinned to +`"false"` — see the comment at the top of `components/statestore-redis.yaml`. ## Sidecar topology Dev compose runs one sidecar bound to the `learnstack-api` app id: ``` -┌──────────────────────────┐ ┌─────────────────────────────────────┐ -│ dotnet run │ │ daprd │ -│ → http://localhost:5080 │ │ ./daprd -app-id learnstack-api \ │ -│ │ │ -app-port 5080 \ │ -│ │◄──┤ -dapr-http-port 3500 \ │ -│ │ │ -dapr-grpc-port 50001 \ │ -│ │ │ -placement-host-address \ │ -│ │ │ dapr-placement:50005 \ │ -│ │ │ -components-path /comp \ │ -│ │ │ -config /config/... │ -└──────────────────────────┘ └─────────────────────────────────────┘ +┌──────────────────────────┐ ┌──────────────────────────────────────────┐ +│ dotnet run │ │ daprd │ +│ → host:5080 │ │ ./daprd -app-id learnstack-api \ │ +│ │ │ -app-port 5080 \ │ +│ │◄──┤ -app-channel-address \ │ +│ │ │ host.docker.internal \ │ +│ │ │ -dapr-http-port 3500 \ │ +│ │ │ -dapr-grpc-port 50001 \ │ +│ │ │ -placement-host-address \ │ +│ │ │ dapr-placement:50005 \ │ +│ │ │ -resources-path /components \ │ +│ │ │ -config /config/dapr-config.yaml +└──────────────────────────┘ └──────────────────────────────────────────┘ ``` The .NET host runs **outside the container network** during active dev -(developers `dotnet run` from their workstation). The Dapr sidecar inside the -compose network calls back to `host.docker.internal:5080` in production-shape -deployments; the dev compose default targets `learnstack-api:5080` and Phase 02b -documents how to switch when the .NET host moves inside compose. +(developers `dotnet run` from their workstation). `-app-channel-address +host.docker.internal` is what makes inbound subscription deliveries reach +the host — daprd's default of `127.0.0.1` would resolve to inside the +sidecar container, and every subscription would silently fail to deliver. +The `extra_hosts: host.docker.internal:host-gateway` YAML anchor in +`infra/compose/dev.yml` maps the alias on Linux (Docker Desktop does it +automatically). The placement service (`dapr-placement`) is required even though actors are out of scope — `daprd` won't start without it. @@ -71,6 +78,19 @@ All dev-only. Production wires Vault with AppRole / Kubernetes auth and loads the token through Dapr's `secretKeyRef` indirection so the literal token never appears in the component YAML. +### Vault token duplication + +The literal `learnstack-dev-root-token` appears in **two files**: + +- `infra/compose/dev.yml` — `vault` service command + env var (the token + Vault `-dev` mode boots with). +- `infra/dapr/components/secretstore-vault.yaml` — `vaultToken` metadata + (the token Dapr authenticates to Vault with). + +These MUST stay in lockstep. Phase 07 (DX) wires both to a single +`.env.example` source so the duplication goes away; until then, change +both places together. + ## What does NOT live here - The `IEventBus` / `ICacheService` / `ISecretProvider` implementations — @@ -78,3 +98,5 @@ token never appears in the component YAML. - Outbox dispatcher (`OutboxProcessor` polling + dispatch) — Phase 02b. - Per-module `inbox_messages` table + `IInboxGuard` — Phase 02b. - Production Vault setup (HA mode, auto-unseal, AppRole policies) — Phase 11. +- Additional sidecars for the Hub overlay (separate app id, separate + consumer group) — Phase 02c. diff --git a/infra/dapr/components/pubsub-kafka.yaml b/infra/dapr/components/pubsub-kafka.yaml index 315d320..93ca0ff 100644 --- a/infra/dapr/components/pubsub-kafka.yaml +++ b/infra/dapr/components/pubsub-kafka.yaml @@ -4,6 +4,14 @@ # component. Topic naming follows the `learnstack.{module}.{aggregate}` # convention enforced by `Dapr_PubSub_TopicNames_FollowConvention` # architecture test. +# +# IMPORTANT: `consumerGroup` is pinned to the LearnStack core app id +# (`learnstack-api`). Adding a second consumer (a second app id, the Hub +# overlay in packet 7, a separate worker process) requires a DIFFERENT +# consumer group — copy this component, do not reuse the YAML with a +# different `app-id`. Sharing a consumer group across distinct apps +# silently splits the topic partitions between them and breaks +# at-least-once semantics for both. apiVersion: dapr.io/v1alpha1 kind: Component diff --git a/infra/dapr/components/statestore-redis.yaml b/infra/dapr/components/statestore-redis.yaml index 51b24be..25b85b0 100644 --- a/infra/dapr/components/statestore-redis.yaml +++ b/infra/dapr/components/statestore-redis.yaml @@ -1,9 +1,13 @@ # Dapr state store — Redis backend. -# Backs `ICacheService` (L2 cache) and any Dapr actor state (not adopted -# per ADR-0014 non-goals, but the metadata is left actor-capable for -# future Dapr feature expansion behind its own ADR). Module code never -# imports `IDistributedCache` / `StackExchange.Redis` directly per +# Backs `ICacheService` (L2 cache). Module code never imports +# `IDistributedCache` / `StackExchange.Redis` directly per # `ICacheService_Is_OnlyCacheAbstraction` architecture test. +# +# `actorStateStore: "false"` is INTENTIONAL — per ADR-0014 non-goals, +# Dapr actors are NOT adopted. Flipping this to "true" requires a new +# ADR + an architecture-test update. The Dapr docs are explicit that the +# flag must be "true" only when the application code consumes the actors +# building block; declaring it true unused signals false intent. apiVersion: dapr.io/v1alpha1 kind: Component @@ -18,4 +22,4 @@ spec: - name: redisPassword value: "" - name: actorStateStore - value: "true" + value: "false" diff --git a/infra/keycloak/realms/learnstack-hub.json b/infra/keycloak/realms/learnstack-hub.json index 8f165c2..bbe7a31 100644 --- a/infra/keycloak/realms/learnstack-hub.json +++ b/infra/keycloak/realms/learnstack-hub.json @@ -94,7 +94,8 @@ "temporary": false } ], - "realmRoles": ["hub-operator"] + "realmRoles": ["hub-operator"], + "requiredActions": ["CONFIGURE_TOTP"] } ], "smtpServer": { From 28ea75021738670f479d144ffa7525cc6640cb7f Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 18:18:27 +0300 Subject: [PATCH 05/11] chore(infra): bump dev-stack images to latest stable + swap kafka-ui fork MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Picks up upstream patches and minors across every pinned image where a straightforward upgrade is safe. MinIO is deliberately left alone here — the Bitnami repo was archived and the project is migrating to SeaweedFS in a separate ADR-gated commit. Redis 7.4 and Postgres 16 are intentionally kept at the current major because a bump there is a license / migration decision that needs its own ADR. Upgrades - Postgres 16.6 → 16.14-alpine (16.x patch) - Mailpit v1.21 → v1.29.7 - Meilisearch v1.11 → v1.44.0 (33 minors; dev compose, `down -v` re-seeds) - Keycloak 26.0 → 26.6.2 (26.x patch + minor) - LiveKit v1.8.0 → v1.12.0 - Coturn 4.6 → 4.11.0 - Confluent CP Kafka 7.8.0 → 8.2.1 (CP 8.x ⇒ Kafka 4.x; KRaft stays the default broker mode so the existing env-var block continues to apply) - Vault 1.18 → 1.21.4 (community edition; Enterprise's 2.0 jump does not affect this image) - Dapr daprd + placement 1.14.4 → 1.17.7 - APISIX 3.10.0-debian → 3.16.0-debian (data_plane mode unchanged) Fork swap - kafka-ui: `provectuslabs/kafka-ui:v0.7.2` → `ghcr.io/kafbat/kafka-ui:latest` The upstream `provectuslabs/kafka-ui` had no release since 2024-04 and is effectively abandoned with known CVEs; `kafbat/kafka-ui` is the active community fork sharing the same env-var contract. Deliberately NOT bumped (need their own ADR) - Redis 7.4 → 8.x: the 8.x line is tri-licensed (AGPL+SSPL+RSALv2). Choice between Redis 8.x and the Valkey fork is a license + governance call. - Postgres 16 → 17/18: major upgrades affect extension compatibility + RLS-specific defaults; needs a migration plan. Already-removed (out of scope for this packet, not a regression) - APISIX dashboard was removed in 07175ba when APISIX switched to file-driven standalone mode (no etcd ⇒ no dashboard). Compose README port table updated to match. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - No port mappings changed; no name collisions. - Live `docker compose up` smoke test still pending (Docker daemon unreachable on this workstation). Co-Authored-By: Claude Opus 4.7 (1M context) --- infra/compose/README.md | 24 ++++++++++++------------ infra/compose/dev.yml | 27 +++++++++++++++------------ 2 files changed, 27 insertions(+), 24 deletions(-) diff --git a/infra/compose/README.md b/infra/compose/README.md index ee8ce8a..ca64dbe 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -13,17 +13,17 @@ eventing → secrets → Dapr sidecar → gateway). Packets 1-6 shipped; packets | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| PostgreSQL 16 | `postgres:16.6-alpine` | `localhost:5432` | `learnstack` / `learnstack` | +| PostgreSQL 16 | `postgres:16.14-alpine` | `localhost:5432` | `learnstack` / `learnstack` | | Redis 7 | `redis:7.4-alpine` | `localhost:6379` | — | | MinIO | `minio/minio:RELEASE.2025-01-20T14-49-07Z` | `localhost:9000` (S3), `localhost:9001` (console) | `learnstack` / `learnstack-dev-secret` | -| Mailpit | `axllent/mailpit:v1.21` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | -| Meilisearch | `getmeili/meilisearch:v1.11` | `localhost:7700` | master key `learnstack-dev-master-key` | +| Mailpit | `axllent/mailpit:v1.29.7` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | +| Meilisearch | `getmeili/meilisearch:v1.44.0` | `localhost:7700` | master key `learnstack-dev-master-key` | ### Identity (Phase 01 packet 4) | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| Keycloak | `quay.io/keycloak/keycloak:26.0` | `localhost:8080` | master admin `admin` / `admin-dev-secret` | +| Keycloak | `quay.io/keycloak/keycloak:26.6.2` | `localhost:8080` | master admin `admin` / `admin-dev-secret` | Two realms imported on first boot from `infra/keycloak/realms/`: @@ -42,8 +42,8 @@ or a manual `DROP DATABASE keycloak; CREATE DATABASE keycloak OWNER learnstack;` | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| LiveKit OSS | `livekit/livekit-server:v1.8.0` | `ws://localhost:7880` (signaling), `tcp/7881` (TCP fallback), `tcp/7882` (TURN/TLS), `udp/50000-50100` (media) | API key `devkey` / secret `devsecret-32-byte-min-length-padding-xyz` | -| Coturn | `coturn/coturn:4.6` | `udp+tcp/3478` (STUN/TURN), `tcp/5349` (TURN/TLS), `udp/49152-49200` (relay range) | TURN user `devuser` / password `devsecret` | +| LiveKit OSS | `livekit/livekit-server:v1.12.0` | `ws://localhost:7880` (signaling), `tcp/7881` (TCP fallback), `tcp/7882` (TURN/TLS), `udp/50000-50100` (media) | API key `devkey` / secret `devsecret-32-byte-min-length-padding-xyz` | +| Coturn | `coturn/coturn:4.11.0` | `udp+tcp/3478` (STUN/TURN), `tcp/5349` (TURN/TLS), `udp/49152-49200` (relay range) | TURN user `devuser` / password `devsecret` | LiveKit config at `infra/livekit/livekit.yaml`; Coturn config at `infra/coturn/turnserver.conf`. See [../livekit/README.md](../livekit/README.md) @@ -54,12 +54,12 @@ recording / consent / cost-tracking story. | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| Kafka (KRaft) | `confluentinc/cp-kafka:7.8.0` | `localhost:9092` (in-cluster only — see note below) | none (`PLAINTEXT`, `authType: none`) | -| kafka-ui | `provectuslabs/kafka-ui:v0.7.2` | `localhost:8081` | open UI (dev only) | -| Vault | `hashicorp/vault:1.18` | `localhost:8200` | root token `learnstack-dev-root-token` | -| Dapr placement | `daprio/placement:1.14.4` | `localhost:50005` | — | -| Dapr sidecar (api) | `daprio/daprd:1.14.4` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | -| APISIX | `apache/apisix:3.10.0-debian` | `localhost:9080` (HTTP), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | none (file-driven standalone — no Admin API) | +| Kafka (KRaft) | `confluentinc/cp-kafka:8.2.1` | `localhost:9092` (in-cluster only — see note below) | none (`PLAINTEXT`, `authType: none`) | +| kafka-ui | `ghcr.io/kafbat/kafka-ui:latest` | `localhost:8081` | open UI (dev only) | +| Vault | `hashicorp/vault:1.21.4` | `localhost:8200` | root token `learnstack-dev-root-token` | +| Dapr placement | `daprio/placement:1.17.7` | `localhost:50005` | — | +| Dapr sidecar (api) | `daprio/daprd:1.17.7` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | +| APISIX | `apache/apisix:3.16.0-debian` | `localhost:9080` (HTTP), `localhost:9443` (HTTPS), `localhost:9091` (metrics) | none (file-driven standalone — no Admin API) | Configs: diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index c9bb88d..5322cab 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -26,7 +26,7 @@ x-host-gateway: &host-gateway services: postgres: - image: postgres:16.6-alpine + image: postgres:16.14-alpine container_name: learnstack-postgres restart: unless-stopped environment: @@ -85,7 +85,7 @@ services: retries: 10 mailpit: - image: axllent/mailpit:v1.21 + image: axllent/mailpit:v1.29.7 container_name: learnstack-mailpit restart: unless-stopped ports: @@ -107,7 +107,7 @@ services: retries: 10 meilisearch: - image: getmeili/meilisearch:v1.11 + image: getmeili/meilisearch:v1.44.0 container_name: learnstack-meilisearch restart: unless-stopped environment: @@ -131,7 +131,7 @@ services: # one realm are never accepted on the other realm's endpoints. See # ../keycloak/README.md. keycloak: - image: quay.io/keycloak/keycloak:26.0 + image: quay.io/keycloak/keycloak:26.6.2 container_name: learnstack-keycloak restart: unless-stopped command: ["start-dev", "--import-realm"] @@ -180,7 +180,7 @@ services: # (Phase 08c); the LiveKit SDK is never imported by any module — only by # `LearnStack.Infrastructure.LiveClassroom.LiveKit`. livekit: - image: livekit/livekit-server:v1.8.0 + image: livekit/livekit-server:v1.12.0 container_name: learnstack-livekit restart: unless-stopped # `--node-ip 127.0.0.1` and `rtc.use_external_ip: false` in @@ -208,7 +208,7 @@ services: retries: 10 coturn: - image: coturn/coturn:4.6 + image: coturn/coturn:4.11.0 container_name: learnstack-coturn restart: unless-stopped command: ["-c", "/etc/turnserver.conf"] @@ -232,7 +232,7 @@ services: # Phase 07 (DX) ships either an explicit EXTERNAL listener or a # documented kafka-ui-only workflow. kafka: - image: confluentinc/cp-kafka:7.8.0 + image: confluentinc/cp-kafka:8.2.1 container_name: learnstack-kafka restart: unless-stopped environment: @@ -263,8 +263,11 @@ services: retries: 20 start_period: 30s + # Kafka UI — using the active community fork `kafbat/kafka-ui` because the + # original `provectuslabs/kafka-ui` was abandoned (no release since + # 2024-04). The fork preserves the same env-var contract. kafka-ui: - image: provectuslabs/kafka-ui:v0.7.2 + image: ghcr.io/kafbat/kafka-ui:latest container_name: learnstack-kafka-ui restart: unless-stopped environment: @@ -287,7 +290,7 @@ services: # If you change one you MUST change the other. Phase 07 (DX) wires both # to a single `.env.example` source so the duplication goes away. vault: - image: hashicorp/vault:1.18 + image: hashicorp/vault:1.21.4 container_name: learnstack-vault restart: unless-stopped command: ["server", "-dev", "-dev-root-token-id=learnstack-dev-root-token", "-dev-listen-address=0.0.0.0:8200"] @@ -310,7 +313,7 @@ services: # Placement service required by daprd even though actors are out of scope # per ADR-0014 non-goals (daprd will not boot without it). dapr-placement: - image: daprio/placement:1.14.4 + image: daprio/placement:1.17.7 container_name: learnstack-dapr-placement restart: unless-stopped command: ["./placement", "-log-level", "info"] @@ -325,7 +328,7 @@ services: # anchor at the top of the file maps `host.docker.internal` on Linux. # See ../dapr/README.md. dapr-sidecar-api: - image: daprio/daprd:1.14.4 + image: daprio/daprd:1.17.7 container_name: learnstack-dapr-sidecar-api restart: unless-stopped <<: *host-gateway @@ -373,7 +376,7 @@ services: # the standalone-no-etcd commitment of ADR-0015. Diff-review of # `apisix.yaml` replaces the dashboard for the dev workflow. apisix: - image: apache/apisix:3.10.0-debian + image: apache/apisix:3.16.0-debian container_name: learnstack-apisix restart: unless-stopped <<: *host-gateway From 9c4a0c62ff08ad452c7f4b9de2442f6dde0c96aa Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 18:22:58 +0300 Subject: [PATCH 06/11] docs(adr): ADR-0029 SeaweedFS supersedes MinIO storage row MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lands the architectural decision that the dev compose + the `IStorageProvider` adapter will follow when switching object-storage backends. The chain - ADR-0029 is the new decision (Accepted, 2026-05-19): self-hosted SeaweedFS behind the existing S3-shaped `IStorageProvider` contract, Apache 2.0 license, no phone-home, single-binary self-hostable, S3 gateway compatibility sufficient for everything `IStorageProvider` consumes today. - ADR-0002 gets Amendment 1 narrowed to the storage row only — every other choice in ADR-0002 (.NET 10, EF Core, Postgres, Redis, Next.js, modular monolith) is unchanged. The amendment cites the trigger (MinIO `minio/minio` repo archived 2026-04 + the licensing trajectory that removed the no-phone-home posture ADR-0020 Self-Hosted Air-Gapped depends on) and points forward to ADR-0029. - `decisions/README.md` active-ADR table gets a row for 0029 in its numeric slot. Hard-rule check - ADR numbers stay sequential (0029 is the next free slot; 0023-0028 are reserved drafts per the existing reservation table). ADR-0029 takes the next unused number, not a reserved one. - ADR-0002's Decision section is not rewritten; the Amendment-block pattern this repo already uses for ADR-0003/0004/0006/0010 is applied verbatim. - No fifth Hub endpoint, no Verticals folder, no domain-flavoured names; ADR-0014's three Dapr building blocks unchanged. The compose service swap + the 27-doc reference sweep land in a separate commit so the ADR can be reviewed in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/decisions/0002-initial-architecture.md | 30 ++- .../0029-object-storage-seaweedfs.md | 203 ++++++++++++++++++ docs/decisions/README.md | 1 + 3 files changed, 231 insertions(+), 3 deletions(-) create mode 100644 docs/decisions/0029-object-storage-seaweedfs.md diff --git a/docs/decisions/0002-initial-architecture.md b/docs/decisions/0002-initial-architecture.md index e8abea6..8a26484 100644 --- a/docs/decisions/0002-initial-architecture.md +++ b/docs/decisions/0002-initial-architecture.md @@ -2,11 +2,14 @@ ## Status -Accepted +Accepted (Amendment 1: 2026-05-19 — storage backend changed from MinIO to +SeaweedFS per [ADR-0029](0029-object-storage-seaweedfs.md); see Amendment at +the bottom of this document. Every other choice in this ADR — .NET 10, +ASP.NET Core, EF Core, PostgreSQL, Redis, Next.js, modular monolith — stands.) ## Decision -LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL, Redis, MinIO, and Next.js. +LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL, Redis, SeaweedFS (see Amendment 1), and Next.js. ## Context @@ -20,6 +23,27 @@ The team has stronger familiarity with .NET, so .NET 10 is preferred over Go for - EF Core should be the default ORM. - PostgreSQL should be the primary database. - Redis should be used for caching and distributed coordination where needed. -- MinIO should be used locally for S3-compatible object storage. +- SeaweedFS should be used locally for S3-compatible object storage (Amendment 1). - Next.js should be used for public rendering, admin studio, and portals initially. +--- + +## Amendment 1 — Storage backend: MinIO → SeaweedFS (2026-05-19) + +The original storage row picked MinIO. Two things changed: + +1. The `minio/minio` container repository was archived in 2026-04 with no + ongoing free-tier image stream. +2. MinIO Inc.'s licensing trajectory removed the no-phone-home, + no-license-key posture that the Self-Hosted Air-Gapped deployment mode + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires. + +[ADR-0029](0029-object-storage-seaweedfs.md) records the replacement +decision: SeaweedFS sits behind the same `IStorageProvider` S3-shaped +contract (no code-level adapter contract change), Apache 2.0 licensed, +single-binary self-hostable. The rest of ADR-0002 is unchanged. + +Every doc that previously said "MinIO" should be read as "SeaweedFS" for +operational guidance; conceptual rules ("tenant key-prefix isolation", +"swap to AWS S3 in SaaS through `IStorageProvider`") were never +backend-specific and stand verbatim. diff --git a/docs/decisions/0029-object-storage-seaweedfs.md b/docs/decisions/0029-object-storage-seaweedfs.md new file mode 100644 index 0000000..196983c --- /dev/null +++ b/docs/decisions/0029-object-storage-seaweedfs.md @@ -0,0 +1,203 @@ +# ADR-0029: Object Storage — SeaweedFS + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "MinIO" choice on +the storage row only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **Original choice (MinIO) lost its free-tier path.** The MinIO Inc. policy + shift around the AGPL community edition and the archival of the official + `minio/minio` container repository in 2026-04 mean there is no supported + upstream stream of new tagged images for the self-hosted, no-license-server + posture LearnStack relies on. Continuing on the last free image + (`RELEASE.2025-09-07T16-13-09Z`) is a frozen-image dead end, not a + maintainable foundation. +- **Provider portability is non-negotiable.** Per + [ADR-0002](0002-initial-architecture.md) + [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md), + every external dependency sits behind a `LearnStack`-defined interface. + `IStorageProvider` already abstracts the storage adapter — the choice of + backend behind the interface should be an operational call, not a code + rewrite. +- **Self-Hosted + Air-Gapped must work.** The triple-deployment model + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires the + storage backend to ship as a single self-contained binary or container + with no phone-home, no license-key check, no SaaS dependency. +- **S3 API compatibility is the integration contract.** The `IStorageProvider` + adapter that the rest of the stack talks to is written against the S3 API + (signed URLs, multipart upload, key-prefix isolation per + [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md)). + Any backend that does not expose a high-fidelity S3 API forces a second + adapter layer. +- **Day-1 multi-region story is a Phase-11 concern, not Day-1.** The MVP + scope is single-region; the chosen backend must not paint us into a + corner if Phase 11 adds replication, but multi-region is not required + today. +- **Operational footprint must stay one-process-small.** The dev compose + is already 14 services; a storage backend that introduces three or four + more processes (or a dedicated control plane) is disproportionate cost. + +## Considered Options + +1. **SeaweedFS (chosen).** Self-hosted single binary, native S3-compatible + API, Apache 2.0 license, active upstream, optional volume + filer + S3 + gateway split-or-merged per deployment size. +2. **Stay on the last free MinIO image, indefinitely (rejected).** + `minio/minio:RELEASE.2025-09-07T16-13-09Z` works today but freezes the + stack to a no-longer-supported image — including security patches. This + makes ADR-0002's storage row a long-term tech debt magnet. +3. **Garage S3 (rejected for now).** Rust-based, Apache 2.0, smaller + feature set. Strong fit for edge / small deployments; weaker + production-scale story and a smaller operational community. Worth + revisiting if SeaweedFS develops a blocker. +4. **Ceph RGW (rejected).** Heavy-weight; designed for petabyte clusters + with a separate operator team. Disproportionate for LearnStack's + single-binary Self-Hosted posture; production multi-region story is + solid but is solving a problem we do not yet have. +5. **Cloud-managed S3 only (rejected — incompatible with deployment + model).** Amazon S3 / R2 / B2 are excellent for SaaS; they directly + violate Self-Hosted Air-Gapped (ADR-0020). A SaaS-only deployment + could swap behind `IStorageProvider`, but the *default* must be + self-hostable. + +## Decision + +LearnStack adopts **SeaweedFS** as the object-storage backend behind the +`IStorageProvider` adapter for all four deployment modes (Development / +SaaS / Dedicated / SelfHosted). The S3 gateway exposes the same S3 API +surface the MinIO-based adapter relied on, so adapter code remains +unchanged in signature. + +Image: `chrislusf/seaweedfs:latest` (pinned to a specific tag per +[Standards 12 § Image Conventions](../standards/12-infrastructure.md); +the dev compose pins the current stable tag). + +This ADR **supersedes the storage choice in ADR-0002 only** — the rest of +ADR-0002 (Postgres, Redis, modular monolith) is unchanged. + +## Context + +The repository accreted ~27 docs that name MinIO directly (Standards 11, +Standards 12, Standards 20, ADR-0002, ADR-0003, ADR-0014, ADR-0017, +ADR-0018, architecture 02 / 03 / 04 / 05 / 07 / 08 / 09 / 16 / 18 / 23 / +25 / 29 / 32, decisions/README). The Day-1 commitment was: provider +portability behind `IStorageProvider`, single-binary self-hostable +backend, S3 API as the integration contract. MinIO satisfied all three at +the time; SeaweedFS satisfies all three today **and** is not on a +license-shift trajectory. + +The bulk of the MinIO references describe **storage characteristics that +remain true** — tenant key-prefix isolation (`{tenant_id}/...`), +production swap-out to AWS S3, partition-friendly key layout. Those rules +do not change. The backend name does, and a tightly-scoped doc sweep +plus a runtime config swap close out the migration. + +### Why MinIO got us here and why we're moving off + +MinIO was the right choice in 2024–early 2026 because it shipped as a +self-hosted binary, exposed a high-fidelity S3 API, and stayed on a +permissive license. The 2026-04 archival of the `minio/minio` Docker Hub +repository plus MinIO Inc.'s pivot to a commercial-first posture removed +the *no-license-server, no phone-home* path the Self-Hosted Air-Gapped +deployment mode (ADR-0020) requires. The platform-level commitment is to +keep that mode first-class; the backend choice must follow. + +### Why SeaweedFS + +- **Single binary**, multi-process when scale needs it. The dev compose + runs one container; production can split master / volume / filer / S3 + gateway across replicas. +- **Native S3 gateway** (`weed s3` subcommand) exposing a near-complete + S3 API surface — signed URLs, multipart upload, bucket policies, key + versioning. Sufficient for what `IStorageProvider` consumes today. +- **Apache 2.0 license**, no telemetry phone-home, no license key + required. +- **Active upstream** (commits weekly, stable release cadence). +- **Operational primitives we already need**: tiered storage (hot → + warm), erasure coding (Phase 11 cost-optimization), built-in metrics + (Prometheus-compatible). + +### Why not Garage + +Garage's feature set is tighter (no native erasure coding; smaller +ecosystem). The trade-off is conscious and we re-consider Garage if +SeaweedFS produces a concrete blocker — at that point Garage is one +ADR + one adapter-config swap away. + +### Adapter-level impact: none + +`IStorageProvider` already abstracts the storage backend behind a S3- +shaped contract (signed URL issuance, multipart upload coordination, +bucket + key access, server-side encryption). The SeaweedFS S3 gateway +implements the subset `IStorageProvider` uses. The composition root in +each deployment mode picks the backend's endpoint + credentials; no +module-level code change is required. + +## Consequences + +### Positive + +- Self-Hosted Air-Gapped stays first-class — no phone-home, no license + key, no commercial-tier gating. +- The dev compose stays single-container for storage (no operator- + pattern overhead at dev time). +- Apache 2.0 license matches the rest of LearnStack's OSS posture + (Standards 20 § Self-Hosted Infrastructure Preferred). +- Production tiered storage + erasure coding give us a clear cost- + reduction lever in Phase 11 without a backend swap. +- The decision **also** answers "what happens if MinIO Inc. tightens + licensing further" — we're already off. + +### Negative + +- One-time doc + scaffold migration: ~27 doc references + the dev + compose service + the `IStorageProvider` adapter implementation + (Phase 02a). The adapter change is small because both backends speak + S3, but the doc sweep is mechanical work. +- SeaweedFS S3 gateway is not 100% S3 surface — a few edge endpoints + (CloudFront-specific extensions, S3 Object Lambda) do not exist. + None are on the LearnStack-side dependency list today; flagging so + Phase 02a's adapter-write does not assume them. +- Smaller operational community than MinIO at its peak. Mitigated by + active upstream and the architecture-level provider-portability + commitment — switching backends behind `IStorageProvider` remains a + composition-root edit. + +### Neutral + +- The `MINIO_` env-var names in the dev compose become `SEAWEEDFS_` (or + the equivalent SeaweedFS variables). Names move; semantics do not. +- Production AWS S3 / Cloudflare R2 / Backblaze B2 swap-in continues to + work through `IStorageProvider`; the dev backend choice is independent + of the production backend choice. + +## Implementation Notes + +- Phase 01 packet 6 cleanup (this ADR's commit): the dev compose `minio` + service is replaced by a `seaweedfs` service exposing the S3 gateway + on the same `localhost:9000` port (drop-in for any dev URL that + hardcoded the old endpoint), with the master / volume processes on + their own internal ports. The MinIO console (port 9001) becomes + SeaweedFS's filer UI. +- Phase 02a (Storage adapter): `LearnStack.Infrastructure.Storage.SeaweedFS` + ships the `IStorageProvider` implementation. The signed-URL, + multipart-upload, and bucket-prefix conventions in + [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md) + stay verbatim — the adapter speaks S3 underneath. +- Phase 11 (production): erasure coding policy + tiered storage tuning; + multi-region replication evaluation; production credential rotation + (today's dev credentials live in dev compose only). + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original storage row, now superseded for this slot. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) — Self-Hosted Air-Gapped requirement that motivated the move. +- [Standards 12 § Object Storage Operations](../standards/12-infrastructure.md) +- [Standards 20 § Self-Hosted Infrastructure](../standards/20-infrastructure-stack.md) +- [architecture/16-media-pipeline.md](../architecture/16-media-pipeline.md) — media + recording storage paths. +- SeaweedFS upstream: . diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 0e9f743..2c6d668 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -37,6 +37,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0020 | [Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) | SaaS / Dedicated / Self-Hosted from one codebase; phone-home + RSA-signed key + 30-day grace | | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | +| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | ## Superseded ADRs From 0d7aad24e55eb1545d214fde5107b918a018bf7f Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 18:33:20 +0300 Subject: [PATCH 07/11] feat(infra): replace MinIO with SeaweedFS across dev compose + docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lands the ADR-0029 decision: the dev compose `minio` service becomes `seaweedfs` (single binary packing master + volume + filer + S3 gateway) and every MinIO reference in the corpus moves to SeaweedFS — except the two intentional historical mentions in ADR-0002 (Amendment 1) and ADR-0029 (the decision record), plus the inline "replaces / matches the prior MinIO endpoint" notes in dev.yml + compose README + seaweedfs/README that document the migration. The S3 access surface is a drop-in: SeaweedFS S3 gateway is port-mapped to the same host ports MinIO used (9000 = S3 API, 9001 = filer UI in place of the old MinIO console), and the dev S3 credentials match the prior defaults (`learnstack` / `learnstack-dev-secret`) so any local script, env var, or test fixture continues to work unchanged. Adds - `infra/compose/dev.yml`: `seaweedfs` service (`chrislusf/seaweedfs:3.94`, command-mode `server -dir=/data -master -volume -filer -s3 -s3.port=8333 -s3.config=/etc/s3-identities.json -metricsPort=9091`), healthcheck against `/cluster/healthz` on the master HTTP API. - `infra/seaweedfs/s3-identities.json` — S3 identity config matching the prior MinIO root credentials (dev-only; production loads from Vault). - `infra/seaweedfs/README.md` — access surface, dev credentials, tenant key-prefix isolation rule (unchanged), re-seed procedure, what does NOT live here. Removes - `infra/compose/dev.yml`: `minio` service definition + `MINIO_*` env vars + the `minio-data` named volume (replaced by `seaweedfs-data`). - Per-file `MinIO`/`minio` references across 26 docs (Standards 00 / 06 / 09 / 10 / 11 / 12 / 15 / 20; Architecture 02 / 03 / 04 / 05 / 07 / 08 / 09 / 16 / 18 / 23 / 25 / 29 / 32; Decisions 0003 / 0014 / 0017 / 0018 / README), plus glossary, root README, CLAUDE.md, 4 roadmap phase docs, `.gitignore` (`minio-data/` → `seaweedfs-data/`), and 4 skill files. Preserved (intentional) - `docs/decisions/0002-initial-architecture.md` — Amendment 1 names the swap explicitly. - `docs/decisions/0029-object-storage-seaweedfs.md` — the decision record. - `infra/compose/dev.yml` inline comments — "SeaweedFS replaces MinIO", "matches the prior MinIO endpoint", "matches prior MinIO credentials", "Filer UI (replaces the MinIO console)" — historical context for the next reader. - `infra/compose/README.md` — "applied to MinIO, it applies to SeaweedFS" — the tenant-key-prefix rule is backend-independent. - `infra/seaweedfs/README.md` — references the prior MinIO surface for drop-in continuity. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - Compose-config expansion shows `seaweedfs` service + `seaweedfs-data` volume; `minio` references are gone outside the four intentional sites. - All markdown link sweeps on changed docs clean. - Live `docker compose up` smoke test deferred (Docker daemon not running on this workstation); will ride along with Phase 07's `make dev` smoke. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/README.md | 2 +- .claude/skills/add-integration-test/SKILL.md | 4 +- .claude/skills/local-dev-setup/SKILL.md | 12 ++-- .claude/skills/standards-check/SKILL.md | 2 +- .gitignore | 2 +- CLAUDE.md | 4 +- README.md | 6 +- docs/architecture/02-domain-model.md | 2 +- docs/architecture/03-module-boundaries.md | 4 +- .../architecture/04-technical-architecture.md | 14 ++--- docs/architecture/05-mvp-scope.md | 2 +- docs/architecture/07-in-app-live-classroom.md | 8 +-- docs/architecture/08-livekit-cost-model.md | 2 +- docs/architecture/09-tenant-isolation.md | 6 +- docs/architecture/16-media-pipeline.md | 14 ++--- docs/architecture/18-webrtc-build-vs-adopt.md | 2 +- docs/architecture/23-data-protection.md | 6 +- docs/architecture/25-deployment-models.md | 6 +- docs/architecture/29-dapr-integration.md | 2 +- .../32-tenant-customization-model.md | 2 +- .../0003-tenant-isolation-defense-in-depth.md | 2 +- docs/decisions/0014-adopt-dapr.md | 2 +- .../0017-tenant-organization-hierarchy.md | 2 +- .../0018-tenant-driven-customization-model.md | 2 +- docs/decisions/README.md | 4 +- docs/glossary.md | 4 +- docs/roadmap/phase-00-product-architecture.md | 2 +- docs/roadmap/phase-01-repository-tooling.md | 8 +-- docs/roadmap/phase-04-cms-media-pages.md | 2 +- docs/roadmap/phase-08c-classroom.md | 4 +- docs/standards/00-principles.md | 4 +- docs/standards/06-testing.md | 4 +- docs/standards/09-error-handling.md | 2 +- docs/standards/10-observability.md | 2 +- docs/standards/11-security.md | 2 +- docs/standards/12-infrastructure.md | 6 +- docs/standards/15-performance.md | 2 +- docs/standards/20-infrastructure-stack.md | 2 +- infra/compose/README.md | 12 +++- infra/compose/dev.yml | 51 ++++++++++------ infra/livekit/README.md | 2 +- infra/seaweedfs/README.md | 61 +++++++++++++++++++ infra/seaweedfs/s3-identities.json | 14 +++++ 43 files changed, 198 insertions(+), 100 deletions(-) create mode 100644 infra/seaweedfs/README.md create mode 100644 infra/seaweedfs/s3-identities.json diff --git a/.claude/skills/README.md b/.claude/skills/README.md index 7f3aa53..f13ceaa 100644 --- a/.claude/skills/README.md +++ b/.claude/skills/README.md @@ -114,7 +114,7 @@ index plus the relevant [Architecture](../../docs/architecture/) doc. | Skill | When to use | |-------|-------------| -| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Redis, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, MinIO, LiveKit, Meilisearch. | +| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Redis, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, SeaweedFS, LiveKit, Meilisearch. | | [seed-tenant](seed-tenant/SKILL.md) | Provisioning a tenant (with its default organization, customization data, seed users) for local development. | ## Authoring a new skill diff --git a/.claude/skills/add-integration-test/SKILL.md b/.claude/skills/add-integration-test/SKILL.md index f3c29f0..7a0adc0 100644 --- a/.claude/skills/add-integration-test/SKILL.md +++ b/.claude/skills/add-integration-test/SKILL.md @@ -31,7 +31,7 @@ architecture test) plus any other invariant the change touches. See - Outbox → consumer round-trip. - Audit-pipeline-writes-the-expected-row. - Provider-adapter contract test that talks to a containerised real provider - (LiveKit OSS, MinIO, Meilisearch). + (LiveKit OSS, SeaweedFS, Meilisearch). ## When not to use @@ -45,7 +45,7 @@ architecture test) plus any other invariant the change touches. See |-------|----------|-------------| | Scenario | Yes | A short name + setup + act + assert. | | Seed | Yes | Minimum tenants / orgs / users / customization data the scenario needs. | -| Required containers | Yes | Postgres always; add Redis / Kafka / Meilisearch / LiveKit / MinIO as needed. | +| Required containers | Yes | Postgres always; add Redis / Kafka / Meilisearch / LiveKit / SeaweedFS as needed. | | Tenant context | Yes | Which tenant + org the act phase runs as. | ## Workflow diff --git a/.claude/skills/local-dev-setup/SKILL.md b/.claude/skills/local-dev-setup/SKILL.md index eab515a..e72a554 100644 --- a/.claude/skills/local-dev-setup/SKILL.md +++ b/.claude/skills/local-dev-setup/SKILL.md @@ -2,7 +2,7 @@ name: local-dev-setup description: > Bring up the LearnStack local stack — Postgres, Redis, Vault, Kafka, Dapr - sidecar, APISIX, Keycloak (two realms), MinIO, LiveKit OSS, Meilisearch — via + sidecar, APISIX, Keycloak (two realms), SeaweedFS, LiveKit OSS, Meilisearch — via `docker-compose` plus the project's `make dev` orchestrator. USE FOR: first-time workstation setup, restoring a broken local environment, switching between `DeploymentMode` for testing. DO NOT USE FOR: production deployment (separate @@ -15,7 +15,7 @@ description: > ## Purpose Stand up a full LearnStack stack on a developer workstation so backend + frontend -can run against real Postgres / Redis / Kafka / Vault / Keycloak / MinIO / +can run against real Postgres / Redis / Kafka / Vault / Keycloak / SeaweedFS / LiveKit / Meilisearch / APISIX — the same components production uses ([12-infrastructure.md § Local Infrastructure](../../../docs/standards/12-infrastructure.md), [20-infrastructure-stack.md](../../../docs/standards/20-infrastructure-stack.md)). @@ -103,7 +103,7 @@ The components and their default ports: | Kafka UI | 9094 | Optional UI for topics. | | Vault (dev mode) | 8200 | Secrets backend; `root` token, **not** for production. | | Keycloak | 8080 | Two realms: `learnstack` (tenants) + `learnstack-hub` (operators). | -| MinIO | 9000 (API) / 9001 (console) | Object storage. | +| SeaweedFS | 9000 (S3 API) / 9001 (filer UI) / 9333 (master) | Object storage (single dev binary: master + volume + filer + S3 gateway). | | Meilisearch | 7700 | Search. | | LiveKit | 7880 (API) / 7881 (TLS) / 7882 (RTC) | Live classroom. | | coturn | 3478 / 5349 | TURN for LiveKit. | @@ -128,7 +128,7 @@ The first `make dev` run additionally: 4. Seeds the two demo tenants' customization data (`TenantContentType`, `TenantPageBlock`, `TenantLevelTaxonomy`, …) so the page renderer has something to render. -5. Creates the MinIO buckets. +5. Creates the SeaweedFS buckets. 6. Creates the Meilisearch indexes. For a clean re-seed: @@ -150,8 +150,8 @@ curl -fsS http://localhost:9080/healthz | jq open http://localhost:8080/realms/learnstack/account open http://localhost:8080/realms/learnstack-hub/account -# MinIO console -open http://localhost:9001 # minioadmin / minioadmin +# SeaweedFS filer UI (replaces the MinIO console of the prior stack) +open http://localhost:9001 # S3 access: learnstack / learnstack-dev-secret # Web app open http://localhost:3000 # one of the demo tenants diff --git a/.claude/skills/standards-check/SKILL.md b/.claude/skills/standards-check/SKILL.md index 4ea8da7..1a76c18 100644 --- a/.claude/skills/standards-check/SKILL.md +++ b/.claude/skills/standards-check/SKILL.md @@ -92,7 +92,7 @@ Walk every item in [CLAUDE.md § Hard rules](../../../CLAUDE.md) and - [ ] **Tenant + organization isolation defense-in-depth from day one** — marker + filter + RLS + arch test wherever a tenant-owned entity lands. - [ ] **Self-hosted infrastructure preferred** (Keycloak with two realms, - LiveKit OSS, MinIO, Meilisearch, Kafka, Vault). + LiveKit OSS, SeaweedFS, Meilisearch, Kafka, Vault). - [ ] **No domain-specific code in any module.** Domain shape = tenant data per ADR-0018. - [ ] **Foundation building blocks are Day-1.** Dapr / APISIX / audit diff --git a/.gitignore b/.gitignore index b402073..2a4ef79 100644 --- a/.gitignore +++ b/.gitignore @@ -37,7 +37,7 @@ logs/ # Local infrastructure data .data/ -minio-data/ +seaweedfs-data/ postgres-data/ redis-data/ diff --git a/CLAUDE.md b/CLAUDE.md index 876c25d..7d7a88c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,7 +26,7 @@ scaffold under `backend/` (core + 7 modules × 4 projects + 4 test projects including the non-skippable `LearnStack.Tests.Architecture`), the `pnpm` frontend monorepo under `frontend/` (`apps/web` Next.js App Router + `packages/{config,ui,sdk}`), and the core local-dev compose -stack at `infra/compose/dev.yml` (Postgres / Redis / MinIO / Mailpit / +stack at `infra/compose/dev.yml` (Postgres / Redis / SeaweedFS / Mailpit / Meilisearch). The remaining Phase-01 packets — Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, `.env.example`, CI, `make seed` — land incrementally; see @@ -88,7 +88,7 @@ let the entry point pick it. - **Standards changes cite an ADR.** A new standard rule or a change to an existing one is paired with an ADR when the rule is non-trivial. - **Modular monolith with four cross-module mechanisms** ([ADR-0010](docs/decisions/0010-cross-module-communication.md)): application contract, intra-module domain event, integration event via outbox (dispatched through Dapr pub/sub per Amendment 1), read-model projection. No fifth. - **Tenant + organization isolation is defense-in-depth from day one** ([ADR-0003 Amendment 1](docs/decisions/0003-tenant-isolation-defense-in-depth.md), [ADR-0017](docs/decisions/0017-tenant-organization-hierarchy.md)): tenant + organization context + EF query filters + PostgreSQL RLS + architecture tests. -- **Self-hosted infrastructure preferred** for Keycloak (auth, with two realms — `learnstack` + `learnstack-hub`), LiveKit OSS (live classroom), MinIO (object storage), Meilisearch (search), Kafka (pub/sub backend), Vault (secrets). See ADRs 0004, 0005, 0014. +- **Self-hosted infrastructure preferred** for Keycloak (auth, with two realms — `learnstack` + `learnstack-hub`), LiveKit OSS (live classroom), SeaweedFS (object storage), Meilisearch (search), Kafka (pub/sub backend), Vault (secrets). See ADRs 0004, 0005, 0014. - **The core platform stays domain-generic.** Domain-specific shapes (CEFR levels, English placement-test scoring, kyu/dan ranks, yoga asana catalogs, …) live as **tenant customization data** ([ADR-0018](docs/decisions/0018-tenant-driven-customization-model.md)), never as code in any module. There is no `Verticals/` folder. ADR-0011 is superseded. - **Foundation building blocks are Day-1, not Phase-11.** Dapr (`IEventBus`/`ICacheService`/`ISecretProvider`), APISIX gateway, audit infrastructure, organization scope, entitlement projection socket, host-to-tenant resolver, and architecture tests all ship in Phase 02a — not as later hardening. - **Provider adapters everywhere.** Payments, auth, storage, search, live classroom, notifications, **event bus, cache, secrets, Hub HTTPS contract, entitlement source, host resolver** — all sit behind interfaces. No SaaS lock-in in `Domain` or `Application`. See [20-infrastructure-stack.md](docs/standards/20-infrastructure-stack.md). diff --git a/README.md b/README.md index d2f9e32..5a32761 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ Dedicated control plane, plan editor, custom-domain admin, and license-key issua Phase 01 in progress. The repository now holds the .NET 10 solution scaffold (7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` architecture test), the `pnpm` frontend monorepo (`apps/web` + `packages/{config,ui,sdk}`), -and the core local-dev `docker-compose` stack (Postgres / Redis / MinIO / Mailpit / +and the core local-dev `docker-compose` stack (Postgres / Redis / SeaweedFS / Mailpit / Meilisearch). Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, and CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). @@ -35,7 +35,7 @@ CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository [ADR-0014](docs/decisions/0014-adopt-dapr.md). - **API Gateway:** **APISIX** in standalone YAML-reload mode per [ADR-0015](docs/decisions/0015-api-gateway-apisix.md). -- **Object storage:** MinIO locally, S3-compatible storage in production. +- **Object storage:** SeaweedFS locally, S3-compatible storage in production. - **Search:** Meilisearch initially. - **Frontend:** Next.js 15 (App Router), TypeScript, React. **One** application (`apps/web`) with route segments for public, studio, and portal — multi-app split @@ -57,7 +57,7 @@ CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository - **Live classroom:** In-app WebRTC; **self-hosted LiveKit OSS** is the default; LiveKit Cloud available behind the same `ILiveClassProvider` interface. A custom WebRTC SFU is explicitly out of scope. -- **Recording:** Supported via LiveKit Egress to S3/MinIO; tenant-configurable; +- **Recording:** Supported via LiveKit Egress to S3/SeaweedFS; tenant-configurable; consent-aware; off by default. - **Deployment:** Triple deployment model per [ADR-0020](docs/decisions/0020-triple-deployment-hybrid-license.md) — diff --git a/docs/architecture/02-domain-model.md b/docs/architecture/02-domain-model.md index 1217491..0edb2ba 100644 --- a/docs/architecture/02-domain-model.md +++ b/docs/architecture/02-domain-model.md @@ -348,7 +348,7 @@ These entities cover the **runtime** of a Live Session: actual rooms, tokens, re | `LiveRoom` | Yes | Runtime room created via `ILiveClassProvider`. Lives for the duration of a `LiveSession`. | | `LiveRoomToken` | Inside LiveRoom | Short-lived join token; scoped to user + room + role. | | `LiveRoomProvider` | Reference | The provider implementation that owns this room. | -| `LiveRecording` | Yes | Recording metadata, consent state, retention. File lives in MinIO/S3. | +| `LiveRecording` | Yes | Recording metadata, consent state, retention. File lives in SeaweedFS/S3. | | `LiveSessionEvent` | Append-only | join / leave / screen-share / recording started / network drop. Feeds `ClassroomEvent` analytics. | See [In-App Live Classroom](07-in-app-live-classroom.md) for the provider abstraction. diff --git a/docs/architecture/03-module-boundaries.md b/docs/architecture/03-module-boundaries.md index 94b830c..0506530 100644 --- a/docs/architecture/03-module-boundaries.md +++ b/docs/architecture/03-module-boundaries.md @@ -178,7 +178,7 @@ Per-tenant block shapes resolve through Tenant Customization (`TenantPageBlock`). ### Media -Owns media assets, object storage metadata, file lifecycle, variants, asset access policies. Knows about MinIO/S3 via the storage provider adapter. +Owns media assets, object storage metadata, file lifecycle, variants, asset access policies. Knows about SeaweedFS/S3 via the storage provider adapter. ### Education Catalog Owns programs, courses, course versions, categories, levels, tags, instructor profiles, @@ -261,7 +261,7 @@ backend/ LearnStack.Api/ # ASP.NET host (single deployment unit) LearnStack.Application/ # composition root, MediatR pipeline LearnStack.Domain/ # shared kernel domain pieces - LearnStack.Infrastructure/ # EF, Redis, MinIO, OpenTelemetry, Dapr wiring + LearnStack.Infrastructure/ # EF, Redis, SeaweedFS, OpenTelemetry, Dapr wiring LearnStack.Infrastructure.Audit/ # audit interceptor + state capture + MediatR behavior LearnStack.SharedKernel/ # ids, audit fields, errors, paging, IEventBus, # ICacheService, ISecretProvider, IEntitlementProvider diff --git a/docs/architecture/04-technical-architecture.md b/docs/architecture/04-technical-architecture.md index 9cab4fd..bc4b6c1 100644 --- a/docs/architecture/04-technical-architecture.md +++ b/docs/architecture/04-technical-architecture.md @@ -12,7 +12,7 @@ | Pub/Sub | **Apache Kafka via Dapr Pub/Sub** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) — outbox dispatch target | | Secrets | **HashiCorp Vault via Dapr Secret Store** (or env-var fallback in Dev) | | Distributed runtime | **Dapr 1.14+** sidecar pattern (pub/sub, state, secrets) | -| Object storage | MinIO (local), S3-compatible (production) | +| Object storage | SeaweedFS (local), S3-compatible (production) | | Background jobs | Hangfire (Postgres storage) | | Search | Meilisearch (initial), OpenSearch (later, if needed). See [ADR 0012](../decisions/0012-search-strategy.md) | | Auth | Keycloak (self-hosted OIDC) — two realms: `learnstack` (tenant users) + `learnstack-hub` (operators); ADR-0004 Amendment 1 | @@ -53,7 +53,7 @@ flowchart LR subgraph data["Data Plane"] pg[(PostgreSQL)] redis[(Redis)] - minio[(MinIO / S3)] + seaweedfs[(SeaweedFS / S3)] meili[(Meilisearch)] kafka[(Kafka)] vault[(HashiCorp Vault)] @@ -87,13 +87,13 @@ flowchart LR daprd2 --> vault api --> pg - api --> minio + api --> seaweedfs api --> meili api --> kc_main api --> lk workers --> pg - workers --> minio - egress --> minio + workers --> seaweedfs + egress --> seaweedfs lk --> egress hubapi -- "mTLS + signed JWT + HMAC
POST /api/internal/*" --> api @@ -123,7 +123,7 @@ Module boundaries and dependency rules are in [Module Boundaries](03-module-boun | `Api` | HTTP endpoints, auth middleware, request binding, OpenAPI emission, tenant resolution middleware. | | `Application` | Use cases (MediatR commands/queries), validation, transactions, pipeline behaviors. | | `Domain` | Entities, aggregates, value objects, domain services, domain events. | -| `Infrastructure` | EF Core, Redis, MinIO, Hangfire, OpenTelemetry, external adapters. | +| `Infrastructure` | EF Core, Redis, SeaweedFS, Hangfire, OpenTelemetry, external adapters. | | `Modules.*` | Bounded feature areas, each with their own `Application` / `Domain` / `Infrastructure` internals and a public `Application.Contracts` surface. | ## Multi-Tenancy @@ -208,7 +208,7 @@ Detailed conventions: [Frontend Architecture](14-frontend-architecture.md) and [ ``` postgres redis -minio + minio-console +seaweedfs # single dev binary: master + volume + filer + S3 gateway meilisearch keycloak livekit-server diff --git a/docs/architecture/05-mvp-scope.md b/docs/architecture/05-mvp-scope.md index 668e104..2779aa0 100644 --- a/docs/architecture/05-mvp-scope.md +++ b/docs/architecture/05-mvp-scope.md @@ -121,7 +121,7 @@ customization aggregates are first-class in MVP: - Draft / published workflow with preview tokens. ### Media -- Upload to MinIO via signed URLs. +- Upload to SeaweedFS via signed URLs. - Asset metadata, folders, tags. - Image variants on upload. - Use assets inside content and page blocks. diff --git a/docs/architecture/07-in-app-live-classroom.md b/docs/architecture/07-in-app-live-classroom.md index d9a5a75..96dbf28 100644 --- a/docs/architecture/07-in-app-live-classroom.md +++ b/docs/architecture/07-in-app-live-classroom.md @@ -65,7 +65,7 @@ flowchart LR subgraph Infra REDIS[(Redis)] - S3[(S3 / MinIO)] + S3[(S3 / SeaweedFS)] TURN[Coturn TURN/STUN] end @@ -87,7 +87,7 @@ The flow: 2. LearnStack verifies enrollment / session permission and issues a **scoped, short-lived LiveKit join token**. 3. The client connects to LiveKit OSS via WebRTC (`wss://livekit.`), using TURN relay only when direct connectivity is blocked. 4. LiveKit emits webhooks (`participant_joined`, `track_published`, `room_finished`, etc.); LearnStack records them as `LiveSessionEvent` rows. -5. Optional recording: LearnStack triggers a LiveKit Egress job; the Egress worker writes the composite recording to S3/MinIO, and LearnStack stores `LiveRecording` metadata. +5. Optional recording: LearnStack triggers a LiveKit Egress job; the Egress worker writes the composite recording to S3/SeaweedFS, and LearnStack stores `LiveRecording` metadata. ## Provider-Agnostic Core @@ -180,7 +180,7 @@ Defaults: - Recording is **off by default** at the tenant level. - A tenant administrator can enable it for the entire tenant, for a course, or for a specific session. - Composite (single-file) recording is the default mode. Track-based recording is available for advanced post-processing scenarios. -- Recordings are written to S3/MinIO; retention defaults to **30 days** and is configurable per tenant up to a tenant-wide retention cap. See [16-media-pipeline.md](16-media-pipeline.md) § Recordings for the storage pipeline and [23-data-protection.md](23-data-protection.md) § Right to Erasure for deletion under KVKK / GDPR. +- Recordings are written to S3/SeaweedFS; retention defaults to **30 days** and is configurable per tenant up to a tenant-wide retention cap. See [16-media-pipeline.md](16-media-pipeline.md) § Recordings for the storage pipeline and [23-data-protection.md](23-data-protection.md) § Right to Erasure for deletion under KVKK / GDPR. - Recordings require consent: the classroom UI shows a "Recording" indicator while active, and the tenant onboarding agreement covers consent. ## MVP Scope for the Classroom @@ -215,7 +215,7 @@ A working self-hosted deployment needs: 2. **Redis** — for multi-node coordination (single-node deployments can skip but multi-node needs it). 3. **TURN server** — Coturn behind UDP/TCP ports, plus TLS for TURNS. Required for users behind restrictive NATs and corporate networks. 4. **Egress workers** — separate containers, started on demand by LiveKit when a recording is requested. -5. **Object storage** — S3 or MinIO with a `recordings/` bucket. +5. **Object storage** — S3 or SeaweedFS with a `recordings/` bucket. 6. **Monitoring** — LiveKit exports Prometheus metrics; Grafana dashboards are available upstream. 7. **Webhooks** — `https://api./webhooks/livekit` with HMAC signature verification. diff --git a/docs/architecture/08-livekit-cost-model.md b/docs/architecture/08-livekit-cost-model.md index 46de38e..7e7e5e0 100644 --- a/docs/architecture/08-livekit-cost-model.md +++ b/docs/architecture/08-livekit-cost-model.md @@ -242,7 +242,7 @@ This is the basis for the firm decision in [ADR 0005](../decisions/0005-live-cla - Each tenant can enable recording, with retention configurable up to a tenant-level cap. - Track classroom cost metrics (participant minutes, bandwidth, recording minutes, egress CPU) from the first production deployment. See [Phase 09](../roadmap/phase-09-billing-integrations-analytics.md). - Default retention: 30 days. Long-term retention requires explicit tenant action. -- Recording files live in S3/MinIO; metadata lives in PostgreSQL. Deletion is a two-step process. +- Recording files live in S3/SeaweedFS; metadata lives in PostgreSQL. Deletion is a two-step process. - Bandwidth provider choice for the SFU node is a real architectural decision. Default to a bandwidth-friendly provider; only run the SFU on AWS/GCP when there is a regulatory or proximity reason. ## Recommendation diff --git a/docs/architecture/09-tenant-isolation.md b/docs/architecture/09-tenant-isolation.md index dc987db..fe925f2 100644 --- a/docs/architecture/09-tenant-isolation.md +++ b/docs/architecture/09-tenant-isolation.md @@ -30,7 +30,7 @@ two scopes (tenant + organization): | PostgreSQL | RLS policy `tenant_id = current_setting('app.tenant_id', true)::uuid` | RLS policy `organization_id IS NULL OR organization_id = current_setting('app.organization_id', true)::uuid` | | Identity | Single-realm `learnstack` with `tenant_id` JWT claim (default per [ADR-0004](../decisions/0004-authentication-strategy.md); realm-per-tenant is an opt-in for enterprise isolation only) | `organization_id` JWT claim populated from active org membership | | Cache | Cache key auto-prefixed `{tenant_id}:{key}` | `{tenant_id}:{org_id}:{key}` when org context set | -| Files (MinIO) | Object key prefix `tenants/{tenant_id}/...` | `tenants/{tenant_id}/organizations/{org_id}/...` for org-scoped assets | +| Files (SeaweedFS) | Object key prefix `tenants/{tenant_id}/...` | `tenants/{tenant_id}/organizations/{org_id}/...` for org-scoped assets | | Search (Meilisearch) | `tenant_id` as mandatory filter | `organization_id = X OR organization_id IS NULL` clause when org context | | Jobs (Hangfire) | `JobParams.TenantId` mandatory | `JobParams.OrganizationId` nullable | | Audit (ADR-0016) | `audit_log.tenant_id` mandatory | `audit_log.organization_id` nullable | @@ -187,13 +187,13 @@ public abstract class PlatformJob : LearnStackJob | `Hangfire_JobPayloads_IncludeTenantId` | Reflection: every `LearnStackJob` subclass's `TParams` has `TenantId`. | | `LearnStackJob_RunAsync_SetsTenantBeforeExecute` | Source-grep + reflection: `RunAsync` is non-virtual; `SetTenant(...)` precedes `ExecuteAsync(...)`. | | `No_DirectDaprClient_OutsideInfrastructure` | Roslyn source scan: `Dapr.Client.*` only in `LearnStack.Infrastructure.{Caching, Messaging, Secrets}`. | -| `Provider_SDK_Types_NotImported_InDomain` | Provider SDK types (Stripe, Iyzico, LiveKit, Keycloak admin, MinIO) only in `LearnStack.Infrastructure.*` adapters. | +| `Provider_SDK_Types_NotImported_InDomain` | Provider SDK types (Stripe, Iyzico, LiveKit, Keycloak admin, SeaweedFS) only in `LearnStack.Infrastructure.*` adapters. | ## Storage, cache, search, audit, logs Tenant + org isolation applies outside PostgreSQL too: -### Storage (MinIO) +### Storage (SeaweedFS) ``` tenants/{tenant_id}/organizations/{org_id}/courses/{course_id}/... ← org-scoped diff --git a/docs/architecture/16-media-pipeline.md b/docs/architecture/16-media-pipeline.md index 6be523a..deab72b 100644 --- a/docs/architecture/16-media-pipeline.md +++ b/docs/architecture/16-media-pipeline.md @@ -18,14 +18,14 @@ Both share the same object storage and tenant-scoped key layout described below. ## Object Storage -- **Local development**: MinIO running in Docker Compose, S3-compatible API. -- **Production**: S3-compatible storage. Specific provider chosen per deployment (AWS S3, Backblaze B2, Wasabi, Cloudflare R2, MinIO on owned infrastructure). The provider choice is operational, not architectural — the application sees the S3 API. +- **Local development**: SeaweedFS running in Docker Compose, S3-compatible API. +- **Production**: S3-compatible storage. Specific provider chosen per deployment (AWS S3, Backblaze B2, Wasabi, Cloudflare R2, SeaweedFS on owned infrastructure). The provider choice is operational, not architectural — the application sees the S3 API. Buckets are per-environment, not per-tenant. Tenant isolation in storage is enforced by the key prefix and signed-URL scoping; bucket-per-tenant explodes operational complexity and is not used. ## Key Layout -Per [09-tenant-isolation.md § Storage (MinIO)](09-tenant-isolation.md) the canonical +Per [09-tenant-isolation.md § Storage (SeaweedFS)](09-tenant-isolation.md) the canonical key prefix is `tenants/{tenant_id}/...` (with `organizations/{org_id}/...` segment for org-scoped assets). The full key layout used by the media module: @@ -59,7 +59,7 @@ Direct-to-S3 with a server-issued, scoped, time-limited PUT URL. The .NET API ne sequenceDiagram participant UI as Studio participant API as .NET API - participant S3 as S3 / MinIO + participant S3 as S3 / SeaweedFS participant W as Worker UI->>API: POST /v1/media/upload-intent (filename, mime, size) @@ -152,7 +152,7 @@ CDN-cached URLs (the public mode) are versioned by content hash so cache invalid A CDN sits in front of the storage backend for public and tenant-scoped reads. The CDN: - Honours `Cache-Control` set by the API. -- Honours signed URLs (CloudFront-style or signed-cookie equivalent for private content; for fully self-hosted MinIO the application proxies private reads via a short-lived URL with no CDN caching). +- Honours signed URLs (CloudFront-style or signed-cookie equivalent for private content; for fully self-hosted SeaweedFS the application proxies private reads via a short-lived URL with no CDN caching). - Provides per-tenant access metrics (bytes served, request count) that feed into the analytics pipeline. ## Recordings (Live Classroom) @@ -188,7 +188,7 @@ sequenceDiagram participant Portal participant API participant Provider as Live Provider - participant Storage as MinIO/S3 + participant Storage as SeaweedFS/S3 Learner->>Portal: Join recorded session Portal->>API: Request join token @@ -267,4 +267,4 @@ A managed adapter takes over transcoding, manifest generation, and CDN; the Lear - **Re-encoding profile changes** invalidate previous variants. Source retention buys re-encode capability; the schedule for retention extension is a per-tenant configuration. - **MIME-type lying** — clients can claim a different MIME than the file body. Always re-detect after upload using a magic-byte check before processing. - **Public-bucket misconfiguration** — buckets are private by default; "public" assets are served by ACL on the object, not by making the bucket public. A `public-read` bucket is an operational red flag. -- **Storage egress in self-hosted MinIO** — when MinIO runs on owned infrastructure, the bottleneck shifts to the colo's bandwidth. Plan capacity accordingly. +- **Storage egress in self-hosted SeaweedFS** — when SeaweedFS runs on owned infrastructure, the bottleneck shifts to the colo's bandwidth. Plan capacity accordingly. diff --git a/docs/architecture/18-webrtc-build-vs-adopt.md b/docs/architecture/18-webrtc-build-vs-adopt.md index b432752..75e4021 100644 --- a/docs/architecture/18-webrtc-build-vs-adopt.md +++ b/docs/architecture/18-webrtc-build-vs-adopt.md @@ -39,7 +39,7 @@ flowchart TD Learner --> SFU["SFU Media Server"] Instructor --> SFU SFU --> Recording["Recording / Egress"] - Recording --> Storage["MinIO / S3"] + Recording --> Storage["SeaweedFS / S3"] Backend["LearnStack Backend"] --> Signaling Backend --> Storage ``` diff --git a/docs/architecture/23-data-protection.md b/docs/architecture/23-data-protection.md index d46a729..3ab83f1 100644 --- a/docs/architecture/23-data-protection.md +++ b/docs/architecture/23-data-protection.md @@ -129,7 +129,7 @@ Within a regional instance, the following components must run in-region: - PostgreSQL primary + WAL archive. - Redis (entitlement cache, L1 invalidation). -- MinIO / S3-compatible object storage (recordings, media). +- SeaweedFS / S3-compatible object storage (recordings, media). - Meilisearch. - Kafka. - LiveKit SFU + Egress (a learner joining from another continent still has their @@ -167,7 +167,7 @@ These are tracked as Phase-11+ work and are not implementation blockers for the For tenants subject to KVKK / GDPR, LearnStack acts as a **data processor** while the tenant is the **data controller**. The processor agreement template lives outside this repository (legal). Engineering-side commitments: -- Sub-processor list maintained in the tenant onboarding pack (Keycloak host, LiveKit host, S3 / MinIO provider, email provider, SMS provider, payment provider). +- Sub-processor list maintained in the tenant onboarding pack (Keycloak host, LiveKit host, S3 / SeaweedFS provider, email provider, SMS provider, payment provider). - Sub-processor changes are notified to tenants 30 days in advance via the Tenancy module's notification channel. - Security incident notification timeline: within 72 hours of discovery (GDPR Article 33 baseline). @@ -175,7 +175,7 @@ For tenants subject to KVKK / GDPR, LearnStack acts as a **data processor** whil - **PII redaction in logs.** Configured at the logging pipeline; the `[PiiSensitive]` attribute on a property excludes it from log emission ([10-observability.md](../standards/10-observability.md)). - **PII redaction in error reports.** Sentry receives redacted payloads; integration tests assert the redaction layer cannot be bypassed. -- **Encryption at rest.** Storage provider (S3 / MinIO) + PostgreSQL volumes use the provider's at-rest encryption; specific configuration documented in [12-infrastructure.md](../standards/12-infrastructure.md). +- **Encryption at rest.** Storage provider (S3 / SeaweedFS) + PostgreSQL volumes use the provider's at-rest encryption; specific configuration documented in [12-infrastructure.md](../standards/12-infrastructure.md). - **TLS in transit.** [11-security.md](../standards/11-security.md) § Transport. - **Backups.** Per-tenant deletion must propagate to backups within the platform's RPO + 1 backup cycle; the `delete-from-backups` job runs nightly. - **Subject-access timeline.** GDPR Article 12 requires response within 30 days. Engineering target: export bundle ready within 7 days of request; deletion completed within 30 days. diff --git a/docs/architecture/25-deployment-models.md b/docs/architecture/25-deployment-models.md index c051b73..4db66f3 100644 --- a/docs/architecture/25-deployment-models.md +++ b/docs/architecture/25-deployment-models.md @@ -66,7 +66,7 @@ Internet │ ├──► PostgreSQL (managed, RLS policies enforced) ├──► Redis (managed, multi-tenant via key prefix) - ├──► MinIO (multi-tenant via key prefix) + ├──► SeaweedFS (multi-tenant via key prefix) ├──► Meilisearch (per-tenant filter + per-locale index) ├──► Keycloak (single realm with multi-tenant claims) ├──► LiveKit (shared SFU pool) @@ -95,7 +95,7 @@ Internet │ ├──► Dedicated PostgreSQL instance (no shared rows; still RLS-protected for org scope) ├──► Dedicated Redis instance - ├──► Dedicated MinIO instance + ├──► Dedicated SeaweedFS instance ├──► Dedicated Meilisearch ├──► Dedicated Keycloak realm (could be in shared Keycloak cluster with realm-per-tenant) ├──► Shared LiveKit pool (with per-tenant resource caps) OR dedicated SFU @@ -118,7 +118,7 @@ Internet │ ├──► PostgreSQL (customer-managed) ├──► Redis (customer-managed) - ├──► MinIO or S3-compatible (customer-managed) + ├──► SeaweedFS or S3-compatible (customer-managed) ├──► Meilisearch (customer-managed) ├──► Keycloak (customer-managed; LearnStack ships pre-configured realm export) ├──► LiveKit (customer-managed OR fall back to LiveKit Cloud) diff --git a/docs/architecture/29-dapr-integration.md b/docs/architecture/29-dapr-integration.md index 48673fa..e96fc78 100644 --- a/docs/architecture/29-dapr-integration.md +++ b/docs/architecture/29-dapr-integration.md @@ -137,7 +137,7 @@ Secret path schema: secret/learnstack/postgres connection-string, ssl-cert secret/learnstack/redis password secret/learnstack/keycloak base-url, admin-username, admin-password -secret/learnstack/minio endpoint, access-key, secret-key +secret/learnstack/seaweedfs endpoint, access-key, secret-key secret/learnstack/meilisearch master-key, public-key secret/learnstack/livekit api-key, api-secret, ws-url secret/learnstack/coturn shared-secret diff --git a/docs/architecture/32-tenant-customization-model.md b/docs/architecture/32-tenant-customization-model.md index bbc5fe9..099e7d4 100644 --- a/docs/architecture/32-tenant-customization-model.md +++ b/docs/architecture/32-tenant-customization-model.md @@ -367,7 +367,7 @@ LearnStack core owns: - **Aggregate lifecycle** — `Course → CourseVersion → Module → Lesson → LessonItem` hierarchy. Tenant can't redefine that an enrollment binds to a `CourseVersion`. - **Auth flow** — Keycloak realm structure, JWT shape, MFA enforcement. -- **Storage layout** — MinIO bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...`. +- **Storage layout** — SeaweedFS bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...`. - **Outbox / event-bus contract** — integration event shapes are LearnStack-defined. - **Primitive set** — adding `whiteboard` or `3d-model` primitive is a LearnStack release. - **Audit pipeline** — modules cannot opt out of audit; per-(module, operation) toggling diff --git a/docs/decisions/0003-tenant-isolation-defense-in-depth.md b/docs/decisions/0003-tenant-isolation-defense-in-depth.md index b115a6a..979b991 100644 --- a/docs/decisions/0003-tenant-isolation-defense-in-depth.md +++ b/docs/decisions/0003-tenant-isolation-defense-in-depth.md @@ -43,7 +43,7 @@ table without altering the tenant-level guarantees. | **Organization filter (new)** | `organization_id` column on org-scoped entities + EF query filter + RLS policy + architecture test | | Identity | Keycloak realm-per-tenant + `organization_id` JWT claim populated from active org | | Cache | Cache keys auto-prefixed `{tenant_id}:{organization_id}:{key}` when org context set; `{tenant_id}:{key}` otherwise | -| Files (MinIO) | Object key prefix `tenants/{tenant_id}/organizations/{org_id}/...` when org-scoped; `tenants/{tenant_id}/...` when tenant-wide | +| Files (SeaweedFS) | Object key prefix `tenants/{tenant_id}/organizations/{org_id}/...` when org-scoped; `tenants/{tenant_id}/...` when tenant-wide | | Search (Meilisearch) | Query filter `tenant_id = X AND (organization_id = Y OR organization_id IS NULL)` for org-context queries | | Jobs (Hangfire) | Job payload carries `TenantId` (mandatory) + `OrganizationId?` (when applicable) | | Audit | Every audit row carries `tenant_id` (mandatory) + `organization_id?` (when applicable) — ADR-0016 | diff --git a/docs/decisions/0014-adopt-dapr.md b/docs/decisions/0014-adopt-dapr.md index 0773818..275355d 100644 --- a/docs/decisions/0014-adopt-dapr.md +++ b/docs/decisions/0014-adopt-dapr.md @@ -33,7 +33,7 @@ LearnStack is a modular monolith multi-tenant PaaS for education. It needs: - Cross-module integration events with at-least-once delivery, durable replay, multi-tenant fan-out. The outbox pattern (ADR-0010, ADR-0006) requires a dispatch target. - Distributed cache (L2) on top of in-process memory cache (L1) for tenant-scoped reads. -- Secret access for Keycloak admin credentials, MinIO access keys, LiveKit API secrets, +- Secret access for Keycloak admin credentials, SeaweedFS access keys, LiveKit API secrets, Stripe/Iyzico API keys, exchange rate API keys, etc. The platform plans triple deployment (SaaS / Dedicated / Self-Hosted; ADR-0020). All three diff --git a/docs/decisions/0017-tenant-organization-hierarchy.md b/docs/decisions/0017-tenant-organization-hierarchy.md index cdf68a0..261ad0a 100644 --- a/docs/decisions/0017-tenant-organization-hierarchy.md +++ b/docs/decisions/0017-tenant-organization-hierarchy.md @@ -177,7 +177,7 @@ public OrganizationId? OrganizationId { get; private set; } // null = tenant-w | **Organization filter** | `organization_id` column + EF query filter + RLS policy + arch test | | Identity | Keycloak realm-per-tenant; `organization_id` JWT claim populated from active org | | Cache | Cache keys auto-prefixed `{tenant_id}:{organization_id}:{key}` when org context set | -| Files (MinIO) | Bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...` | +| Files (SeaweedFS) | Bucket prefix `tenants/{tenant_id}/organizations/{org_id}/...` | | Search (Meilisearch) | Query filter `tenant_id = X AND (org_id = Y OR org_id = null)` (tenant-wide content visible to all orgs) | | Jobs (Hangfire) | Job payload carries both `TenantId` and `OrganizationId?` | | Audit | Every audit row carries `tenant_id` (mandatory) + `organization_id?` | diff --git a/docs/decisions/0018-tenant-driven-customization-model.md b/docs/decisions/0018-tenant-driven-customization-model.md index 9f78b97..da74290 100644 --- a/docs/decisions/0018-tenant-driven-customization-model.md +++ b/docs/decisions/0018-tenant-driven-customization-model.md @@ -387,7 +387,7 @@ primitive based on its JSON Schema type and format, composes the result. aggregate's lifecycle. - **Authentication and authorization plumbing** — Keycloak realm structure, JWT emission, permission policy evaluation. -- **Storage layout** — MinIO bucket prefixes, object key conventions. +- **Storage layout** — SeaweedFS bucket prefixes, object key conventions. - **Outbox / event-bus contract** — integration event shapes are LearnStack-defined. - **Generic primitive set** — adding a new primitive (e.g. `whiteboard`) is a LearnStack release, not a tenant action. Tenants compose existing primitives. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 2c6d668..500e49f 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -16,7 +16,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | # | Title | Topic | |---|---|---| | 0001 | [Platform Name](0001-platform-name.md) | Name and naming conventions | -| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Redis + MinIO + Next.js; modular monolith | +| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Redis + SeaweedFS + Next.js; modular monolith | | 0003 | [Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) | Query filters + RLS + audit + architecture tests (Amendment 1: Organization scope, 2026-05-18) | | 0004 | [Authentication Strategy](0004-authentication-strategy.md) | Off-the-shelf identity provider preferred over hand-rolled auth (Amendment 1: `learnstack-hub` realm, 2026-05-18) | | 0005 | [Live Classroom Media Stack](0005-live-classroom-media-stack.md) | LiveKit OSS self-hosted by default; LiveKit Cloud optional; no custom SFU | @@ -37,7 +37,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0020 | [Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) | SaaS / Dedicated / Self-Hosted from one codebase; phone-home + RSA-signed key + 30-day grace | | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | -| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | +| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's SeaweedFS row | ## Superseded ADRs diff --git a/docs/glossary.md b/docs/glossary.md index e47ae19..d60f402 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -75,7 +75,7 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | **Live Attendance** | A computed or recorded record of who joined a Live Session, for how long, and in what role. | | **Live Session Material** | A file, link, or content entry attached to a Live Session and visible inside the classroom. | | **Live Session Event** | An append-only event emitted during a Live Session (join, leave, screen-share start, recording start, etc.). | -| **Live Recording** | Metadata for a recording produced by the provider's egress pipeline. The file lives in MinIO / S3; LearnStack stores the metadata and consent state. | +| **Live Recording** | Metadata for a recording produced by the provider's egress pipeline. The file lives in SeaweedFS / S3; LearnStack stores the metadata and consent state. | > **Cohort vs. Classroom vs. Live Session.** Cohort is a *group of people*. Live Session is a *scheduled event*. Live Room is the *runtime artifact* of a Live Session. Earlier drafts used `Classroom` for both group and runtime; the term `Classroom` is deprecated in favor of the explicit `Cohort` / `Live Session` / `Live Room` split. @@ -194,7 +194,7 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | **Right to Erasure** | The user's right to have their personal data deleted, subject to retention exceptions (legal hold, financial records). | | **Anonymisation** | Replacement of PII fields with pseudonymous values; row stays for analytics / audit integrity. Distinct from soft delete and hard delete. | | **Consent Record** | An append-only per-purpose record (terms of service, recording, marketing). "Changing one's mind" creates a new record, never edits an old one. | -| **Sub-processor** | A third-party service LearnStack uses to process tenant data (Keycloak, LiveKit, S3 / MinIO, email provider, ...). Changes require 30-day tenant notice. | +| **Sub-processor** | A third-party service LearnStack uses to process tenant data (Keycloak, LiveKit, S3 / SeaweedFS, email provider, ...). Changes require 30-day tenant notice. | ## Audit diff --git a/docs/roadmap/phase-00-product-architecture.md b/docs/roadmap/phase-00-product-architecture.md index af72de9..2bf46bd 100644 --- a/docs/roadmap/phase-00-product-architecture.md +++ b/docs/roadmap/phase-00-product-architecture.md @@ -70,7 +70,7 @@ later should be made explicit before implementation begins. - .NET 10 backend. - EF Core and PostgreSQL. -- Redis and MinIO. +- Redis and SeaweedFS. - Next.js frontend. - Modular monolith. - Shared-database multi-tenancy for the initial implementation. diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 7f9a40e..25e139f 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -21,7 +21,7 @@ > `pnpm-lock.yaml` committed; `postinstall` hook stubs `.next/types/routes.d.ts`. > > **Packet 3 — Core dev compose ✅** -> `infra/compose/dev.yml` with PostgreSQL 16, Redis 7, MinIO + console, Mailpit +> `infra/compose/dev.yml` with PostgreSQL 16, Redis 7, SeaweedFS + console, Mailpit > (binary `readyz` healthcheck), Meilisearch — pinned tags, healthchecks, > named volumes, dev-only credential banners. > @@ -161,7 +161,7 @@ Docker Compose under `infra/compose/`: - PostgreSQL 16. - Redis 7. -- MinIO + MinIO console. +- SeaweedFS + SeaweedFS console. - Mailpit (outbound email). - Meilisearch. - LiveKit OSS + Coturn (for in-app classroom development). @@ -204,7 +204,7 @@ Two compose files: - Working backend solution scaffolded with modular layout. - Working frontend workspace with the single Next.js app. -- Local Docker Compose infrastructure with PostgreSQL, Redis, MinIO, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. +- Local Docker Compose infrastructure with PostgreSQL, Redis, SeaweedFS, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. - Initial CI pipeline. - Local development documentation. - `make seed` populating two demo tenants + one platform admin user. @@ -213,7 +213,7 @@ Two compose files: - A new developer can clone the repository and start the local environment by following one document. - Backend API responds on `GET /healthz`. -- PostgreSQL, Redis, MinIO, LiveKit, Coturn, Keycloak all run locally via compose. +- PostgreSQL, Redis, SeaweedFS, LiveKit, Coturn, Keycloak all run locally via compose. - Frontend builds and serves the three route segments. - CI passes on `main`. - The architecture-test project is set up and green even before domain features exist. diff --git a/docs/roadmap/phase-04-cms-media-pages.md b/docs/roadmap/phase-04-cms-media-pages.md index f9456c1..3f7a85b 100644 --- a/docs/roadmap/phase-04-cms-media-pages.md +++ b/docs/roadmap/phase-04-cms-media-pages.md @@ -94,7 +94,7 @@ The block system ships with: ### Media Library -- MinIO upload. +- SeaweedFS upload. - Asset metadata. - Folder and tag organization. - Image dimensions. diff --git a/docs/roadmap/phase-08c-classroom.md b/docs/roadmap/phase-08c-classroom.md index 493d45c..3afe532 100644 --- a/docs/roadmap/phase-08c-classroom.md +++ b/docs/roadmap/phase-08c-classroom.md @@ -62,7 +62,7 @@ Recording is **opt-in**, off by default per tenant. The consent flow is part of - Per-participant consent state captured before the room is joined; absence blocks the join. - Recording metadata persisted: storage key, duration, consent state, retention deadline. - Recording can be configured at policy / metadata level even when execution is disabled — for tenants that want the audit trail without the bandwidth. -- LiveKit Egress writes the recording to S3 / MinIO; LearnStack does not transcode in this phase. +- LiveKit Egress writes the recording to S3 / SeaweedFS; LearnStack does not transcode in this phase. ### Provider Webhooks @@ -104,7 +104,7 @@ Failures at any layer produce a Problem Details response with a specific `code` - Cross-tenant join attempt (identity `{tenantA}:{userId}` against a room owned by tenant B) is rejected. - Attendance reflects participant join/leave streams correctly across reconnections. - Recording cannot start without consent; consent state is recorded in `LiveRecording`. -- Recording, when enabled, writes to S3 / MinIO with the correct key prefix. +- Recording, when enabled, writes to S3 / SeaweedFS with the correct key prefix. - LiveKit provider webhook handler is signature-verified, idempotent, and tenant-scoped. ## Risks diff --git a/docs/standards/00-principles.md b/docs/standards/00-principles.md index bc98e56..2099349 100644 --- a/docs/standards/00-principles.md +++ b/docs/standards/00-principles.md @@ -39,7 +39,7 @@ The modular monolith works only because modules pretend to be services. Cross-mo Anything that crosses the LearnStack boundary — payments, email, SMS, search, storage, identity, live-class media, the **Hub**, **entitlement source**, **host→tenant resolution**, **event bus**, **cache**, **secret store** — lives behind an interface. -The domain code knows nothing about Stripe, Postmark, MinIO, Keycloak, LiveKit, Dapr, +The domain code knows nothing about Stripe, Postmark, SeaweedFS, Keycloak, LiveKit, Dapr, Kafka, Redis, Vault, or the Hub. Provider-specific code lives in `Infrastructure.` packages. Swapping a provider is a composition-root edit, not a code change. @@ -66,7 +66,7 @@ A failing test should tell a reader what behavior we promised. Test names read l ## 9. Default to Boring -We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Redis, MinIO, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. +We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Redis, SeaweedFS, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. ## 10. Cost-Aware From Day One diff --git a/docs/standards/06-testing.md b/docs/standards/06-testing.md index dbf34ea..04f6328 100644 --- a/docs/standards/06-testing.md +++ b/docs/standards/06-testing.md @@ -11,7 +11,7 @@ Test pyramid, conventions, and what every change must cover. flowchart TB e2e[End-to-end / Playwright
handful of golden flows] contract[Contract & API tests
OpenAPI + provider fakes] - integration[Integration tests
Testcontainers Postgres / Redis / MinIO] + integration[Integration tests
Testcontainers Postgres / Redis / SeaweedFS] arch[Architecture tests
module boundaries + tenant invariants] unit[Unit tests
domain + application + UI logic] @@ -39,7 +39,7 @@ We invest most at **unit + integration**. Architecture tests are zero-flake. E2E ### Integration Tests -- Real Postgres + Redis + MinIO via Testcontainers. +- Real Postgres + Redis + SeaweedFS via Testcontainers. - One Postgres database per test class (or Respawn between tests). - Real module configuration; no mocked repositories. - Cover happy path, edge cases, and **every tenant-isolation invariant**. diff --git a/docs/standards/09-error-handling.md b/docs/standards/09-error-handling.md index fd624e2..62b1673 100644 --- a/docs/standards/09-error-handling.md +++ b/docs/standards/09-error-handling.md @@ -63,7 +63,7 @@ Standard error codes (machine-readable, stable): ``` LearnStackException (base) ├── DomainException (domain invariant broken from inside, programmer error) -├── InfrastructureException (DB, Redis, MinIO transient) +├── InfrastructureException (DB, Redis, SeaweedFS transient) ├── ProviderException (upstream provider error) │ ├── PaymentProviderException │ ├── LiveClassProviderException diff --git a/docs/standards/10-observability.md b/docs/standards/10-observability.md index ee16cd7..2e068c8 100644 --- a/docs/standards/10-observability.md +++ b/docs/standards/10-observability.md @@ -99,7 +99,7 @@ Auto-instrument: - Hangfire job invocations. - Outbox dispatcher batches. - Redis client calls. -- MinIO/S3 SDK calls. +- SeaweedFS/S3 SDK calls. - LiveKit provider calls. Manual spans: diff --git a/docs/standards/11-security.md b/docs/standards/11-security.md index dfdc7dd..eb3f0e3 100644 --- a/docs/standards/11-security.md +++ b/docs/standards/11-security.md @@ -201,7 +201,7 @@ for the full strategy. Standards-side: - Enforce per-content-type size limits (image: 10 MB, document: 50 MB, video: 5 GB). - Strip EXIF where appropriate. - Store in tenant-scoped object storage prefix. -- Never trust the original filename. Generate a server-side key under the canonical tenant prefix: `tenants/{tenantId}/{category}/{uuid}.{ext}` (with `organizations/{orgId}/` segment for org-scoped assets), per [09-tenant-isolation.md § Storage (MinIO)](../architecture/09-tenant-isolation.md) and [16-media-pipeline.md § Key Layout](../architecture/16-media-pipeline.md). +- Never trust the original filename. Generate a server-side key under the canonical tenant prefix: `tenants/{tenantId}/{category}/{uuid}.{ext}` (with `organizations/{orgId}/` segment for org-scoped assets), per [09-tenant-isolation.md § Storage (SeaweedFS)](../architecture/09-tenant-isolation.md) and [16-media-pipeline.md § Key Layout](../architecture/16-media-pipeline.md). - Virus scan hook (ClamAV or cloud equivalent) before files become accessible. - Signed URLs for private files; TTL ≤ 1 hour. diff --git a/docs/standards/12-infrastructure.md b/docs/standards/12-infrastructure.md index 0f7cfe4..2559f13 100644 --- a/docs/standards/12-infrastructure.md +++ b/docs/standards/12-infrastructure.md @@ -52,7 +52,7 @@ adapter table. ``` postgres redis -minio + minio-console +seaweedfs # single dev binary: master + volume + filer + S3 gateway meilisearch keycloak # two realms: learnstack + learnstack-hub livekit-server @@ -136,7 +136,7 @@ Rules: ## Object Storage Operations -- MinIO local; S3-compatible cloud storage in production. +- SeaweedFS local; S3-compatible cloud storage in production. - One bucket per environment; tenant isolation enforced by key prefix (`{tenant_id}/...`). Bucket-per-tenant is not used. See [Media Pipeline § Key Layout](../architecture/16-media-pipeline.md) and [Tenant Isolation](../architecture/09-tenant-isolation.md). - Lifecycle policies for recording retention. - Cross-region replication for production buckets (optional, behind ADR). @@ -194,7 +194,7 @@ See [10-observability.md](10-observability.md). | Kafka (per broker) | 2 vCPU | 4 GB | 3-broker cluster baseline | | Vault | 1 vCPU | 1 GB | HA mode in production (3 nodes) | | APISIX | 1 vCPU | 1 GB | autoscale 2–4 | -| MinIO | 2 vCPU | 4 GB | scaled by storage tier | +| SeaweedFS | 2 vCPU | 4 GB | scaled by storage tier | | LiveKit SFU | 2 vCPU | 4 GB | per 250 concurrent participants | | LiveKit Egress | 2 vCPU | 4 GB | per ~1.5 concurrent recordings | | coturn | 1 vCPU | 1 GB | bandwidth-bound | diff --git a/docs/standards/15-performance.md b/docs/standards/15-performance.md index 672e95b..f0c5678 100644 --- a/docs/standards/15-performance.md +++ b/docs/standards/15-performance.md @@ -64,7 +64,7 @@ Budgets are reviewed quarterly against measured production metrics. ### Memory - Avoid loading whole result sets when streaming would do. -- Stream large file uploads to MinIO/S3; never buffer the whole file in memory. +- Stream large file uploads to SeaweedFS/S3; never buffer the whole file in memory. - Avoid string concatenation in tight loops; use `StringBuilder` or pooled buffers. ## Frontend Rules diff --git a/docs/standards/20-infrastructure-stack.md b/docs/standards/20-infrastructure-stack.md index 694e04c..604d40e 100644 --- a/docs/standards/20-infrastructure-stack.md +++ b/docs/standards/20-infrastructure-stack.md @@ -238,7 +238,7 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) libraries for these three in application code. - Postgres is accessed directly (EF Core); Dapr's state-store sits on Redis, not Postgres. -- MinIO is accessed via the configured S3-compatible client (no Dapr binding). +- SeaweedFS is accessed via the configured S3-compatible client (no Dapr binding). ## Forbidden diff --git a/infra/compose/README.md b/infra/compose/README.md index ca64dbe..e2cb0ba 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -15,7 +15,7 @@ eventing → secrets → Dapr sidecar → gateway). Packets 1-6 shipped; packets |---------|-------|----------------|---------------------| | PostgreSQL 16 | `postgres:16.14-alpine` | `localhost:5432` | `learnstack` / `learnstack` | | Redis 7 | `redis:7.4-alpine` | `localhost:6379` | — | -| MinIO | `minio/minio:RELEASE.2025-01-20T14-49-07Z` | `localhost:9000` (S3), `localhost:9001` (console) | `learnstack` / `learnstack-dev-secret` | +| SeaweedFS | `chrislusf/seaweedfs:3.94` | `localhost:9000` (S3), `localhost:9001` (filer UI), `localhost:9333` (master) | S3 access `learnstack` / secret `learnstack-dev-secret` | | Mailpit | `axllent/mailpit:v1.29.7` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | | Meilisearch | `getmeili/meilisearch:v1.44.0` | `localhost:7700` | master key `learnstack-dev-master-key` | @@ -108,12 +108,18 @@ The shared credentials above are checked into the repo intentionally — they ar from Vault via `ISecretProvider` (Standards 12 § Secrets Management; Standards 20). Do not reuse these strings anywhere except local Docker. -### Tenant isolation in MinIO +### Tenant isolation in SeaweedFS S3 Tenant isolation in object storage is enforced by **key prefix** (`{tenant_id}/...`), never bucket-per-tenant — see Standards 12 § Object Storage Operations and [docs/architecture/16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). -A single bucket per environment is created at first use. +A single bucket per environment is created at first use by the +`IStorageProvider` adapter. The rule is backend-independent — it +applied to MinIO, it applies to SeaweedFS, it will apply to any future +S3-compatible swap-in. + +See [../seaweedfs/README.md](../seaweedfs/README.md) for the SeaweedFS- +specific dev access surface (filer UI, S3 identity config, re-seed). ## What this file deliberately does NOT bring up yet diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index 5322cab..cd2c0fa 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,7 +1,7 @@ # LearnStack — local dev infrastructure. # # Scope of THIS file (Phase 01 packets 1-6): the data-plane services -# (Postgres / Redis / MinIO / Mailpit / Meilisearch), the self-hosted Keycloak +# (Postgres / Redis / SeaweedFS / Mailpit / Meilisearch), the self-hosted Keycloak # identity provider (two realms), the live-media stack (LiveKit OSS + Coturn), # Kafka (KRaft) + kafka-ui, HashiCorp Vault (-dev mode), the Dapr sidecar + # placement (pub/sub + state + secrets building blocks per ADR-0014), and the @@ -10,8 +10,9 @@ # arrive in Phase-01 packets 7-8. # # Operational rules: Standards 12 § Local Infrastructure + Standards 20. -# Tenant isolation in MinIO is enforced by key prefix (`{tenant_id}/...`) — -# never bucket-per-tenant (Standards 12 § Object Storage Operations). +# Tenant isolation in SeaweedFS S3 is enforced by key prefix +# (`{tenant_id}/...`) — never bucket-per-tenant (Standards 12 § Object +# Storage Operations). name: learnstack-dev @@ -63,23 +64,39 @@ services: timeout: 3s retries: 10 - minio: - image: minio/minio:RELEASE.2025-01-20T14-49-07Z - container_name: learnstack-minio + # SeaweedFS replaces MinIO per ADR-0029. Single dev binary packs master + # + volume + filer + S3 gateway; production splits them per ADR-0029 + # § Implementation Notes. The S3 gateway is port-mapped to the same + # host port (9000) MinIO used so any local script / env var / test + # fixture that hard-coded the S3 endpoint keeps working. + seaweedfs: + image: chrislusf/seaweedfs:3.94 + container_name: learnstack-seaweedfs restart: unless-stopped - command: server /data --console-address ":9001" - environment: - # Dev-only. Production credentials come from Vault via ISecretProvider. - MINIO_ROOT_USER: learnstack - MINIO_ROOT_PASSWORD: learnstack-dev-secret - MINIO_BROWSER_REDIRECT_URL: http://localhost:9001 + command: + - server + - -dir=/data + - -master + - -volume + - -filer + - -s3 + - -s3.port=8333 + - -s3.config=/etc/s3-identities.json + - -metricsPort=9091 ports: - - "9000:9000" # S3 API - - "9001:9001" # console + - "9000:8333" # S3 API gateway (host:9000 matches the prior MinIO endpoint) + - "9001:8888" # Filer UI (replaces the MinIO console) + - "9333:9333" # Master HTTP API + - "8080:8080" # Volume HTTP API volumes: - - minio-data:/data + - seaweedfs-data:/data + # Dev-only S3 identity (matches prior MinIO credentials so callers + # don't rebuild env vars). Production loads identities from Vault. + - ../seaweedfs/s3-identities.json:/etc/s3-identities.json:ro healthcheck: - test: ["CMD", "curl", "-fsS", "http://localhost:9000/minio/health/live"] + # The master HTTP API exposes /cluster/healthz; alpine-based + # SeaweedFS image ships wget. + test: ["CMD", "wget", "-qO-", "http://localhost:9333/cluster/healthz"] interval: 10s timeout: 5s retries: 10 @@ -401,6 +418,6 @@ services: volumes: postgres-data: redis-data: - minio-data: + seaweedfs-data: meilisearch-data: kafka-data: diff --git a/infra/livekit/README.md b/infra/livekit/README.md index 17954b9..9bacf22 100644 --- a/infra/livekit/README.md +++ b/infra/livekit/README.md @@ -53,7 +53,7 @@ The full integration arrives in Phase 08c. Summary: key above; tokens scoped per `(tenant_id, session_id, user_id)` with a short TTL. - **Recording** consumes LiveKit Egress (separate service, lands in - Phase 08c) writing to MinIO via the existing storage provider abstraction. + Phase 08c) writing to SeaweedFS via the existing storage provider abstraction. Recording is **tenant-configurable** and **consent-aware** per ADR-0005 + [16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). - **Cost metrics** (participant minutes, bandwidth, recording minutes) diff --git a/infra/seaweedfs/README.md b/infra/seaweedfs/README.md new file mode 100644 index 0000000..d537014 --- /dev/null +++ b/infra/seaweedfs/README.md @@ -0,0 +1,61 @@ +# SeaweedFS (Dev) + +Self-hosted S3-compatible object storage per +[ADR-0029](../../docs/decisions/0029-object-storage-seaweedfs.md). The .NET +app talks to SeaweedFS through `IStorageProvider`; the SeaweedFS SDK is +never imported by any module — only by +`LearnStack.Infrastructure.Storage.SeaweedFS` (ships Phase 02a). The dev +container packs master + volume + filer + S3 gateway in one binary; the +production topology splits them per ADR-0029 § Implementation Notes. + +## Access + +| Endpoint | Address | Purpose | +|----------|---------|---------| +| S3 API gateway | `http://localhost:9000` | The endpoint `IStorageProvider` talks to (drop-in port-map for the previous MinIO S3 endpoint) | +| Filer / volume UI | `http://localhost:9001` | Bucket browser, replaces the MinIO console | +| Master HTTP API | `http://localhost:9333` | Cluster topology + health (`/cluster/healthz`) | +| Volume HTTP API | `http://localhost:8080` | Internal — read / write blob ops | + +## Dev credentials + +| Surface | Credential | +|---------|------------| +| S3 access key | `learnstack` | +| S3 secret key | `learnstack-dev-secret` | + +Both come from `infra/seaweedfs/s3-identities.json`, mounted read-only at +`/etc/s3.json`. Production loads identities from Vault via +`ISecretProvider` per Standards 12 § Secrets Management; the literals +above are dev-only. + +The credential pair intentionally matches the prior MinIO defaults so any +test fixture, env var, or local script that already hard-coded +`learnstack` / `learnstack-dev-secret` keeps working through the swap. + +## Tenant isolation pattern (unchanged) + +Tenant isolation in object storage is enforced by **key prefix** +(`{tenant_id}/...`), never bucket-per-tenant — see +[Standards 12 § Object Storage Operations](../../docs/standards/12-infrastructure.md) +and [docs/architecture/16-media-pipeline.md](../../docs/architecture/16-media-pipeline.md). +A single bucket per environment is created at first use by the +`IStorageProvider` adapter at startup. + +## Re-seed / wipe + +```bash +docker compose -f infra/compose/dev.yml down seaweedfs +docker volume rm learnstack-dev_seaweedfs-data +docker compose -f infra/compose/dev.yml up -d seaweedfs +``` + +## What does NOT live here + +- The `IStorageProvider` adapter implementation — + `LearnStack.Infrastructure.Storage.SeaweedFS` (Phase 02a). +- Production-mode topology (split master / volume / filer / S3 gateway + containers, erasure-coding policy, tiered storage) — Phase 11. +- Multi-region replication evaluation — Phase 11+. +- The MinIO-era image (`minio/minio:RELEASE.2025-01-20T...`) — removed + in this packet per ADR-0029. diff --git a/infra/seaweedfs/s3-identities.json b/infra/seaweedfs/s3-identities.json new file mode 100644 index 0000000..5ee9ea4 --- /dev/null +++ b/infra/seaweedfs/s3-identities.json @@ -0,0 +1,14 @@ +{ + "identities": [ + { + "name": "learnstack-dev", + "credentials": [ + { + "accessKey": "learnstack", + "secretKey": "learnstack-dev-secret" + } + ], + "actions": ["Admin", "Read", "Write", "List", "Tagging"] + } + ] +} From 3fd9054bea79f9e3f276092400f62d0dfee21fec Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 21:24:32 +0300 Subject: [PATCH 08/11] feat(infra): ADR-0030 Valkey + ADR-0031 Postgres 18 + dev-stack swap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two follow-up architectural decisions landed together because they share the trigger — do major-version + vendor calls while LearnStack is still pre-implementation, so the migration drag is zero. The matching dev compose swap + the doc sweep ride along in the same commit per the "single coherent change" approach the user asked for. ADRs added - ADR-0030: Redis-compatible store moves to Valkey (Linux Foundation, BSD-3-Clause). Drop-in on RESP protocol — Dapr `state.redis` component, `StackExchange.Redis` library, `ICacheService` over Dapr all continue to work; those names are protocol/library/Dapr-provider- type identifiers, not vendor brands. Trigger: Redis Inc.'s 2024-03 license shift removed the unambiguous BSD path; the triple-license (AGPLv3 / RSALv2 / SSPLv1) leaves SaaS + Self-Hosted ambiguity that Valkey resolves without code change. - ADR-0031: PostgreSQL major version pinned to 18.x across all deployment modes. Longest LTS runway (EOL 2030-11), native `gen_uuid_v7()` that the ADR-0023 draft can adopt without an extension, async I/O for sequential scans that benefits the partitioned `audit_log` operator queries. RLS policy syntax + connection-string + role provisioning unchanged from 16 / 17, so the tenant-isolation defense-in-depth pattern (ADR-0003) transfers verbatim. ADR-0002 — Amendment 2 (dated 2026-05-19) bundles both decisions without rewriting ADR-0002's Decision section. decisions/README — two new rows in the active table; the prior "ADR-0029 partially supersedes ADR-0002's SeaweedFS row" wording was a sed-sweep artefact from the earlier MinIO→SeaweedFS commit and is corrected here to "MinIO row" while we're in the index. Compose changes - `postgres:16.14-alpine` → `postgres:18.4-alpine`. - `redis` service → `valkey` service; image `redis:7.4-alpine` → `valkey/valkey:8.1-alpine`; healthcheck `redis-cli ping` → `valkey-cli ping`; volume `redis-data` → `valkey-data`; the `dapr-sidecar-api` `depends_on` entry switches to `valkey`. - Dapr statestore component (`infra/dapr/components/statestore-redis.yaml`): `redisHost: redis:6379` → `redisHost: valkey:6379`. File name keeps the `-redis` suffix because `state.redis` is the Dapr provider-type identifier (RESP-protocol adapter), not the vendor name. Long inline comment explains the distinction. - `.gitignore`: `redis-data/` → `valkey-data/`. Doc sweep — selective - Vendor / image / boring-choice mentions of "Redis" → "Valkey" (Standards 00 § 9, CLAUDE.md hard rules, README.md, glossary, roadmap, compose README, deployment-mode tables, cost model, architecture diagrams, observability standards). - "PostgreSQL 16" / "Postgres 16" → "PostgreSQL 18" / "Postgres 18" across docs, standards, and roadmap. - Library / protocol / Dapr-provider-type names PRESERVED: `StackExchange.Redis`, `IConnectionMultiplexer`, `state.redis` Dapr component, `Microsoft.Extensions.Caching.Redis`, `RedisCacheService` example name. These are not the vendor brand; they are the wire protocol or the .NET type. Restored explicitly after the bulk replace to avoid the rename trap. Preserved intentional context - ADR-0002 Amendment 1 + 2 reference MinIO / Redis as the prior choices being superseded. - ADR-0029 + ADR-0030 + ADR-0031 — the decision records. - ADR-0014 (Adopt Dapr) — `StackExchange.Redis` library reference + Phase 2 restore. - `infra/compose/dev.yml` inline comment — "Valkey replaces Redis per ADR-0030". - `infra/dapr/components/statestore-redis.yaml` — file name + type identifier + inline comment explaining the protocol-vs-vendor split. - `architecture/29-dapr-integration.md` — `StackExchange.Redis` + `RedisCacheService` example name preserved. Deliberately NOT done - `infra/dapr/components/statestore-redis.yaml` file rename. The `state.redis` Dapr provider-type convention means the suffix mirrors the wire protocol; renaming would obscure that signal. - A separate "deprecate Redis 7.4 in production" migration plan — pre-implementation means no production Redis exists yet, no data migration is needed; Phase 11 production deployment lands directly on Valkey. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - Compose-config expansion shows `valkey` service + `valkey-data` volume + `postgres:18.4-alpine`. - Dapr statestore component `redisHost: valkey:6379` verified. - `grep -ln 'PostgreSQL 16\|Postgres 16'` returns zero outside ADR-0002 + ADR-0031. - `grep -ln '\bRedis\b'` matches only the 6 intentional historical context sites (ADR-0002, ADR-0030, ADR-0014, ADR-0029, dev.yml comment, statestore-redis.yaml comment, dapr-integration library-reference text). - Markdown link sweep on every changed doc clean. - Live `docker compose up` smoke deferred (Docker daemon not running on this workstation); will ride along with Phase 07's `make dev`. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/README.md | 2 +- .claude/skills/add-feature-key/SKILL.md | 4 +- .claude/skills/add-integration-test/SKILL.md | 6 +- .claude/skills/local-dev-setup/SKILL.md | 8 +- .claude/skills/run-tests-locally/SKILL.md | 10 +- .gitignore | 2 +- CLAUDE.md | 2 +- README.md | 6 +- backend/Directory.Packages.props | 2 +- backend/README.md | 2 +- .../LearnStack.Tests.Integration.csproj | 2 +- docs/architecture/03-module-boundaries.md | 2 +- .../architecture/04-technical-architecture.md | 8 +- docs/architecture/05-mvp-scope.md | 4 +- docs/architecture/06-extension-model.md | 2 +- docs/architecture/07-in-app-live-classroom.md | 4 +- docs/architecture/08-livekit-cost-model.md | 10 +- docs/architecture/09-tenant-isolation.md | 2 +- docs/architecture/21-feature-flags.md | 10 +- docs/architecture/23-data-protection.md | 2 +- docs/architecture/24-learnstack-hub.md | 2 +- docs/architecture/25-deployment-models.md | 6 +- docs/architecture/29-dapr-integration.md | 10 +- docs/decisions/0002-initial-architecture.md | 53 ++++- .../0010-cross-module-communication.md | 2 +- docs/decisions/0014-adopt-dapr.md | 12 +- docs/decisions/0022-custom-domain-tls.md | 2 +- .../0029-object-storage-seaweedfs.md | 2 +- .../0030-redis-compatible-store-valkey.md | 180 +++++++++++++++++ .../0031-postgresql-major-version.md | 184 ++++++++++++++++++ docs/decisions/README.md | 8 +- docs/glossary.md | 4 +- docs/roadmap/phase-00-product-architecture.md | 2 +- docs/roadmap/phase-01-repository-tooling.md | 10 +- docs/roadmap/phase-02a-kernel-tenancy.md | 4 +- docs/standards/00-principles.md | 4 +- docs/standards/05-database.md | 2 +- docs/standards/06-testing.md | 4 +- docs/standards/09-error-handling.md | 2 +- docs/standards/10-observability.md | 2 +- docs/standards/12-infrastructure.md | 4 +- docs/standards/20-infrastructure-stack.md | 14 +- infra/compose/README.md | 4 +- infra/compose/dev.yml | 26 ++- infra/dapr/README.md | 4 +- infra/dapr/components/statestore-redis.yaml | 13 +- 46 files changed, 535 insertions(+), 115 deletions(-) create mode 100644 docs/decisions/0030-redis-compatible-store-valkey.md create mode 100644 docs/decisions/0031-postgresql-major-version.md diff --git a/.claude/skills/README.md b/.claude/skills/README.md index f13ceaa..038a17d 100644 --- a/.claude/skills/README.md +++ b/.claude/skills/README.md @@ -114,7 +114,7 @@ index plus the relevant [Architecture](../../docs/architecture/) doc. | Skill | When to use | |-------|-------------| -| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Redis, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, SeaweedFS, LiveKit, Meilisearch. | +| [local-dev-setup](local-dev-setup/SKILL.md) | Bringing up the local stack: Postgres, Valkey, Vault, Kafka, Dapr sidecar, APISIX, Keycloak, SeaweedFS, LiveKit, Meilisearch. | | [seed-tenant](seed-tenant/SKILL.md) | Provisioning a tenant (with its default organization, customization data, seed users) for local development. | ## Authoring a new skill diff --git a/.claude/skills/add-feature-key/SKILL.md b/.claude/skills/add-feature-key/SKILL.md index 5769f64..fea7916 100644 --- a/.claude/skills/add-feature-key/SKILL.md +++ b/.claude/skills/add-feature-key/SKILL.md @@ -265,8 +265,8 @@ public async Task LimitKey_Soft_SurfaceBanner_DoesNotBlock() { ... } - **Writing a plan-projected key to `tenant_feature_flags`.** Architecture test rejects. Plan keys belong to the entitlement projection only. - **Reading the key from raw SQL.** Forbidden; use `IFeatureFlags`. -- **Hot path without Redis cache.** Each `IsEnabledAsync` call could become - DB-bound. The Redis layer (60s TTL, eager-invalidated by Dapr event) is +- **Hot path without Valkey cache.** Each `IsEnabledAsync` call could become + DB-bound. The Valkey layer (60s TTL, eager-invalidated by Dapr event) is load-bearing. - **Killswitch without runbook.** The runbook is part of the deliverable. CI does not enforce its presence today; review must. diff --git a/.claude/skills/add-integration-test/SKILL.md b/.claude/skills/add-integration-test/SKILL.md index 7a0adc0..93177cd 100644 --- a/.claude/skills/add-integration-test/SKILL.md +++ b/.claude/skills/add-integration-test/SKILL.md @@ -3,7 +3,7 @@ name: add-integration-test description: > Write a Testcontainers-backed integration test in `backend/tests/LearnStack.Tests.Integration` that exercises a real Postgres + - Redis + (optionally) Dapr stack and asserts behaviour under tenant + organization + Valkey + (optionally) Dapr stack and asserts behaviour under tenant + organization context. USE FOR: cross-tenant / cross-org isolation tests (mandatory for every new `[TenantOwned]` / `[OrganizationScoped]` entity), outbox → consumer round trips, audit-pipeline assertions, RLS-effective-isolation tests. DO NOT USE FOR: @@ -45,7 +45,7 @@ architecture test) plus any other invariant the change touches. See |-------|----------|-------------| | Scenario | Yes | A short name + setup + act + assert. | | Seed | Yes | Minimum tenants / orgs / users / customization data the scenario needs. | -| Required containers | Yes | Postgres always; add Redis / Kafka / Meilisearch / LiveKit / SeaweedFS as needed. | +| Required containers | Yes | Postgres always; add Valkey / Kafka / Meilisearch / LiveKit / SeaweedFS as needed. | | Tenant context | Yes | Which tenant + org the act phase runs as. | ## Workflow @@ -54,7 +54,7 @@ architecture test) plus any other invariant the change touches. See The project ships `TestFixture` that: -- Spins Postgres + Redis (+ optional Kafka via Dapr) via Testcontainers. +- Spins Postgres + Valkey (+ optional Kafka via Dapr) via Testcontainers. - Applies all migrations (per module). - Seeds a baseline platform admin, two tenants, two orgs per tenant. - Exposes `fixture.AsTenant(tenantId, organizationId?)` to scope a block. diff --git a/.claude/skills/local-dev-setup/SKILL.md b/.claude/skills/local-dev-setup/SKILL.md index e72a554..eb5c6ed 100644 --- a/.claude/skills/local-dev-setup/SKILL.md +++ b/.claude/skills/local-dev-setup/SKILL.md @@ -1,7 +1,7 @@ --- name: local-dev-setup description: > - Bring up the LearnStack local stack — Postgres, Redis, Vault, Kafka, Dapr + Bring up the LearnStack local stack — Postgres, Valkey, Vault, Kafka, Dapr sidecar, APISIX, Keycloak (two realms), SeaweedFS, LiveKit OSS, Meilisearch — via `docker-compose` plus the project's `make dev` orchestrator. USE FOR: first-time workstation setup, restoring a broken local environment, switching between @@ -15,7 +15,7 @@ description: > ## Purpose Stand up a full LearnStack stack on a developer workstation so backend + frontend -can run against real Postgres / Redis / Kafka / Vault / Keycloak / SeaweedFS / +can run against real Postgres / Valkey / Kafka / Vault / Keycloak / SeaweedFS / LiveKit / Meilisearch / APISIX — the same components production uses ([12-infrastructure.md § Local Infrastructure](../../../docs/standards/12-infrastructure.md), [20-infrastructure-stack.md](../../../docs/standards/20-infrastructure-stack.md)). @@ -98,7 +98,7 @@ The components and their default ports: | Component | Port | Purpose | |-----------|------|---------| | Postgres | 5432 | Main DB. | -| Redis | 6379 | Cache + Dapr state. | +| Valkey | 6379 | Cache + Dapr state. | | Kafka | 9092 | Dapr pub/sub backend. | | Kafka UI | 9094 | Optional UI for topics. | | Vault (dev mode) | 8200 | Secrets backend; `root` token, **not** for production. | @@ -168,7 +168,7 @@ Edit `.env` to flip `DEPLOYMENT_MODE`: | Value | What happens | |-------|--------------| | `Development` (default) | `InProcessEventBus` + `InMemoryCacheService` + env vars for secrets. Dapr sidecar is still present but not exercised. | -| `SaaS` | `DaprEventBus` (Kafka) + `DaprCacheService` (Redis) + `DaprSecretProvider` (Vault) + `HubEntitlementProvider` pointing at the local Hub. Requires the `learnstack-hub` repo's `make dev` to be running. | +| `SaaS` | `DaprEventBus` (Kafka) + `DaprCacheService` (Valkey) + `DaprSecretProvider` (Vault) + `HubEntitlementProvider` pointing at the local Hub. Requires the `learnstack-hub` repo's `make dev` to be running. | | `Dedicated` | Same as `SaaS` for the composition; in practice the Hub is dedicated to one tenant. | | `SelfHosted` | `SignedLicenseKeyEntitlementProvider` reads `.lic` from `./secrets/license.lic`; no Hub interaction. | diff --git a/.claude/skills/run-tests-locally/SKILL.md b/.claude/skills/run-tests-locally/SKILL.md index e6a5abb..244aa2e 100644 --- a/.claude/skills/run-tests-locally/SKILL.md +++ b/.claude/skills/run-tests-locally/SKILL.md @@ -60,7 +60,7 @@ docker info >/dev/null && echo "docker OK" backend/tests/ LearnStack.Tests.Unit/ # No DB, no Docker. Pure unit tests. LearnStack.Tests.Architecture/ # Reflection + Roslyn + migration-scan rules. - LearnStack.Tests.Integration/ # Testcontainers (Postgres, Redis, optional Dapr). + LearnStack.Tests.Integration/ # Testcontainers (Postgres, Valkey, optional Dapr). LearnStack.Tests.EndToEnd/ # Real HTTP API + frontend smoke. frontend/apps/web/ # Vitest + axe-core + Playwright (E2E). @@ -96,14 +96,14 @@ Common failure messages and fixes: | `Every_TenantOwned_Table_HasRls_With_AppTenantId` | Migration missing `ENABLE ROW LEVEL SECURITY` + the policy. | | `Integration_Event_Handlers_Use_InboxGuard` | Handler skipped `IsAlreadyProcessedAsync`; see [add-integration-event](../add-integration-event/SKILL.md). | | `Dapr_PubSub_TopicNames_FollowConvention` | Topic isn't `learnstack.{module}.{aggregate}`. | -| `Modules_Do_Not_Inject_Redis_Directly` | Use `ICacheService` not `IConnectionMultiplexer`. | +| `Modules_Do_Not_Inject_Valkey_Directly` | Use `ICacheService` not `IConnectionMultiplexer`. | | `LearnStack_Modules_DoNotReference_Hub` | Hub URL or namespace referenced outside the dedicated adapter. | | `No_Source_Folder_Named_Verticals` | A `Verticals/` folder exists; ADR-0018 forbids. | | `Core_Modules_HaveNo_DomainSpecific_Names` | A `Cefr`, `English`, `Asana`, etc. name appears in core. | ### Step 5: Run integration tests -Testcontainers spin Postgres + Redis per class fixture; runtime depends on Docker +Testcontainers spin Postgres + Valkey per class fixture; runtime depends on Docker performance. ```bash @@ -123,7 +123,7 @@ Common failure shapes: | `TenantContextMissingException` | Test forgot `using fixture.AsTenant(...)`. | | Empty result where rows should exist | Wrong tenant context — RLS works as designed. | | `relation "" does not exist` | Migration didn't apply; check the module's `Persistence/Migrations`. | -| Docker container fails to start | Port collision (5432, 6379) — stop local Postgres / Redis. | +| Docker container fails to start | Port collision (5432, 6379) — stop local Postgres / Valkey. | | Test hangs | A handler awaiting Dapr in the dev fallback path; check `IEventBus` registration. | ### Step 6: Run frontend tests @@ -226,7 +226,7 @@ dotnet test --blame-hang --blame-hang-timeout 5min change — the lockfile must match. - **Skipped architecture test.** Forbidden. If a test is marked `[Skip]`, treat it as a bug. -- **Port collisions.** Local Postgres / Redis on default ports collides with +- **Port collisions.** Local Postgres / Valkey on default ports collides with Testcontainers. Stop them. - **`--no-build` after a source change.** Drop the flag — the test would run against stale binaries. diff --git a/.gitignore b/.gitignore index 2a4ef79..8b4eb5c 100644 --- a/.gitignore +++ b/.gitignore @@ -39,7 +39,7 @@ logs/ .data/ seaweedfs-data/ postgres-data/ -redis-data/ +valkey-data/ # Local-only analyses (not shared via git) docs/analysis/ diff --git a/CLAUDE.md b/CLAUDE.md index 7d7a88c..f68a10a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,7 +26,7 @@ scaffold under `backend/` (core + 7 modules × 4 projects + 4 test projects including the non-skippable `LearnStack.Tests.Architecture`), the `pnpm` frontend monorepo under `frontend/` (`apps/web` Next.js App Router + `packages/{config,ui,sdk}`), and the core local-dev compose -stack at `infra/compose/dev.yml` (Postgres / Redis / SeaweedFS / Mailpit / +stack at `infra/compose/dev.yml` (Postgres / Valkey / SeaweedFS / Mailpit / Meilisearch). The remaining Phase-01 packets — Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, `.env.example`, CI, `make seed` — land incrementally; see diff --git a/README.md b/README.md index 5a32761..6648c19 100644 --- a/README.md +++ b/README.md @@ -20,17 +20,17 @@ Dedicated control plane, plan editor, custom-domain admin, and license-key issua Phase 01 in progress. The repository now holds the .NET 10 solution scaffold (7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` architecture test), the `pnpm` frontend monorepo (`apps/web` + `packages/{config,ui,sdk}`), -and the core local-dev `docker-compose` stack (Postgres / Redis / SeaweedFS / Mailpit / +and the core local-dev `docker-compose` stack (Postgres / Valkey / SeaweedFS / Mailpit / Meilisearch). Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, and CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). ## Direction At A Glance - **Backend:** .NET 10, ASP.NET Core, Entity Framework Core, MediatR. -- **Database:** PostgreSQL 16, with Row-Level Security from day one. Tenant + **Organization** +- **Database:** PostgreSQL 18, with Row-Level Security from day one. Tenant + **Organization** defense in depth ([ADR-0003 Amendment 1](docs/decisions/0003-tenant-isolation-defense-in-depth.md), [ADR-0017](docs/decisions/0017-tenant-organization-hierarchy.md)). -- **Cache / Pub-Sub / Secrets:** Redis 7, Kafka, HashiCorp Vault — all accessed via +- **Cache / Pub-Sub / Secrets:** Valkey 7, Kafka, HashiCorp Vault — all accessed via **Dapr** building blocks (`IEventBus`, `ICacheService`, `ISecretProvider`) per [ADR-0014](docs/decisions/0014-adopt-dapr.md). - **API Gateway:** **APISIX** in standalone YAML-reload mode per diff --git a/backend/Directory.Packages.props b/backend/Directory.Packages.props index ca29456..8461572 100644 --- a/backend/Directory.Packages.props +++ b/backend/Directory.Packages.props @@ -37,7 +37,7 @@ suite only runs the WebApplicationFactory smoke test. --> - + diff --git a/backend/README.md b/backend/README.md index 58e52dc..c6c28a3 100644 --- a/backend/README.md +++ b/backend/README.md @@ -37,7 +37,7 @@ backend/ tests/ LearnStack.Tests.Unit/ - LearnStack.Tests.Integration/ # Testcontainers (Postgres/Redis) + LearnStack.Tests.Integration/ # Testcontainers (Postgres/Valkey) LearnStack.Tests.Architecture/ # NetArchTest + filesystem invariants LearnStack.Tests.Contract/ ``` diff --git a/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj b/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj index fa5d581..9ba1939 100644 --- a/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj +++ b/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj @@ -25,7 +25,7 @@ - + diff --git a/docs/architecture/03-module-boundaries.md b/docs/architecture/03-module-boundaries.md index 0506530..4b05c37 100644 --- a/docs/architecture/03-module-boundaries.md +++ b/docs/architecture/03-module-boundaries.md @@ -261,7 +261,7 @@ backend/ LearnStack.Api/ # ASP.NET host (single deployment unit) LearnStack.Application/ # composition root, MediatR pipeline LearnStack.Domain/ # shared kernel domain pieces - LearnStack.Infrastructure/ # EF, Redis, SeaweedFS, OpenTelemetry, Dapr wiring + LearnStack.Infrastructure/ # EF, Valkey, SeaweedFS, OpenTelemetry, Dapr wiring LearnStack.Infrastructure.Audit/ # audit interceptor + state capture + MediatR behavior LearnStack.SharedKernel/ # ids, audit fields, errors, paging, IEventBus, # ICacheService, ISecretProvider, IEntitlementProvider diff --git a/docs/architecture/04-technical-architecture.md b/docs/architecture/04-technical-architecture.md index bc4b6c1..7eba49d 100644 --- a/docs/architecture/04-technical-architecture.md +++ b/docs/architecture/04-technical-architecture.md @@ -7,8 +7,8 @@ | Backend runtime | .NET 10, ASP.NET Core Web API | | Language | C# | | ORM | Entity Framework Core | -| Database | PostgreSQL 16+ (shared schema + RLS isolation; ADR-0003) | -| Cache & coordination | **Redis 7+ via Dapr State Store** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | +| Database | PostgreSQL 18+ (shared schema + RLS isolation; ADR-0003) | +| Cache & coordination | **Valkey 7+ via Dapr State Store** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | | Pub/Sub | **Apache Kafka via Dapr Pub/Sub** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) — outbox dispatch target | | Secrets | **HashiCorp Vault via Dapr Secret Store** (or env-var fallback in Dev) | | Distributed runtime | **Dapr 1.14+** sidecar pattern (pub/sub, state, secrets) | @@ -52,7 +52,7 @@ flowchart LR subgraph data["Data Plane"] pg[(PostgreSQL)] - redis[(Redis)] + redis[(Valkey)] seaweedfs[(SeaweedFS / S3)] meili[(Meilisearch)] kafka[(Kafka)] @@ -123,7 +123,7 @@ Module boundaries and dependency rules are in [Module Boundaries](03-module-boun | `Api` | HTTP endpoints, auth middleware, request binding, OpenAPI emission, tenant resolution middleware. | | `Application` | Use cases (MediatR commands/queries), validation, transactions, pipeline behaviors. | | `Domain` | Entities, aggregates, value objects, domain services, domain events. | -| `Infrastructure` | EF Core, Redis, SeaweedFS, Hangfire, OpenTelemetry, external adapters. | +| `Infrastructure` | EF Core, Valkey, SeaweedFS, Hangfire, OpenTelemetry, external adapters. | | `Modules.*` | Bounded feature areas, each with their own `Application` / `Domain` / `Infrastructure` internals and a public `Application.Contracts` surface. | ## Multi-Tenancy diff --git a/docs/architecture/05-mvp-scope.md b/docs/architecture/05-mvp-scope.md index 2779aa0..7757e2c 100644 --- a/docs/architecture/05-mvp-scope.md +++ b/docs/architecture/05-mvp-scope.md @@ -52,7 +52,7 @@ test pass. ### Platform Kernel - Dapr building blocks wired through `IEventBus` (pub/sub → Kafka), `ICacheService` - (state → Redis), `ISecretProvider` (secrets → Vault). See + (state → Valkey), `ISecretProvider` (secrets → Vault). See [29-dapr-integration.md](29-dapr-integration.md) and [ADR-0014](../decisions/0014-adopt-dapr.md). - APISIX gateway in standalone mode (YAML hot-reload) — JWT verification, CORS, @@ -75,7 +75,7 @@ test pass. per-organization override. - Map custom domains via Hub's `CustomDomain` aggregate; LearnStack mirrors the host→tenant mapping. See [27-custom-domain-tls.md](27-custom-domain-tls.md). -- Per-tenant feature flags (catalog-defined; runtime via Redis cache + Postgres). See +- Per-tenant feature flags (catalog-defined; runtime via Valkey cache + Postgres). See [21-feature-flags.md](21-feature-flags.md). - Per-tenant settings (locale, timezone, default notification sender). diff --git a/docs/architecture/06-extension-model.md b/docs/architecture/06-extension-model.md index 8e01ed9..f5583a0 100644 --- a/docs/architecture/06-extension-model.md +++ b/docs/architecture/06-extension-model.md @@ -35,7 +35,7 @@ The core platform talks to external systems through **interfaces** in | Recording egress | `IRecordingEgressProvider` | | Identity provider | (covered by Keycloak baseline; ADR-0004) | | Pub/Sub | `IEventBus` → Dapr → Kafka (ADR-0014) | -| Cache | `ICacheService` → Dapr → Redis (ADR-0014) | +| Cache | `ICacheService` → Dapr → Valkey (ADR-0014) | | Secret store | `ISecretProvider` → Dapr → Vault (ADR-0014) | Implementations live in `LearnStack.Infrastructure..` projects. diff --git a/docs/architecture/07-in-app-live-classroom.md b/docs/architecture/07-in-app-live-classroom.md index 96dbf28..26bbef6 100644 --- a/docs/architecture/07-in-app-live-classroom.md +++ b/docs/architecture/07-in-app-live-classroom.md @@ -64,7 +64,7 @@ flowchart LR end subgraph Infra - REDIS[(Redis)] + REDIS[(Valkey)] S3[(S3 / SeaweedFS)] TURN[Coturn TURN/STUN] end @@ -212,7 +212,7 @@ Deferred to later phases: A working self-hosted deployment needs: 1. **LiveKit server** — Docker container, behind a TLS reverse proxy on `wss://livekit.`. -2. **Redis** — for multi-node coordination (single-node deployments can skip but multi-node needs it). +2. **Valkey** — for multi-node coordination (single-node deployments can skip but multi-node needs it). 3. **TURN server** — Coturn behind UDP/TCP ports, plus TLS for TURNS. Required for users behind restrictive NATs and corporate networks. 4. **Egress workers** — separate containers, started on demand by LiveKit when a recording is requested. 5. **Object storage** — S3 or SeaweedFS with a `recordings/` bucket. diff --git a/docs/architecture/08-livekit-cost-model.md b/docs/architecture/08-livekit-cost-model.md index 7e7e5e0..c2a22d6 100644 --- a/docs/architecture/08-livekit-cost-model.md +++ b/docs/architecture/08-livekit-cost-model.md @@ -34,7 +34,7 @@ LearnStack owns: - Compute for the SFU and Egress workers. - Outbound bandwidth (often the largest single line item). - TURN/STUN infrastructure (Coturn). -- Redis for multi-node coordination. +- Valkey for multi-node coordination. - Object storage for recordings. - Monitoring, alerting, scaling, regional deployment, incident response. @@ -65,7 +65,7 @@ This would mean building a custom SFU on top of an open-source WebRTC library (M |---|---|---|---|---| | Base / minutes | $50 + $165 overage | $0 | $0 | $0 | | Bandwidth | $381 | ~$0 (in plan) | ~$310 | ~$0 (in plan) | -| Compute (SFU + Redis + TURN) | included | ~$150 | ~$200 | ~$150 | +| Compute (SFU + Valkey + TURN) | included | ~$150 | ~$200 | ~$150 | | Storage (recordings) | included up to 600 min | $5 | $5 | $5 | | Engineering (amortised) | ~$0 | ~$300/mo SRE slice | ~$300/mo SRE slice | **~$3,000–5,000/mo for 12 months, then ongoing** | | **Monthly steady-state** | **~$596** | **~$155** | **~$515** | **~$3,155–5,155** | @@ -136,7 +136,7 @@ A single 4-core SFU node comfortably handles ~250 concurrent participants for ty | Mode | Estimated monthly cost | |---|---| | LiveKit Cloud Ship | $50 (well within included quota) | -| LiveKit OSS Hetzner | ~$130 (1 small SFU + Redis + TURN) | +| LiveKit OSS Hetzner | ~$130 (1 small SFU + Valkey + TURN) | | LiveKit OSS AWS | ~$140 | At this scale, **LiveKit Cloud is cheaper** because the fixed infra cost of self- @@ -144,7 +144,7 @@ hosting is not yet amortised. > **Derivation (2026-05-14 rates).** Cloud: 12,000 minutes / 150,000 included = 8% of > quota, 84 GB / 250 GB = 34% — both inside Ship plan; $50 base only. -> Hetzner self-host: CX21 SFU (~$5) + CX11 Redis (~$3) + CX11 TURN (~$3) ≈ $11 +> Hetzner self-host: CX21 SFU (~$5) + CX11 Valkey (~$3) + CX11 TURN (~$3) ≈ $11 > infra; remaining ~$120 is the SRE-slice amortised cost (~10% of a senior engineer's > on-call rotation at this small workload). > AWS: same compute (~$30 t3.medium × 3) + ~$0 egress (84 GB inside 100 GB pooled @@ -222,7 +222,7 @@ flips the decision. | Mode | Estimated monthly cost | |---|---| | LiveKit Cloud Scale | $500 base + ~$120 transfer overage + $4,380 recording overage ≈ **$5,000** | -| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Redis + TURN + 1 TB recording storage ≈ **$650–800** | +| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Valkey + TURN + 1 TB recording storage ≈ **$650–800** | By this scale, the self-hosted advantage is ~6×. diff --git a/docs/architecture/09-tenant-isolation.md b/docs/architecture/09-tenant-isolation.md index fe925f2..6de34e7 100644 --- a/docs/architecture/09-tenant-isolation.md +++ b/docs/architecture/09-tenant-isolation.md @@ -200,7 +200,7 @@ tenants/{tenant_id}/organizations/{org_id}/courses/{course_id}/... ← org-sco tenants/{tenant_id}/brand/... ← tenant-wide ``` -### Cache (Dapr State Store / Redis) +### Cache (Dapr State Store / Valkey) ``` {tenant_id}:{org_id}:{module}:{entity}:{id} ← org context set diff --git a/docs/architecture/21-feature-flags.md b/docs/architecture/21-feature-flags.md index 13510f5..8392919 100644 --- a/docs/architecture/21-feature-flags.md +++ b/docs/architecture/21-feature-flags.md @@ -139,7 +139,7 @@ Rules: the inbound Dapr event from Hub. See [ADR-0021](../decisions/0021-feature-based-entitlement.md) and [29-dapr-integration.md](29-dapr-integration.md). -- A short-TTL Redis cache (60 s) fronts both tables for hot-path reads. Eager +- A short-TTL Valkey cache (60 s) fronts both tables for hot-path reads. Eager invalidation flows from `learnstack.cache.invalidation` (intra-instance) and from `learnstack.hub.entitlement` (cross-deployment). @@ -160,11 +160,11 @@ Resolution precedence for `IsEnabledAsync(FeatureKey key, ct)`: genuinely need to read cross-tenant go through a separate `IEntitlementAdminQuery` interface. 2. **If the key's catalog descriptor says `Source = PlanProjected`:** read from - `platform_entitlement_cache.features` (via Redis cache → Postgres). A missing entry + `platform_entitlement_cache.features` (via Valkey cache → Postgres). A missing entry resolves to the catalog default. Per-tenant `tenant_feature_flags` are **never** consulted for plan-projected keys. 3. **If the key's catalog descriptor says `Source = TenantFlag`:** read from - `tenant_feature_flags` (via Redis cache → Postgres). Missing entry → catalog + `tenant_feature_flags` (via Valkey cache → Postgres). Missing entry → catalog default. 4. **Killswitch overlay** (last word): if the corresponding killswitch is flipped `false` platform-wide, the answer becomes `false` regardless of the per-tenant @@ -264,13 +264,13 @@ Both surfaces are MUST-audit security-events (see hasn't refreshed sees the old feature set. Eager invalidation via the Dapr event keeps the typical refresh within seconds; the 15-min TTL is the upper bound. - **Performance.** Hot paths that read flags per call become DB-bound without the - Redis cache; the 60s TTL is the default trade-off. + Valkey cache; the 60s TTL is the default trade-off. ## Roadmap Touchpoints - **Phase 02a** — `tenant_feature_flags` table created in the Tenancy module; the `FeatureKeys` / `LimitKeys` / `KillswitchKeys` catalogs land here. `IFeatureFlags`, - the Redis cache, and the architecture tests ship here. + the Valkey cache, and the architecture tests ship here. - **Phase 02c** (parallel Hub Foundation) — `platform_entitlement_cache`, `IEntitlementProvider` with `NullEntitlementProvider` default + `HubEntitlementProvider` + `SignedLicenseKeyEntitlementProvider` diff --git a/docs/architecture/23-data-protection.md b/docs/architecture/23-data-protection.md index 3ab83f1..e9036d3 100644 --- a/docs/architecture/23-data-protection.md +++ b/docs/architecture/23-data-protection.md @@ -128,7 +128,7 @@ a single LearnStack deployment. In practice: Within a regional instance, the following components must run in-region: - PostgreSQL primary + WAL archive. -- Redis (entitlement cache, L1 invalidation). +- Valkey (entitlement cache, L1 invalidation). - SeaweedFS / S3-compatible object storage (recordings, media). - Meilisearch. - Kafka. diff --git a/docs/architecture/24-learnstack-hub.md b/docs/architecture/24-learnstack-hub.md index 0cd8bf5..16f90aa 100644 --- a/docs/architecture/24-learnstack-hub.md +++ b/docs/architecture/24-learnstack-hub.md @@ -523,7 +523,7 @@ learnstack-hub/ ``` Hub does NOT share Dapr instance with LearnStack — Hub has its own sidecar, its own -`pubsub` component, its own state store (Hub Redis instance), its own secret store +`pubsub` component, its own state store (Hub Valkey instance), its own secret store namespace (`secret/learnstack-hub/*`). **Kafka cluster topology:** Hub and LearnStack share the **same Kafka cluster** in diff --git a/docs/architecture/25-deployment-models.md b/docs/architecture/25-deployment-models.md index 4db66f3..7a9c7c8 100644 --- a/docs/architecture/25-deployment-models.md +++ b/docs/architecture/25-deployment-models.md @@ -65,7 +65,7 @@ Internet [ LearnStack.Host pods (auto-scaled 2-8) ] │ ├──► PostgreSQL (managed, RLS policies enforced) - ├──► Redis (managed, multi-tenant via key prefix) + ├──► Valkey (managed, multi-tenant via key prefix) ├──► SeaweedFS (multi-tenant via key prefix) ├──► Meilisearch (per-tenant filter + per-locale index) ├──► Keycloak (single realm with multi-tenant claims) @@ -94,7 +94,7 @@ Internet [ LearnStack-managed Kubernetes namespace per customer ] │ ├──► Dedicated PostgreSQL instance (no shared rows; still RLS-protected for org scope) - ├──► Dedicated Redis instance + ├──► Dedicated Valkey instance ├──► Dedicated SeaweedFS instance ├──► Dedicated Meilisearch ├──► Dedicated Keycloak realm (could be in shared Keycloak cluster with realm-per-tenant) @@ -117,7 +117,7 @@ Internet [ Customer-owned Kubernetes cluster, customer admin ] │ ├──► PostgreSQL (customer-managed) - ├──► Redis (customer-managed) + ├──► Valkey (customer-managed) ├──► SeaweedFS or S3-compatible (customer-managed) ├──► Meilisearch (customer-managed) ├──► Keycloak (customer-managed; LearnStack ships pre-configured realm export) diff --git a/docs/architecture/29-dapr-integration.md b/docs/architecture/29-dapr-integration.md index e96fc78..fe53a39 100644 --- a/docs/architecture/29-dapr-integration.md +++ b/docs/architecture/29-dapr-integration.md @@ -5,7 +5,7 @@ [ADR-0010](../decisions/0010-cross-module-communication.md). LearnStack uses **Dapr** (Distributed Application Runtime) for three building blocks: -**pub/sub** (Kafka), **state store** (Redis), **secret store** (Vault). Application code +**pub/sub** (Kafka), **state store** (Valkey), **secret store** (Vault). Application code interacts with Dapr only through SharedKernel abstractions (`IEventBus`, `ICacheService`, `ISecretProvider`) — never via `DaprClient` directly. @@ -22,12 +22,12 @@ flowchart LR subgraph Backends[Backends] Kafka[(Kafka)] - Redis[(Redis)] + Valkey[(Valkey)] Vault[(HashiCorp Vault)] end Daprd -- pubsub.kafka --> Kafka - Daprd -- state.redis --> Redis + Daprd -- state.redis --> Valkey Daprd -- secretstore.hashicorp.vault --> Vault subgraph Subscribers[Subscriber side] @@ -40,7 +40,7 @@ flowchart LR The sidecar shares the network namespace of the app pod (Docker compose `network_mode: "service:learnstack-api"`; Kubernetes via `dapr.io/enabled` annotation). -The app talks to the sidecar via `localhost`, never directly to Kafka / Redis / Vault. +The app talks to the sidecar via `localhost`, never directly to Kafka / Valkey / Vault. ## 2. Components @@ -80,7 +80,7 @@ Topics follow the convention `learnstack.{module}.{aggregate}`. Examples: - `learnstack.hub.entitlement` (Hub-side) - `learnstack.cache.invalidation` (cross-instance L1 cache invalidation) -### `statestore.yaml` — Redis state store +### `statestore.yaml` — Valkey state store ```yaml apiVersion: dapr.io/v1alpha1 diff --git a/docs/decisions/0002-initial-architecture.md b/docs/decisions/0002-initial-architecture.md index 8a26484..5fe29b2 100644 --- a/docs/decisions/0002-initial-architecture.md +++ b/docs/decisions/0002-initial-architecture.md @@ -2,14 +2,23 @@ ## Status -Accepted (Amendment 1: 2026-05-19 — storage backend changed from MinIO to -SeaweedFS per [ADR-0029](0029-object-storage-seaweedfs.md); see Amendment at -the bottom of this document. Every other choice in this ADR — .NET 10, -ASP.NET Core, EF Core, PostgreSQL, Redis, Next.js, modular monolith — stands.) +Accepted with two amendments — see the bottom of this document for the dated +amendment block. Each amendment supersedes a single backend row of the +original Decision section without rewriting the rest of the ADR: + +- **Amendment 1 (2026-05-19):** storage backend MinIO → SeaweedFS per + [ADR-0029](0029-object-storage-seaweedfs.md). +- **Amendment 2 (2026-05-19):** cache + state backend Redis → Valkey per + [ADR-0030](0030-redis-compatible-store-valkey.md); PostgreSQL major + version pinned to 18.x per + [ADR-0031](0031-postgresql-major-version.md). + +Every other choice in this ADR — .NET 10, ASP.NET Core, EF Core, modular +monolith, Next.js — stands. ## Decision -LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL, Redis, SeaweedFS (see Amendment 1), and Next.js. +LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL 18.x (see Amendment 2), Valkey (see Amendment 2), SeaweedFS (see Amendment 1), and Next.js. ## Context @@ -21,8 +30,8 @@ The team has stronger familiarity with .NET, so .NET 10 is preferred over Go for - The first backend implementation should use .NET 10. - EF Core should be the default ORM. -- PostgreSQL should be the primary database. -- Redis should be used for caching and distributed coordination where needed. +- PostgreSQL 18.x should be the primary database (Amendment 2). +- Valkey should be used for caching and distributed coordination where needed (Amendment 2). - SeaweedFS should be used locally for S3-compatible object storage (Amendment 1). - Next.js should be used for public rendering, admin studio, and portals initially. @@ -47,3 +56,33 @@ Every doc that previously said "MinIO" should be read as "SeaweedFS" for operational guidance; conceptual rules ("tenant key-prefix isolation", "swap to AWS S3 in SaaS through `IStorageProvider`") were never backend-specific and stand verbatim. + +## Amendment 2 — Cache: Redis → Valkey; PostgreSQL: pin major to 18.x (2026-05-19) + +Two backend-row clarifications recorded together because they share the +trigger (do major-version + vendor calls while LearnStack is still +pre-implementation, so the migration drag is zero): + +1. **Cache + state backend Redis → Valkey** per + [ADR-0030](0030-redis-compatible-store-valkey.md). Redis 7.4 was the + last BSD-3-Clause Redis release; the 8.x line ships under a + triple-license (AGPLv3 / RSALv2 / SSPLv1). Valkey is the + Linux-Foundation-governed BSD-3-Clause fork, drop-in compatible on + the RESP protocol. The Dapr `state.redis` component name does not + change — it is the RESP-provider identifier, not a vendor brand. +2. **PostgreSQL major pinned to 18.x** per + [ADR-0031](0031-postgresql-major-version.md). 18 is the longest- + runway LTS available (EOL 2030-11), brings native `gen_uuid_v7()` + that the [ADR-0023 draft](README.md) can adopt without an extension, + and async I/O for sequential scans helps the partitioned `audit_log` + ([ADR-0016](0016-audit-log-subsystem.md)) operator queries. RLS + policy syntax + connection-string + role provisioning are unchanged + from 16 / 17, so the tenant-isolation defense-in-depth pattern + ([ADR-0003](0003-tenant-isolation-defense-in-depth.md)) transfers + verbatim. + +Every doc that previously said "Redis 7" or "PostgreSQL 16" should be +read as "Valkey 8" or "PostgreSQL 18". Library names + protocol +identifiers stay (`StackExchange.Redis`, `IConnectionMultiplexer`, +`state.redis` Dapr component, RESP) — those are protocol/library +identifiers, not vendor brands. diff --git a/docs/decisions/0010-cross-module-communication.md b/docs/decisions/0010-cross-module-communication.md index 4598041..48ab0d1 100644 --- a/docs/decisions/0010-cross-module-communication.md +++ b/docs/decisions/0010-cross-module-communication.md @@ -65,7 +65,7 @@ The following tests live in `LearnStack.Tests.Architecture` and run on every PR: ## Amendment 1 — Dapr pub/sub as outbox dispatch target (2026-05-18) Per [ADR-0014](0014-adopt-dapr.md), LearnStack adopts Dapr for cross-cutting infrastructure -(pub/sub Kafka, state Redis, secrets Vault). This amendment specifies how Mechanism #3 +(pub/sub Kafka, state Valkey, secrets Vault). This amendment specifies how Mechanism #3 (integration event via outbox) is dispatched. **Updated dispatch flow:** diff --git a/docs/decisions/0014-adopt-dapr.md b/docs/decisions/0014-adopt-dapr.md index 275355d..0303e51 100644 --- a/docs/decisions/0014-adopt-dapr.md +++ b/docs/decisions/0014-adopt-dapr.md @@ -15,7 +15,7 @@ LearnStack adopts **Dapr** (Distributed Application Runtime) for three building | Building block | Backend (production) | Component file | |----------------|---------------------|----------------| | Pub/Sub | Apache Kafka | `dapr/components/pubsub.yaml` | -| State store | Redis | `dapr/components/statestore.yaml` | +| State store | Valkey | `dapr/components/statestore.yaml` | | Secret store | HashiCorp Vault | `dapr/components/secretstore-vault.yaml` | Application code interacts with Dapr **exclusively through wrapped abstractions** — @@ -38,8 +38,8 @@ LearnStack is a modular monolith multi-tenant PaaS for education. It needs: The platform plans triple deployment (SaaS / Dedicated / Self-Hosted; ADR-0020). All three must use the same backend abstractions; backend providers may differ per deployment -(Vault in production, file-based secret store in dev; managed Redis in SaaS, self-hosted -Redis on-prem). +(Vault in production, file-based secret store in dev; managed Valkey in SaaS, self-hosted +Valkey on-prem). Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, `Nexora/docs/decisions/0005-transactional-outbox.md`, @@ -50,7 +50,7 @@ Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, module-level coupling to Dapr** — the interfaces live in SharedKernel, the `DaprXxxService` implementation lives in Infrastructure, and modules never import `Dapr.Client`. -- Component swap (Kafka → another broker, Redis → another KV) is a configuration-only +- Component swap (Kafka → another broker, Valkey → another KV) is a configuration-only change with the same interface signatures. - Same sidecar pattern works in Docker Compose (dev), Kubernetes (production), and air-gapped installations (on-prem with bundled sidecar binary). @@ -58,7 +58,7 @@ Nexora's experience (see `Nexora/docs/architecture/COMMUNICATION_FLOW.md`, ## Decision drivers 1. **Provider portability.** A LearnStack deployment may need to replace Kafka with RabbitMQ, - Redis with KeyDB, Vault with AWS Secrets Manager. Dapr's component model treats this as a + Valkey with KeyDB, Vault with AWS Secrets Manager. Dapr's component model treats this as a YAML change, not a code change. 2. **Same abstraction across deployments.** SaaS uses managed Kafka; Self-Hosted uses bundled Kafka; both call the same `IEventBus.PublishAsync`. The differentiating layer is @@ -207,7 +207,7 @@ Three architecture tests enforce the abstraction boundary (added in Phase 02): ### Positive -- Provider portability: switching Kafka → RabbitMQ, Redis → KeyDB, Vault → AWS Secrets +- Provider portability: switching Kafka → RabbitMQ, Valkey → KeyDB, Vault → AWS Secrets Manager is a Dapr component YAML change. - Same code, same components across SaaS / Dedicated / Self-Hosted. - Outbox + at-least-once delivery + retry + DLQ provided by Dapr pub/sub; LearnStack's diff --git a/docs/decisions/0022-custom-domain-tls.md b/docs/decisions/0022-custom-domain-tls.md index 416c776..6e7c477 100644 --- a/docs/decisions/0022-custom-domain-tls.md +++ b/docs/decisions/0022-custom-domain-tls.md @@ -345,7 +345,7 @@ public sealed class TenantMiddleware } ``` -`_hostToTenantResolver` is backed by `ICacheService` (Dapr State / Redis); cache key +`_hostToTenantResolver` is backed by `ICacheService` (Dapr State / Valkey); cache key `hub:host:{host}` invalidated on `CustomDomainActivatedEvent` / `CustomDomainRevokedEvent`. ### Public suffix list validation diff --git a/docs/decisions/0029-object-storage-seaweedfs.md b/docs/decisions/0029-object-storage-seaweedfs.md index 196983c..2c2895d 100644 --- a/docs/decisions/0029-object-storage-seaweedfs.md +++ b/docs/decisions/0029-object-storage-seaweedfs.md @@ -78,7 +78,7 @@ Image: `chrislusf/seaweedfs:latest` (pinned to a specific tag per the dev compose pins the current stable tag). This ADR **supersedes the storage choice in ADR-0002 only** — the rest of -ADR-0002 (Postgres, Redis, modular monolith) is unchanged. +ADR-0002 (Postgres, Valkey, modular monolith) is unchanged. ## Context diff --git a/docs/decisions/0030-redis-compatible-store-valkey.md b/docs/decisions/0030-redis-compatible-store-valkey.md new file mode 100644 index 0000000..808c83b --- /dev/null +++ b/docs/decisions/0030-redis-compatible-store-valkey.md @@ -0,0 +1,180 @@ +# ADR-0030: Redis-compatible Store — Valkey + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "Redis" choice on +the cache + state-store row only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **Redis Inc. left BSD in 2024-03.** Redis 7.4 was the last BSD-3-Clause + release. From 7.4 onward the project ships under a triple-license + (AGPLv3 / RSALv2 / SSPLv1) where the user picks one. None of those are + permissive OSS in the BSD sense, and SSPL in particular has not been + tested in court for indirect-SaaS usage — a gray area we do not want a + Self-Hosted tenant's legal team to discover months into a deployment. +- **Valkey** is the Linux-Foundation-governed BSD-3-Clause fork of Redis + 7.2.4. AWS, Google, Oracle, Ericsson, Snap, Tencent and others are + first-tier sponsors; the governance is vendor-neutral by design. + Valkey 8.x reached protocol + command parity with Redis 7.4 in + 2024-11 and added meaningful CPU/RAM improvements on top + (multi-threaded I/O, smarter command pipelining). +- **Drop-in compatibility.** Valkey speaks the RESP protocol identically + to Redis. Every library the LearnStack stack uses — `StackExchange.Redis` + (consumed only inside `LearnStack.Infrastructure` per ADR-0014), + Dapr's `state.redis` component, `ICacheService` over Dapr — sees the + same wire protocol. Switching backends is one image tag and one + `redisHost` value. +- **Provider portability is non-negotiable.** Per + [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md) + the cache + state backend already sits behind `ICacheService` (Dapr + building block). The provider portability claim of ADR-0014 only + holds when the underlying provider is not on a license-shift + trajectory; Valkey honours that claim, Redis Inc. 8.x trajectory + weakens it. +- **Self-Hosted Air-Gapped must work.** Triple-deployment model + ([ADR-0020](0020-triple-deployment-hybrid-license.md)) requires a + backend with no phone-home, no license-key check, no commercial-tier + gating. Valkey satisfies that; the SSPL/RSALv2 sides of Redis 8.x + introduce ambiguity at the Self-Hosted boundary. +- **Ecosystem signal.** Debian 13, Ubuntu 25.04, RHEL 10 made Valkey the + default `redis`-named package. AWS ElastiCache for Valkey is ~20% + cheaper than ElastiCache for Redis. The boring-choice principle + (Standards 00 § 9) now points at Valkey, not Redis. + +## Considered Options + +1. **Valkey** (chosen). BSD-3-Clause, Linux Foundation governance, + RESP-protocol drop-in for Redis 7.4. +2. **Stay on Redis 7.4 indefinitely** (rejected). The 7.4 line gets + security patches under the old license, but no new features arrive; + the rest of the world moves to either Valkey or Redis 8.x, and our + image freeze becomes an EOL clock. +3. **Adopt Redis 8.x (AGPLv3 selection)** (rejected). AGPL applied to a + network-reachable backend in a SaaS context is a copyleft surface a + proprietary platform like LearnStack should not adopt without legal + review; the gain over Valkey is small (vendor parity), the risk is + real (license interpretation). +4. **DragonflyDB** (rejected). High-performance RESP-compatible + in-memory store, BSL-licensed (also source-available, not OSS by OSI + definition). Solves the wrong problem — performance is not the + bottleneck, license clarity is. +5. **KeyDB** (rejected). Active development slowed substantially after + Snap acquisition; the project's future under EVA Information Security + is unclear. Not a stable bet. + +## Decision + +LearnStack adopts **Valkey** as the Redis-compatible cache + state +backend behind `ICacheService` (Dapr `state.redis` component) for all +four deployment modes. The Dapr component name `state.redis` is the +**Dapr provider-type identifier** (RESP-compatible store) — it does not +imply the Redis Inc. brand and does not change. + +Image (dev compose): `valkey/valkey:8.1-alpine`, pinned per +[Standards 12 § Image Conventions](../standards/12-infrastructure.md). +Production swaps to AWS ElastiCache for Valkey / equivalent managed +offering through the composition root. + +This ADR **supersedes the cache + state-store row of ADR-0002 only** — +the rest of ADR-0002 (.NET 10, EF Core, Next.js, modular monolith) +stays. Together with [ADR-0029 (SeaweedFS)](0029-object-storage-seaweedfs.md), +both backend rows of ADR-0002 now have explicit successor decisions. + +## Context + +The original choice (Redis) was correct in 2023 — BSD-3-Clause, widely +known, every library + cloud provider supported it. The 2024-03 license +shift removed the unambiguous BSD path; the BSD-3-Clause continuation +moved to the Valkey fork. + +The platform-level commitment to keep Self-Hosted Air-Gapped first-class +(ADR-0020) is what forces the choice now rather than later: a Self- +Hosted tenant inherits whatever license the backend ships with, and we +do not want their security/legal review to find SSPL-laden code paths +they then have to interpret. Valkey moves that conversation off the +table entirely. + +### Backward-compat: what does NOT change + +- **`ICacheService`** interface (`LearnStack.SharedKernel`) — unchanged. +- **Dapr `state.redis` component name** — unchanged. The component is the + RESP-protocol adapter; Valkey is RESP, so it consumes the same + component. +- **`StackExchange.Redis` library usage** — unchanged. The library + connects to anything speaking RESP. +- **`IConnectionMultiplexer` forbidden-in-modules rule** (Standards 20) + — unchanged. The rule is about not importing the library outside + `LearnStack.Infrastructure`; the rule does not care about the backend + vendor. +- **Architecture tests** `ICacheService_Is_OnlyCacheAbstraction`, + `Modules_DoNotReference_DaprPackage` — unchanged. + +### What DOES change + +- `infra/compose/dev.yml`: service `redis` → `valkey`; image + `redis:7.4-alpine` → `valkey/valkey:8.1-alpine`; volume `redis-data` + → `valkey-data`; `depends_on: redis:` callers → `valkey:`. +- `infra/dapr/components/statestore-redis.yaml`: `redisHost: redis:6379` + → `redisHost: valkey:6379` (the file name stays — `state.redis` is the + Dapr provider-type convention). +- The "boring choice" enumeration in standards / CLAUDE.md / README.md + / ADR-0002: "Redis 7" → "Valkey 8". +- `.gitignore`: `redis-data/` → `valkey-data/`. + +## Consequences + +### Positive + +- License surface stays BSD-3-Clause — no AGPL/SSPL/RSALv2 to interpret. +- Self-Hosted Air-Gapped stays first-class. +- Vendor-neutral governance (Linux Foundation under) — no single-vendor + policy-shift risk. +- AWS ElastiCache for Valkey ~20% cheaper than for Redis — direct SaaS + margin win. +- Drop-in: zero application-code change required; Dapr component swap is + one YAML edit. + +### Negative + +- One-time dev image swap + the doc/scaffold sweep. Both mechanical. +- Smaller commercial-support market than Redis Inc.'s, today. Mitigated + by Linux Foundation backing + the major cloud providers committing + managed offerings. +- A subset of Redis Inc.'s newest commercial features (Redis Search, + Redis JSON commercial extensions in 8.x) does not exist in Valkey. + None of these are on LearnStack's roadmap; flagging so a future + feature ADR that wants them knows the choice. + +### Neutral + +- The `MINIO_`-style env-var rename trap does not apply here — Valkey + honours the same `REDISCLI_AUTH` / connection-string conventions + Redis does. +- Production swap to AWS / GCP / Azure managed offerings remains a + composition-root edit through `ICacheService`. + +## Implementation Notes + +- **This commit** (Phase 01 packet 6 cleanup): dev compose service swap; + Dapr component `redisHost` update; doc + standards sweep; `.gitignore` + volume name update. +- **Phase 02a** (Cache adapter): `LearnStack.Infrastructure.Caching.Dapr` + consumes `state.redis` component against Valkey — no code-level + awareness of the backend vendor. +- **Phase 11** (production hardening): production sizing + Valkey HA + topology decision (Sentinel vs Cluster mode) lives in its own ADR if + it diverges from the default. + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original cache + state row, now partially superseded. +- [ADR-0014 Adopt Dapr](0014-adopt-dapr.md) — `ICacheService` over Dapr `state.redis`. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) — Self-Hosted Air-Gapped requirement that motivated the move. +- [Standards 12 § Local Infrastructure](../standards/12-infrastructure.md) +- [Standards 20 § Composition Root](../standards/20-infrastructure-stack.md) +- Valkey upstream: . diff --git a/docs/decisions/0031-postgresql-major-version.md b/docs/decisions/0031-postgresql-major-version.md new file mode 100644 index 0000000..2f60b5d --- /dev/null +++ b/docs/decisions/0031-postgresql-major-version.md @@ -0,0 +1,184 @@ +# ADR-0031: PostgreSQL — Start on Major Version 18 + +## Status + +Accepted + +**Date:** 2026-05-19 +**Deciders:** @platform +**Supersedes (partial):** ADR-0002 — Initial Architecture (the "PostgreSQL" +major-version choice only; the rest of ADR-0002 stands) + +## Decision Drivers + +- **LearnStack is pre-implementation.** No migrations exist yet, no + production database exists, no data exists. The migration-drag cost + of choosing the wrong major version is zero today and grows + exponentially after the first deployed schema. +- **Postgres 18 is the longest-runway LTS available.** EOL 2030-11 + versus 16 LTS at 2028-11. Starting on 18 buys an extra two years of + upstream patches before any forced major upgrade. +- **`gen_uuid_v7()` is native in 18.** LearnStack's + [ADR-0023 (Strongly-typed ID source generator)](README.md) is a + pending draft considering UUIDv7 as the canonical id format + (time-ordered, index-friendly). Postgres 18 ships a built-in + `gen_uuid_v7()` SQL function — DB-side DEFAULT values become trivial, + the app side keeps the strongly-typed wrapping, no extension is + required. Postgres 16 / 17 force a choice between an extension + (`pg_uuidv7`) and app-side generation; Postgres 18 closes that gap + natively. +- **Async I/O for sequential scans (Postgres 18).** On NVMe-backed + production hardware the perf improvement is measurable and + particularly relevant for the partitioned `audit_log` (per + [ADR-0016](0016-audit-log-subsystem.md)) scans that operators run + during incident review. +- **OAuth authentication (Postgres 18).** Opens a future option to + shorten the Keycloak → Postgres auth path for diagnostic / + break-glass scenarios. Not adopted today (the + [ADR-0004](0004-authentication-strategy.md) realm-based posture + stays), but worth recording as a Phase-11 lever. +- **EF Core + Npgsql provider parity.** `Npgsql.EntityFrameworkCore.PostgreSQL` + 10.0.0 (already pinned in `Directory.Packages.props`) supports + Postgres 18 features. Switching majors does not require a provider + bump; the same csproj graph works. +- **RLS-specific defaults are unchanged.** Tenant + organization + isolation defense-in-depth ([ADR-0003 Amendment 1](0003-tenant-isolation-defense-in-depth.md)) + uses the `PERMISSIVE` / `RESTRICTIVE` RLS-policy primitives + the + `app.tenant_id` / `app.organization_id` session-var pattern. None of + this changed across Postgres 16 / 17 / 18; the policy shape we will + write in Phase 02a works identically on 18. + +## Considered Options + +1. **Start on PostgreSQL 18.x** (chosen). Newest LTS, longest runway, + native UUIDv7, async I/O perf, pre-implementation = zero migration + cost. +2. **Stay on 16 LTS until production** (rejected). Defers the major + upgrade to Phase 11 where every migration already exists; the + `pg_upgrade` exercise + extension recompile + app-side UUIDv7 swap + become a multi-day operational task instead of a zero-cost image + bump. +3. **Skip to 17, defer 18 until proven** (rejected). 17 is already + superseded by 18 LTS; adopting 17 buys nothing 18 doesn't, and + forces the same upgrade question in 18 months. +4. **Adopt 18 only for dev, keep 16 for production** (rejected). + Splits the deployment-mode portability promise (ADR-0020) — every + feature has to be tested on both major versions, every migration + has to be written for the lowest-common-denominator. The single- + binary, single-major-Postgres posture is simpler and safer. + +## Decision + +LearnStack's primary RDBMS is **PostgreSQL 18.x** across all four +deployment modes. Dev compose pins `postgres:18.4-alpine`. EF Core +provider (`Npgsql.EntityFrameworkCore.PostgreSQL` 10.0.0) targets 18. +Tenant + organization RLS policies are written for the 18 syntax (no +divergence from 16 / 17 here; the choice is forward-looking, not a +compatibility break). + +This ADR **supersedes the PostgreSQL major-version choice in ADR-0002 +only** — the rest of ADR-0002 (EF Core, modular monolith, …) stays. +Together with [ADR-0029 (SeaweedFS)](0029-object-storage-seaweedfs.md) +and [ADR-0030 (Valkey)](0030-redis-compatible-store-valkey.md), all +three backend rows of ADR-0002 now have explicit successor decisions. + +## Context + +### Why now + +Every major upgrade gets harder as the schema grows. A pre- +implementation codebase has no schema. The cost of choosing 18 today is +one image tag change + a `docker compose down -v`; the cost of doing it +in Phase 11 is `pg_upgrade` across every production cluster + extension +recompilation + a possible app-side UUIDv7 generator swap if the +extension we picked early diverges from the 18 native function. + +### What 18 brings that we directly benefit from + +| 18 feature | LearnStack benefit | +|------------|--------------------| +| `gen_uuid_v7()` built-in | ADR-0023 draft can pick "DB-side DEFAULT" without committing to an extension | +| Async I/O for sequential scans | `audit_log` partition scans (ADR-0016) — operator query latency | +| OAuth authentication | Optional shortcut for Phase 11 break-glass paths (not adopted today) | +| Virtual generated columns | Computed columns for `LocalizedMessage`-like derived data (Phase 02a+) | +| `EXPLAIN (ANALYZE)` improvements | Day-to-day query tuning | + +### What does NOT change + +- **EF Core provider** — `Npgsql.EntityFrameworkCore.PostgreSQL` 10.0.0 + already supports 18. +- **RLS policy syntax** — identical in 16 / 17 / 18. +- **Connection-string + role provisioning** — same. +- **Session-var pattern** for `app.tenant_id` / `app.organization_id` + in transaction-local config — unchanged. +- **`pg_partman` / `pg_stat_statements` / `pgcrypto` / `citext`** — + all 18-ready in recent releases. +- **Dapr state store + outbox dispatcher patterns** — RDBMS-agnostic. + +### What the upgrade costs in dev + +A single line in `infra/compose/dev.yml` (`postgres:16.14-alpine` → +`postgres:18.4-alpine`) plus a one-time `docker compose -f +infra/compose/dev.yml down -v && up -d` to wipe the incompatible +catalog. No code change, no migration change (there are no migrations +yet). + +### What the upgrade costs in production (deferred to Phase 11) + +The first production deployment lands in Phase 11 anyway. Choosing 18 +now means Phase 11 deploys *fresh* on 18; there is no `pg_upgrade` +exercise to schedule. If LearnStack ever runs an earlier production +preview, that preview already runs on 18 — no major upgrade needed. + +## Consequences + +### Positive + +- Longest support runway (EOL 2030-11) — five-year horizon before any + forced major upgrade. +- Native UUIDv7 — ADR-0023 design space widens. +- Async I/O perf — direct benefit to `audit_log` and any future + read-heavy partitioned table. +- Phase 11 production deployment ships on the modern LTS without a + separate "upgrade Postgres" mini-project on its critical path. + +### Negative + +- 18 is younger than 16 LTS — community knowledge base is thinner; a + rare edge-case query plan regression may take longer to resolve via + StackOverflow / mailing list. Mitigated by EF Core provider maturity + + the broader Postgres ecosystem's quick uptake of LTS releases. +- Some Postgres-as-a-service offerings (some smaller cloud regions, a + handful of niche providers) may lag 18 availability by 1–2 quarters. + Mitigated by ADR-0020 portability: the SaaS / Dedicated deployment + modes can pin to a specific managed offering and the Self-Hosted + modes ship with the container image. + +### Neutral + +- Backup tooling (`pg_basebackup`, `pg_dump`) shape is unchanged — + Phase 11 backup runbooks stay the same template. +- `Standards 12 § Database Operations` continues to apply verbatim + (daily logical backups for dev-grade restore, continuous WAL + archiving in production). + +## Implementation Notes + +- **This commit** (Phase 01 packet 6 cleanup): dev compose image bump; + ADR-0002 Amendment 2 references this decision; doc sweep across + Standards 12 / Architecture / Standards 20. +- **Phase 02a** (Platform kernel): first EF migration targets Postgres + 18; if ADR-0023 picks UUIDv7, evaluate DB-side `gen_uuid_v7()` as the + default-value generator. +- **Phase 11** (production hardening): production sizing, backup + cadence, replication topology — all written for 18. + +## References + +- [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original PostgreSQL major-version row, now partially superseded. +- [ADR-0003 Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) — RLS pattern unchanged across 16/17/18. +- [ADR-0016 Audit Log Subsystem](0016-audit-log-subsystem.md) — partitioned `audit_log` benefits from async I/O. +- [ADR-0023 Strongly-typed ID source generator](README.md) — draft; UUIDv7 native in 18 widens the design space. +- [Standards 05 — Database](../standards/05-database.md) +- [Standards 12 § Database Operations](../standards/12-infrastructure.md) +- PostgreSQL 18 release notes: . diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 500e49f..fc56028 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -16,7 +16,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | # | Title | Topic | |---|---|---| | 0001 | [Platform Name](0001-platform-name.md) | Name and naming conventions | -| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Redis + SeaweedFS + Next.js; modular monolith | +| 0002 | [Initial Architecture](0002-initial-architecture.md) | .NET 10 + EF Core + PostgreSQL + Valkey + SeaweedFS + Next.js; modular monolith | | 0003 | [Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) | Query filters + RLS + audit + architecture tests (Amendment 1: Organization scope, 2026-05-18) | | 0004 | [Authentication Strategy](0004-authentication-strategy.md) | Off-the-shelf identity provider preferred over hand-rolled auth (Amendment 1: `learnstack-hub` realm, 2026-05-18) | | 0005 | [Live Classroom Media Stack](0005-live-classroom-media-stack.md) | LiveKit OSS self-hosted by default; LiveKit Cloud optional; no custom SFU | @@ -28,7 +28,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0011 | _Superseded — see below_ | Was: Vertical Extension Points | | 0012 | [Search Strategy](0012-search-strategy.md) | Meilisearch; one instance per env; index-per-(kind, locale); tenant_id as query filter | | 0013 | [Page Block Schema Versioning](0013-page-block-schema-versioning.md) | `(key, schemaVersion)` tuple; immutable schemas; lazy + bulk migration; placeholder on unknown version | -| 0014 | [Adopt Dapr](0014-adopt-dapr.md) | Dapr building blocks for pub/sub (Kafka), state (Redis), secrets (Vault); abstracted behind SharedKernel interfaces | +| 0014 | [Adopt Dapr](0014-adopt-dapr.md) | Dapr building blocks for pub/sub (Kafka), state (Valkey), secrets (Vault); abstracted behind SharedKernel interfaces | | 0015 | [API Gateway with APISIX](0015-api-gateway-apisix.md) | APISIX standalone mode; JWT + rate limit + CORS + correlation-id at the edge; defense-in-depth | | 0016 | [Audit Log Subsystem](0016-audit-log-subsystem.md) | `LearnStack.Modules.Audit`; EF interceptor + `IAuditStateCapture` + `AuditLogBehavior`; partitioned `audit_log` table; retention | | 0017 | [Tenant + Organization Hierarchy](0017-tenant-organization-hierarchy.md) | Two-level: Tenant → Organization; permission scope Platform / Tenant / Organization | @@ -37,7 +37,9 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0020 | [Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) | SaaS / Dedicated / Self-Hosted from one codebase; phone-home + RSA-signed key + 30-day grace | | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | -| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's SeaweedFS row | +| 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | +| 0030 | [Valkey-compatible Store — Valkey](0030-redis-compatible-store-valkey.md) | Valkey (Linux Foundation, BSD-3-Clause) for the cache + Dapr state-store backend; RESP-protocol drop-in; partially supersedes ADR-0002's Valkey row | +| 0031 | [PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) | Pin primary RDBMS major version to PostgreSQL 18; native `gen_uuid_v7()` + async I/O + longest LTS runway; partially supersedes ADR-0002's PostgreSQL row | ## Superseded ADRs diff --git a/docs/glossary.md b/docs/glossary.md index d60f402..048f500 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -245,9 +245,9 @@ This glossary defines LearnStack-specific terms. When a term is ambiguous across | Term | Definition | |------|------------| -| **Dapr Building Blocks** | The three Dapr abstractions LearnStack uses: pub/sub (Kafka), state (Redis), secrets (Vault) per [ADR-0014](decisions/0014-adopt-dapr.md). Service invocation, workflow, bindings, and actors are out of scope. | +| **Dapr Building Blocks** | The three Dapr abstractions LearnStack uses: pub/sub (Kafka), state (Valkey), secrets (Vault) per [ADR-0014](decisions/0014-adopt-dapr.md). Service invocation, workflow, bindings, and actors are out of scope. | | **`IEventBus`** | Interface for publishing integration events. Backed by `DaprEventBus` (production) or `InProcessEventBus` (development). The `OutboxProcessor` is the only sanctioned caller. | -| **`ICacheService`** | Interface for cache reads / writes. Backed by `DaprCacheService` (production, Redis-backed) or `InMemoryCacheService` (development). Cache keys carry `{tenant_id}` prefix. | +| **`ICacheService`** | Interface for cache reads / writes. Backed by `DaprCacheService` (production, Valkey-backed) or `InMemoryCacheService` (development). Cache keys carry `{tenant_id}` prefix. | | **`ISecretProvider`** | Interface for secret reads. Backed by `DaprSecretProvider` (production, Vault) or `EnvironmentSecretProvider` (development). Secret namespace `learnstack/{deployment}/{module}/{key}`. | | **`IEntitlementProvider`** | Interface for the entitlement source. Implementations: `NullEntitlementProvider` (dev), `HubEntitlementProvider` (SaaS / Dedicated), `SignedLicenseKeyEntitlementProvider` (Self-Hosted). | | **`IHostToTenantResolver`** | Interface for host → `(tenant_id, organization_id?)` resolution. Backed by `platform_host_to_tenant`. | diff --git a/docs/roadmap/phase-00-product-architecture.md b/docs/roadmap/phase-00-product-architecture.md index 2bf46bd..0fa8772 100644 --- a/docs/roadmap/phase-00-product-architecture.md +++ b/docs/roadmap/phase-00-product-architecture.md @@ -70,7 +70,7 @@ later should be made explicit before implementation begins. - .NET 10 backend. - EF Core and PostgreSQL. -- Redis and SeaweedFS. +- Valkey and SeaweedFS. - Next.js frontend. - Modular monolith. - Shared-database multi-tenancy for the initial implementation. diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 25e139f..05347f5 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -21,7 +21,7 @@ > `pnpm-lock.yaml` committed; `postinstall` hook stubs `.next/types/routes.d.ts`. > > **Packet 3 — Core dev compose ✅** -> `infra/compose/dev.yml` with PostgreSQL 16, Redis 7, SeaweedFS + console, Mailpit +> `infra/compose/dev.yml` with PostgreSQL 18, Valkey 7, SeaweedFS + console, Mailpit > (binary `readyz` healthcheck), Meilisearch — pinned tags, healthchecks, > named volumes, dev-only credential banners. > @@ -159,8 +159,8 @@ in the `learnstack-hub` repository* per Docker Compose under `infra/compose/`: -- PostgreSQL 16. -- Redis 7. +- PostgreSQL 18. +- Valkey 7. - SeaweedFS + SeaweedFS console. - Mailpit (outbound email). - Meilisearch. @@ -204,7 +204,7 @@ Two compose files: - Working backend solution scaffolded with modular layout. - Working frontend workspace with the single Next.js app. -- Local Docker Compose infrastructure with PostgreSQL, Redis, SeaweedFS, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. +- Local Docker Compose infrastructure with PostgreSQL, Valkey, SeaweedFS, Mailpit, Meilisearch, LiveKit, Coturn, Keycloak. - Initial CI pipeline. - Local development documentation. - `make seed` populating two demo tenants + one platform admin user. @@ -213,7 +213,7 @@ Two compose files: - A new developer can clone the repository and start the local environment by following one document. - Backend API responds on `GET /healthz`. -- PostgreSQL, Redis, SeaweedFS, LiveKit, Coturn, Keycloak all run locally via compose. +- PostgreSQL, Valkey, SeaweedFS, LiveKit, Coturn, Keycloak all run locally via compose. - Frontend builds and serves the three route segments. - CI passes on `main`. - The architecture-test project is set up and green even before domain features exist. diff --git a/docs/roadmap/phase-02a-kernel-tenancy.md b/docs/roadmap/phase-02a-kernel-tenancy.md index 5c3f61c..5f23284 100644 --- a/docs/roadmap/phase-02a-kernel-tenancy.md +++ b/docs/roadmap/phase-02a-kernel-tenancy.md @@ -59,7 +59,7 @@ They are codified in: Per [ADR-0014](../decisions/0014-adopt-dapr.md): - Dapr sidecar runs in dev `docker-compose.yml`. Pub/sub component → Kafka. State - component → Redis. Secrets component → Vault (dev mode). + component → Valkey. Secrets component → Vault (dev mode). - `DaprEventBus`, `DaprCacheService`, `DaprSecretProvider` implementations ship in `LearnStack.Infrastructure`. - Topic naming convention enforced by `Dapr_PubSub_TopicNames_FollowConvention` @@ -219,7 +219,7 @@ The architecture test project starts going green during this phase. Phase 02a co - `Dapr_PubSub_TopicNames_FollowConvention`. - `AuditEntry_Inherits_Entity_Not_AuditableEntity`. - `LearnStack_Modules_DoNotReference_Hub`. -- `Modules_Do_Not_Inject_Redis_Directly`, `Modules_Do_Not_Read_Entitlement_Cache_Directly`, +- `Modules_Do_Not_Inject_Valkey_Directly`, `Modules_Do_Not_Read_Entitlement_Cache_Directly`, `Modules_Do_Not_Write_AuditLog_Directly`. - `Modules_Do_Not_Reference_DeploymentMode` — modules never read `DeploymentMode` directly; the composition root selects provider implementations once. See diff --git a/docs/standards/00-principles.md b/docs/standards/00-principles.md index 2099349..6a767e6 100644 --- a/docs/standards/00-principles.md +++ b/docs/standards/00-principles.md @@ -40,7 +40,7 @@ Anything that crosses the LearnStack boundary — payments, email, SMS, search, identity, live-class media, the **Hub**, **entitlement source**, **host→tenant resolution**, **event bus**, **cache**, **secret store** — lives behind an interface. The domain code knows nothing about Stripe, Postmark, SeaweedFS, Keycloak, LiveKit, Dapr, -Kafka, Redis, Vault, or the Hub. Provider-specific code lives in +Kafka, Valkey, Vault, or the Hub. Provider-specific code lives in `Infrastructure.` packages. Swapping a provider is a composition-root edit, not a code change. @@ -66,7 +66,7 @@ A failing test should tell a reader what behavior we promised. Test names read l ## 9. Default to Boring -We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Redis, SeaweedFS, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. +We use mature, well-known technologies until measured pain forces an exception. PostgreSQL, Valkey, SeaweedFS, ASP.NET Core, Next.js, Hangfire — boring on purpose. The interesting parts of LearnStack are the education domain and the platform composition, not the infrastructure choices. ## 10. Cost-Aware From Day One diff --git a/docs/standards/05-database.md b/docs/standards/05-database.md index caa74e6..d914d4f 100644 --- a/docs/standards/05-database.md +++ b/docs/standards/05-database.md @@ -13,7 +13,7 @@ PostgreSQL schema, EF Core, and migration conventions. ## Database -- **PostgreSQL 16+** in all environments. +- **PostgreSQL 18+** in all environments. - One database per environment; single schema (`public`). - One `DbContext` per module (not one global). - Schema migrations live with the owning module. diff --git a/docs/standards/06-testing.md b/docs/standards/06-testing.md index 04f6328..ef333e2 100644 --- a/docs/standards/06-testing.md +++ b/docs/standards/06-testing.md @@ -11,7 +11,7 @@ Test pyramid, conventions, and what every change must cover. flowchart TB e2e[End-to-end / Playwright
handful of golden flows] contract[Contract & API tests
OpenAPI + provider fakes] - integration[Integration tests
Testcontainers Postgres / Redis / SeaweedFS] + integration[Integration tests
Testcontainers Postgres / Valkey / SeaweedFS] arch[Architecture tests
module boundaries + tenant invariants] unit[Unit tests
domain + application + UI logic] @@ -39,7 +39,7 @@ We invest most at **unit + integration**. Architecture tests are zero-flake. E2E ### Integration Tests -- Real Postgres + Redis + SeaweedFS via Testcontainers. +- Real Postgres + Valkey + SeaweedFS via Testcontainers. - One Postgres database per test class (or Respawn between tests). - Real module configuration; no mocked repositories. - Cover happy path, edge cases, and **every tenant-isolation invariant**. diff --git a/docs/standards/09-error-handling.md b/docs/standards/09-error-handling.md index 62b1673..54243bb 100644 --- a/docs/standards/09-error-handling.md +++ b/docs/standards/09-error-handling.md @@ -63,7 +63,7 @@ Standard error codes (machine-readable, stable): ``` LearnStackException (base) ├── DomainException (domain invariant broken from inside, programmer error) -├── InfrastructureException (DB, Redis, SeaweedFS transient) +├── InfrastructureException (DB, Valkey, SeaweedFS transient) ├── ProviderException (upstream provider error) │ ├── PaymentProviderException │ ├── LiveClassProviderException diff --git a/docs/standards/10-observability.md b/docs/standards/10-observability.md index 2e068c8..8289cd7 100644 --- a/docs/standards/10-observability.md +++ b/docs/standards/10-observability.md @@ -98,7 +98,7 @@ Auto-instrument: - MediatR commands and queries. - Hangfire job invocations. - Outbox dispatcher batches. -- Redis client calls. +- Valkey client calls. - SeaweedFS/S3 SDK calls. - LiveKit provider calls. diff --git a/docs/standards/12-infrastructure.md b/docs/standards/12-infrastructure.md index 2559f13..4f01a35 100644 --- a/docs/standards/12-infrastructure.md +++ b/docs/standards/12-infrastructure.md @@ -177,7 +177,7 @@ See [10-observability.md](10-observability.md). - The `/api/internal/*` route set is reachable only through a separate APISIX route guarded by the `mtls` plugin, with the LearnStack-internal CA pinned. - Strict ingress rules; only documented ports open. -- Private VPC for backend services; database, Redis, Kafka, Vault not on public +- Private VPC for backend services; database, Valkey, Kafka, Vault not on public internet. - Outbound calls allow-listed where feasible. Hub outbound traffic is allow-listed per environment. @@ -190,7 +190,7 @@ See [10-observability.md](10-observability.md). | Dapr sidecar (per API/worker pod) | 0.25 vCPU | 256 MB | runs alongside each pod | | Workers (per instance) | 1 vCPU | 2 GB | autoscale 1–4 | | Postgres | 4 vCPU | 16 GB | initial; scale as needed | -| Redis | 1 vCPU | 2 GB | initial | +| Valkey | 1 vCPU | 2 GB | initial | | Kafka (per broker) | 2 vCPU | 4 GB | 3-broker cluster baseline | | Vault | 1 vCPU | 1 GB | HA mode in production (3 nodes) | | APISIX | 1 vCPU | 1 GB | autoscale 2–4 | diff --git a/docs/standards/20-infrastructure-stack.md b/docs/standards/20-infrastructure-stack.md index 604d40e..304f272 100644 --- a/docs/standards/20-infrastructure-stack.md +++ b/docs/standards/20-infrastructure-stack.md @@ -49,7 +49,7 @@ Rules: | Concern | `Development` | `SaaS` | `Dedicated` | `SelfHostedOnline` | `SelfHostedAirGapped` | |---|---|---|---|---|---| | Event bus | `InProcessEventBus` (MediatR) | `DaprEventBus` → Kafka | `DaprEventBus` → Kafka | `DaprEventBus` → Kafka (single-broker OK) | `DaprEventBus` → Kafka (single-broker OK) | -| Cache | `InMemoryCacheService` | `DaprCacheService` → Redis | `DaprCacheService` → Redis | `DaprCacheService` → Redis | `DaprCacheService` → Redis | +| Cache | `InMemoryCacheService` | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | `DaprCacheService` → Valkey | | Secrets | `EnvironmentSecretProvider` | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault | `DaprSecretProvider` → Vault or file | | Entitlement | `NullEntitlementProvider` | `HubEntitlementProvider` | `HubEntitlementProvider` | `HubEntitlementProvider` (phone-home) | `SignedLicenseKeyEntitlementProvider` | | Host → tenant | Config / single tenant | Hub-mirrored projection | Hub-mirrored projection | Hub-mirrored projection | Config / `.lic` claim | @@ -80,9 +80,9 @@ ADR-0014 non-goals; do not introduce them without a new ADR. ### `ICacheService` (state) -- All Redis access goes through `ICacheService`. Direct `IConnectionMultiplexer` / +- All Valkey access goes through `ICacheService`. Direct `IConnectionMultiplexer` / `IDistributedCache` injections are forbidden by the architecture test - `Modules_Do_Not_Inject_Redis_Directly`. + `Modules_Do_Not_Inject_Valkey_Directly`. - Cache keys are `{tenant_id}:{module}:{logical-name}`. The `tenant_id` prefix is **mandatory** even when a value is platform-wide — use the sentinel `"platform"` tenant id rather than omitting the prefix. @@ -99,7 +99,7 @@ The most-referenced read paths follow this layered policy; mismatches across doc (e.g. "60s cache" vs "15-min TTL") refer to different layers of the same cache, not different decisions: -| Key family | L1 (in-process `IMemoryCache`) | L2 (Dapr state → Redis) | Eager invalidation event | +| Key family | L1 (in-process `IMemoryCache`) | L2 (Dapr state → Valkey) | Eager invalidation event | |---|---|---|---| | `hub:host:{host}` (host → tenant) | 2 min | 15 min | `learnstack.hub.custom-domain.activated/.deactivated` | | `hub:entitlement:{tenant_id}` (plan projection) | 60 s | 15 min (upper bound; Hub-push refresh resets it) | `learnstack.hub.entitlement` | @@ -191,7 +191,7 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) - `IFeatureFlags.IsEnabledAsync(FeatureKey)` is the only sanctioned read path. Direct SQL against `platform_entitlement_cache` outside the Tenancy module's infrastructure is forbidden (architecture test `Modules_Do_Not_Read_Entitlement_Cache_Directly`). -- Cache TTL for the in-process / Redis layer is 60s. Eager invalidation flows from the +- Cache TTL for the in-process / Valkey layer is 60s. Eager invalidation flows from the Dapr event; the TTL is the safety net, not the typical refresh window. - For air-gapped deployments, `SignedLicenseKeyEntitlementProvider` reads a signed `.lic` file and runs the same projection write path; the rest of the system is @@ -234,9 +234,9 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) - LearnStack monolith → APISIX edge → backend pods (1+). - Dapr sidecar runs alongside every backend pod. -- Kafka, Redis, Vault are accessed only via the Dapr sidecar. No direct client +- Kafka, Valkey, Vault are accessed only via the Dapr sidecar. No direct client libraries for these three in application code. -- Postgres is accessed directly (EF Core); Dapr's state-store sits on Redis, not +- Postgres is accessed directly (EF Core); Dapr's state-store sits on Valkey, not Postgres. - SeaweedFS is accessed via the configured S3-compatible client (no Dapr binding). diff --git a/infra/compose/README.md b/infra/compose/README.md index e2cb0ba..18a6fe0 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -13,8 +13,8 @@ eventing → secrets → Dapr sidecar → gateway). Packets 1-6 shipped; packets | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| -| PostgreSQL 16 | `postgres:16.14-alpine` | `localhost:5432` | `learnstack` / `learnstack` | -| Redis 7 | `redis:7.4-alpine` | `localhost:6379` | — | +| PostgreSQL 18 | `postgres:18.4-alpine` | `localhost:5432` | `learnstack` / `learnstack` | +| Valkey 8 | `valkey/valkey:8.1-alpine` | `localhost:6379` | — | | SeaweedFS | `chrislusf/seaweedfs:3.94` | `localhost:9000` (S3), `localhost:9001` (filer UI), `localhost:9333` (master) | S3 access `learnstack` / secret `learnstack-dev-secret` | | Mailpit | `axllent/mailpit:v1.29.7` | `localhost:1025` (SMTP), `localhost:8025` (UI) | accepts any auth | | Meilisearch | `getmeili/meilisearch:v1.44.0` | `localhost:7700` | master key `learnstack-dev-master-key` | diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index cd2c0fa..1dec2ad 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -1,7 +1,7 @@ # LearnStack — local dev infrastructure. # # Scope of THIS file (Phase 01 packets 1-6): the data-plane services -# (Postgres / Redis / SeaweedFS / Mailpit / Meilisearch), the self-hosted Keycloak +# (PostgreSQL 18 / Valkey / SeaweedFS / Mailpit / Meilisearch), the self-hosted Keycloak # identity provider (two realms), the live-media stack (LiveKit OSS + Coturn), # Kafka (KRaft) + kafka-ui, HashiCorp Vault (-dev mode), the Dapr sidecar + # placement (pub/sub + state + secrets building blocks per ADR-0014), and the @@ -27,7 +27,7 @@ x-host-gateway: &host-gateway services: postgres: - image: postgres:16.14-alpine + image: postgres:18.4-alpine container_name: learnstack-postgres restart: unless-stopped environment: @@ -49,17 +49,23 @@ services: timeout: 5s retries: 10 - redis: - image: redis:7.4-alpine - container_name: learnstack-redis + # Valkey replaces Redis per ADR-0030. RESP-protocol drop-in (Dapr + # `state.redis` component, StackExchange.Redis library, ICacheService + # all continue to work unchanged — those names are the protocol / + # library / Dapr provider-type identifier, not the vendor brand). + # The service name is `valkey` so a new reader sees the right backend; + # `redis-cli` continues to ship inside the Valkey image as a symlink. + valkey: + image: valkey/valkey:8.1-alpine + container_name: learnstack-valkey restart: unless-stopped - command: ["redis-server", "--appendonly", "yes"] + command: ["valkey-server", "--appendonly", "yes"] ports: - "6379:6379" volumes: - - redis-data:/data + - valkey-data:/data healthcheck: - test: ["CMD", "redis-cli", "ping"] + test: ["CMD", "valkey-cli", "ping"] interval: 5s timeout: 3s retries: 10 @@ -378,7 +384,7 @@ services: depends_on: kafka: condition: service_healthy - redis: + valkey: condition: service_healthy vault: condition: service_healthy @@ -417,7 +423,7 @@ services: volumes: postgres-data: - redis-data: + valkey-data: seaweedfs-data: meilisearch-data: kafka-data: diff --git a/infra/dapr/README.md b/infra/dapr/README.md index 3126f1b..5f6c4d6 100644 --- a/infra/dapr/README.md +++ b/infra/dapr/README.md @@ -8,7 +8,7 @@ non-goals. | Building block | Backend (dev) | Component file | Application interface | |----------------|---------------|----------------|-----------------------| | Pub/Sub | Kafka (`kafka:9092`) | `components/pubsub-kafka.yaml` | `IEventBus` (Phase 02b) | -| State store | Redis (`redis:6379`) | `components/statestore-redis.yaml` | `ICacheService` (Phase 02a) | +| State store | Valkey (`valkey:6379`, RESP protocol) | `components/statestore-redis.yaml` | `ICacheService` (Phase 02a) | | Secret store | Vault (`http://vault:8200`, dev mode) | `components/secretstore-vault.yaml` | `ISecretProvider` (Phase 02a) | Service invocation, workflow, bindings, **actors**, configuration, and @@ -72,7 +72,7 @@ and ship in Phase 02b. Architecture tests |---------|------------| | Vault root token | `learnstack-dev-root-token` | | Kafka auth | none (`authType: none`, `disableTls: true`) | -| Redis password | (empty) | +| Valkey password | (empty) | All dev-only. Production wires Vault with AppRole / Kubernetes auth and loads the token through Dapr's `secretKeyRef` indirection so the literal diff --git a/infra/dapr/components/statestore-redis.yaml b/infra/dapr/components/statestore-redis.yaml index 25b85b0..29e8a48 100644 --- a/infra/dapr/components/statestore-redis.yaml +++ b/infra/dapr/components/statestore-redis.yaml @@ -1,8 +1,17 @@ -# Dapr state store — Redis backend. +# Dapr state store — RESP-protocol backend (Valkey). # Backs `ICacheService` (L2 cache). Module code never imports # `IDistributedCache` / `StackExchange.Redis` directly per # `ICacheService_Is_OnlyCacheAbstraction` architecture test. # +# `spec.type: state.redis` is the Dapr PROVIDER-TYPE identifier — it +# names the RESP-protocol adapter, NOT the Redis Inc. brand. Per +# ADR-0030 the actual backend is Valkey 8.x (Linux Foundation, +# BSD-3-Clause); Valkey is RESP-protocol-compatible with Redis 7.4 so +# Dapr's `state.redis` adapter consumes it without change. The file +# name keeps the `-redis` suffix because that mirrors the Dapr +# provider-type convention; renaming it would just hide the wire +# protocol the component speaks. +# # `actorStateStore: "false"` is INTENTIONAL — per ADR-0014 non-goals, # Dapr actors are NOT adopted. Flipping this to "true" requires a new # ADR + an architecture-test update. The Dapr docs are explicit that the @@ -18,7 +27,7 @@ spec: version: v1 metadata: - name: redisHost - value: redis:6379 + value: valkey:6379 - name: redisPassword value: "" - name: actorStateStore From cdb100ada6427909a5fc6a78bedbd1e581d703b8 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 22:28:51 +0300 Subject: [PATCH 09/11] =?UTF-8?q?fix:=20PR=20#1=20review=20=E2=80=94=20add?= =?UTF-8?q?ress=2027=20valid=20findings,=20partial=201?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Triaged 28 findings. 27 applied verbatim; F7 (`learnstack-hub-web` fullScopeAllowed=true) intentionally kept with inline dev-posture comment because the reviewer's full fix (`fullScopeAllowed=false` + explicit scopeMappings) would break the demo-operator login flow in dev — production realm provisioning is a separate flow and is the right place to harden public clients with least-privilege scope maps. Blockers / real bugs - `Testcontainers.Valkey` doesn't exist on NuGet — sed-sweep collateral damage from the Redis→Valkey doc pass. Reverted to `Testcontainers.Redis` in both `Directory.Packages.props` and `LearnStack.Tests.Integration.csproj`, with an inline DO-NOT-RENAME comment explaining the protocol-vs-vendor split (the Testcontainers .NET library publishes its RESP-compatible test container under the `Testcontainers.Redis` identifier; the container itself can target any RESP-compatible image including Valkey). - Host port 8080 collision between SeaweedFS volume HTTP API and Keycloak. Moved SeaweedFS volume API to host port 8084 (container port 8080 unchanged); updated `infra/seaweedfs/README.md` accordingly. - ADR-0002 Decision + Consequences sections were edited in 3fd9054 in violation of CLAUDE.md's "never edit an Accepted ADR's decision section" rule. Reverted both sections to their original 2026 wording (PostgreSQL / Redis / MinIO); kept the Status amendment block and the two dated Amendment blocks at the bottom; added a note in the Status section explaining the immutability + amendment-pattern split. Security hardening (Keycloak realms) - `learnstack-api` (confidential service): `directAccessGrantsEnabled` flipped to false (Direct Access Grants = deprecated OIDC ROPC, not needed for the .NET API's standard-flow + client_credentials usage); `webOrigins` tightened to `[]` (backend-only client, CORS is handled at apps/web + APISIX). `fullScopeAllowed` kept true with an inline `_devNote_fullScopeAllowed` field explaining the dev-vs-production posture. - `learnstack-web` + `learnstack-hub-web` (public PKCE): inline dev- posture note added to each client's `description` field, pointing production realm provisioning at the explicit-scopeMappings pattern. Doc accuracy - Cost model (`08-livekit-cost-model.md`): "1 TB recording storage" was mathematically wrong against the document's own 60 MB/min assumption for 300,000 minutes. Corrected to ~18 TB + Hetzner BX21-tier €70/mo ballpark; total Hetzner monthly bumped from $650-800 to $720-870. - Standards 20 TTL guidance: split the "in-process / Valkey layer 60s" one-liner into explicit L1 (in-process `IMemoryCache` = 60s) and L2 (Dapr state → Valkey = 15-min upper bound) clauses. Same split applied in the `add-feature-key` skill's pitfall section. - Standards 05 + Standards 20: added ADR-0029 / ADR-0030 / ADR-0031 to the `Derives from:` provenance headers. - ADR-0014 component file table: corrected `dapr/components/pubsub.yaml` → `infra/dapr/components/pubsub-kafka.yaml` (+ statestore-redis.yaml + secretstore-vault.yaml), with a note that the `-redis` suffix is the Dapr provider-type convention, not vendor branding. - `architecture/29-dapr-integration.md` statestore section: added a callout that `state.redis` + `redisHost` are Dapr / RESP-protocol identifiers, not vendor markers; runtime points at the `valkey` service per ADR-0030. - `architecture/04-technical-architecture.md`: "PostgreSQL 18+" → "PostgreSQL 18.x" (major-pin policy per ADR-0031); "Valkey 7+ via Dapr State Store" → "Valkey 8.x via Dapr State Store" with citation; stale `redis` line in Local Infrastructure list → `valkey`. - ADR-0002 + ADR-0031 ADR-0023 link targets: `README.md` → `README.md#open-adr-drafts` (3 places) since ADR-0023 is a reserved draft listed in the decisions index, not a standalone file. - Stale Phase-01 status text refreshed in CLAUDE.md, README.md, `roadmap/phase-01-repository-tooling.md`, and `Standards 12 § Local Infrastructure` compose list (removed `apisix-dashboard` + `redis` references; added Valkey / SeaweedFS / Postgres 18 ADR pointers). - `local-dev-setup` skill: deployment-mode table now uses `SelfHostedOnline` / `SelfHostedAirGapped` (the current model from Standards 12) instead of the obsolete `SelfHosted`; removed the APISIX dashboard row from the dev port table. Hygiene - Kafka KRaft `CLUSTER_ID` switched from the freeform string `learnstack-dev-cluster-id-1` to a real URL-safe Base64-encoded UUID (`ofPH0p5rSlK52BxOX3qLPQ`) matching the format `kafka-storage.sh random-uuid` emits. - Markdown lint (MD040) fence-language fixes: `text` added to four fenced blocks (`Standards 06` test-pyramid Mermaid + text fallback; `architecture/09-tenant-isolation.md` storage example; `infra/dapr/README.md` sidecar topology ASCII; `infra/apisix/README.md` plugin chain ASCII). Skipped with reason - **F7 (learnstack-hub-web fullScopeAllowed=true → false + explicit scopeMappings)**: the full fix would block demo-operator from obtaining a hub-operator role claim in dev, breaking the operator portal demo flow. Production realm provisioning (separate from this dev seed JSON, per `infra/keycloak/README.md`) is the right place to apply least-privilege scope maps. The kept-true value carries an inline note pointing at that. Verification - `docker compose -f infra/compose/dev.yml config -q` exits 0. - Both Keycloak realm JSONs parse cleanly. - No `Testcontainers.Valkey` references remain outside the DO-NOT-RENAME explanatory comment. - Only Keycloak claims host port 8080; SeaweedFS volume API now on 8084. - ADR-0002 Decision section reads "PostgreSQL, Redis, MinIO" verbatim (the original); current-state guidance lives in the two dated Amendment blocks at the bottom. - All three ADR-0023 links point at `README.md#open-adr-drafts`. - Kafka CLUSTER_ID is a valid 22-char URL-safe Base64 UUID. - Markdown link sweep on every changed doc clean. Co-Authored-By: Claude Opus 4.7 (1M context) --- .claude/skills/add-feature-key/SKILL.md | 8 ++++--- .claude/skills/local-dev-setup/SKILL.md | 8 +++---- CLAUDE.md | 18 ++++++++------- README.md | 20 ++++++++++------- backend/Directory.Packages.props | 8 ++++++- .../LearnStack.Tests.Integration.csproj | 2 +- .../architecture/04-technical-architecture.md | 6 ++--- docs/architecture/08-livekit-cost-model.md | 2 +- docs/architecture/09-tenant-isolation.md | 2 +- docs/architecture/29-dapr-integration.md | 9 ++++++++ docs/decisions/0002-initial-architecture.md | 22 +++++++++++-------- docs/decisions/0014-adopt-dapr.md | 6 ++--- .../0031-postgresql-major-version.md | 4 ++-- docs/roadmap/phase-01-repository-tooling.md | 2 +- docs/standards/05-database.md | 6 +++-- docs/standards/06-testing.md | 11 ++++++++++ docs/standards/12-infrastructure.md | 13 +++++------ docs/standards/20-infrastructure-stack.md | 11 +++++++--- infra/apisix/README.md | 2 +- infra/compose/dev.yml | 7 ++++-- infra/dapr/README.md | 2 +- infra/keycloak/realms/learnstack-hub.json | 2 +- infra/keycloak/realms/learnstack.json | 9 ++++---- infra/seaweedfs/README.md | 2 +- 24 files changed, 115 insertions(+), 67 deletions(-) diff --git a/.claude/skills/add-feature-key/SKILL.md b/.claude/skills/add-feature-key/SKILL.md index fea7916..3da4af8 100644 --- a/.claude/skills/add-feature-key/SKILL.md +++ b/.claude/skills/add-feature-key/SKILL.md @@ -265,9 +265,11 @@ public async Task LimitKey_Soft_SurfaceBanner_DoesNotBlock() { ... } - **Writing a plan-projected key to `tenant_feature_flags`.** Architecture test rejects. Plan keys belong to the entitlement projection only. - **Reading the key from raw SQL.** Forbidden; use `IFeatureFlags`. -- **Hot path without Valkey cache.** Each `IsEnabledAsync` call could become - DB-bound. The Valkey layer (60s TTL, eager-invalidated by Dapr event) is - load-bearing. +- **Hot path without the cache stack.** Each `IsEnabledAsync` call could become + DB-bound. The two cache layers (L1 in-process `IMemoryCache` = 60s TTL; + L2 Dapr state → Valkey = 15-min upper bound; both eager-invalidated by the + `learnstack.hub.entitlement` Dapr event) are load-bearing — see Standards 20 + § Configuration / Eager invalidation. - **Killswitch without runbook.** The runbook is part of the deliverable. CI does not enforce its presence today; review must. - **Removing a key without a deprecation cycle.** A rename / remove follows the diff --git a/.claude/skills/local-dev-setup/SKILL.md b/.claude/skills/local-dev-setup/SKILL.md index eb5c6ed..31c69e2 100644 --- a/.claude/skills/local-dev-setup/SKILL.md +++ b/.claude/skills/local-dev-setup/SKILL.md @@ -45,7 +45,7 @@ LiveKit / Meilisearch / APISIX — the same components production uses | Docker Desktop | Yes | Required for every container. | | .NET 10 SDK | Yes | `dotnet --version` returns `10.0.x`. | | Node 20+ + pnpm | Yes | For the frontend. | -| Deployment mode | Yes | `Development` (default) / `SaaS` / `Dedicated` / `SelfHosted`. | +| Deployment mode | Yes | `Development` (default) / `SaaS` / `Dedicated` / `SelfHostedOnline` / `SelfHostedAirGapped` (per [Standards 12 § Deployment Modes](../../../docs/standards/12-infrastructure.md)). | | `.env` (gitignored) | Optional | Local overrides; `.env.example` is the source of truth. | ## Workflow @@ -111,8 +111,7 @@ The components and their default ports: | Mailhog | 8025 | Captures outbound email in dev. | | OTel Collector | 4317 (gRPC) | Local observability. | | Dapr sidecar (per service) | 3500 (HTTP) / 50001 (gRPC) | Building-block runtime. | -| APISIX | 9080 (gateway) / 9180 (admin) | API gateway. | -| APISIX dashboard (optional) | 9000 | Route inspection. | +| APISIX | 9080 (HTTP gateway) / 9443 (HTTPS) / 9091 (Prometheus metrics) | File-driven standalone (`data_plane`) per ADR-0015 — no Admin API, no dashboard companion. `apisix.yaml` is the only source of truth. | | LearnStack API | 5100 | Backend host. | | LearnStack Web (`apps/web`) | 3000 | Frontend dev server. | @@ -170,7 +169,8 @@ Edit `.env` to flip `DEPLOYMENT_MODE`: | `Development` (default) | `InProcessEventBus` + `InMemoryCacheService` + env vars for secrets. Dapr sidecar is still present but not exercised. | | `SaaS` | `DaprEventBus` (Kafka) + `DaprCacheService` (Valkey) + `DaprSecretProvider` (Vault) + `HubEntitlementProvider` pointing at the local Hub. Requires the `learnstack-hub` repo's `make dev` to be running. | | `Dedicated` | Same as `SaaS` for the composition; in practice the Hub is dedicated to one tenant. | -| `SelfHosted` | `SignedLicenseKeyEntitlementProvider` reads `.lic` from `./secrets/license.lic`; no Hub interaction. | +| `SelfHostedOnline` | `HubEntitlementProvider` against the LearnStack-hosted Hub (phone-home daily, 30-day cached-projection grace per ADR-0020). | +| `SelfHostedAirGapped` | `SignedLicenseKeyEntitlementProvider` reads `.lic` from `./secrets/license.lic`; no Hub interaction. | After changing `.env`, restart the API: diff --git a/CLAUDE.md b/CLAUDE.md index f68a10a..2020284 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,15 +21,17 @@ Self-Hosted — backed by the companion **`learnstack-hub`** repository ## What state this is in -**Phase 01 in progress.** The repository now has the .NET 10 solution -scaffold under `backend/` (core + 7 modules × 4 projects + 4 test -projects including the non-skippable `LearnStack.Tests.Architecture`), +**Phase 01 packets 1-6 shipped.** The repository now has the .NET 10 +solution scaffold under `backend/` (core + 7 modules × 4 projects + 4 +test projects including the non-skippable `LearnStack.Tests.Architecture`), the `pnpm` frontend monorepo under `frontend/` (`apps/web` Next.js App -Router + `packages/{config,ui,sdk}`), and the core local-dev compose -stack at `infra/compose/dev.yml` (Postgres / Valkey / SeaweedFS / Mailpit / -Meilisearch). The remaining Phase-01 packets — Keycloak, LiveKit, Kafka, -Vault, Dapr, APISIX, `make` targets, `.env.example`, CI, `make seed` — -land incrementally; see +Router + `packages/{config,ui,sdk}`), and the full local-dev compose +stack at `infra/compose/dev.yml` — PostgreSQL 18, Valkey, SeaweedFS, +Mailpit, Meilisearch, Keycloak (two realms), LiveKit OSS + Coturn, +Kafka + kafka-ui, Vault, Dapr sidecar + placement, APISIX in file- +driven standalone mode. The remaining Phase-01 packets (7-8) — `make` +orchestrator, `.env.example`, pre-commit hook, `e2e.yml`, GitHub Actions +CI, `make seed` — land incrementally; see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). Module-level code references in the docs (e.g. `LearnStack.Modules.Education.Application`, `ILiveClassProvider`, diff --git a/README.md b/README.md index 6648c19..f444ff4 100644 --- a/README.md +++ b/README.md @@ -17,12 +17,14 @@ Dedicated control plane, plan editor, custom-domain admin, and license-key issua ## Status -Phase 01 in progress. The repository now holds the .NET 10 solution scaffold -(7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` +Phase 01 packets 1-6 shipped. The repository now holds the .NET 10 solution +scaffold (7 modules × 4 projects + 4 test projects with `No_Source_Folder_Named_Verticals` architecture test), the `pnpm` frontend monorepo (`apps/web` + `packages/{config,ui,sdk}`), -and the core local-dev `docker-compose` stack (Postgres / Valkey / SeaweedFS / Mailpit / -Meilisearch). Keycloak, LiveKit, Kafka, Vault, Dapr, APISIX, `make` targets, and -CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). +and the full local-dev `docker-compose` stack — PostgreSQL 18, Valkey, SeaweedFS, +Mailpit, Meilisearch, Keycloak (two realms), LiveKit OSS + Coturn, Kafka + kafka-ui, +Vault, Dapr sidecar + placement, APISIX (file-driven standalone). The remaining +Phase-01 packets — `make` targets, `.env.example`, pre-commit, `e2e.yml`, GitHub +Actions CI, `make seed` — land in packets 7-8; see [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md). ## Direction At A Glance @@ -30,10 +32,12 @@ CI land in subsequent Phase-01 packets — see [docs/roadmap/phase-01-repository - **Database:** PostgreSQL 18, with Row-Level Security from day one. Tenant + **Organization** defense in depth ([ADR-0003 Amendment 1](docs/decisions/0003-tenant-isolation-defense-in-depth.md), [ADR-0017](docs/decisions/0017-tenant-organization-hierarchy.md)). -- **Cache / Pub-Sub / Secrets:** Valkey 7, Kafka, HashiCorp Vault — all accessed via - **Dapr** building blocks (`IEventBus`, `ICacheService`, `ISecretProvider`) per +- **Cache / Pub-Sub / Secrets:** Valkey 8 (RESP-protocol Linux-Foundation BSD fork + per [ADR-0030](docs/decisions/0030-redis-compatible-store-valkey.md)), Kafka, + HashiCorp Vault — all accessed via **Dapr** building blocks (`IEventBus`, + `ICacheService`, `ISecretProvider`) per [ADR-0014](docs/decisions/0014-adopt-dapr.md). -- **API Gateway:** **APISIX** in standalone YAML-reload mode per +- **API Gateway:** **APISIX** in file-driven standalone (`data_plane`) mode per [ADR-0015](docs/decisions/0015-api-gateway-apisix.md). - **Object storage:** SeaweedFS locally, S3-compatible storage in production. - **Search:** Meilisearch initially. diff --git a/backend/Directory.Packages.props b/backend/Directory.Packages.props index 8461572..b4ef912 100644 --- a/backend/Directory.Packages.props +++ b/backend/Directory.Packages.props @@ -37,7 +37,13 @@ suite only runs the WebApplicationFactory smoke test. --> - + + diff --git a/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj b/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj index 9ba1939..fa5d581 100644 --- a/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj +++ b/backend/tests/LearnStack.Tests.Integration/LearnStack.Tests.Integration.csproj @@ -25,7 +25,7 @@ - + diff --git a/docs/architecture/04-technical-architecture.md b/docs/architecture/04-technical-architecture.md index 7eba49d..0c082fb 100644 --- a/docs/architecture/04-technical-architecture.md +++ b/docs/architecture/04-technical-architecture.md @@ -7,8 +7,8 @@ | Backend runtime | .NET 10, ASP.NET Core Web API | | Language | C# | | ORM | Entity Framework Core | -| Database | PostgreSQL 18+ (shared schema + RLS isolation; ADR-0003) | -| Cache & coordination | **Valkey 7+ via Dapr State Store** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | +| Database | PostgreSQL 18.x (major pinned per [ADR-0031](../decisions/0031-postgresql-major-version.md); shared schema + RLS isolation; ADR-0003) | +| Cache & coordination | **Valkey 8.x via Dapr State Store** (Linux-Foundation BSD-3 fork of Redis 7.2.4 per [ADR-0030](../decisions/0030-redis-compatible-store-valkey.md); [29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) | | Pub/Sub | **Apache Kafka via Dapr Pub/Sub** ([29-dapr-integration.md](29-dapr-integration.md), [ADR-0014](../decisions/0014-adopt-dapr.md)) — outbox dispatch target | | Secrets | **HashiCorp Vault via Dapr Secret Store** (or env-var fallback in Dev) | | Distributed runtime | **Dapr 1.14+** sidecar pattern (pub/sub, state, secrets) | @@ -207,7 +207,7 @@ Detailed conventions: [Frontend Architecture](14-frontend-architecture.md) and [ ``` postgres -redis +valkey seaweedfs # single dev binary: master + volume + filer + S3 gateway meilisearch keycloak diff --git a/docs/architecture/08-livekit-cost-model.md b/docs/architecture/08-livekit-cost-model.md index c2a22d6..368ff84 100644 --- a/docs/architecture/08-livekit-cost-model.md +++ b/docs/architecture/08-livekit-cost-model.md @@ -222,7 +222,7 @@ flips the decision. | Mode | Estimated monthly cost | |---|---| | LiveKit Cloud Scale | $500 base + ~$120 transfer overage + $4,380 recording overage ≈ **$5,000** | -| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Valkey + TURN + 1 TB recording storage ≈ **$650–800** | +| LiveKit OSS Hetzner | 3× SFU + 2× Egress + Valkey + TURN ≈ **$650–800** compute. Recording storage at 300,000 min × 60 MB/min ≈ **18 TB** → ~€70/mo on Hetzner BX21-tier (€3.81/TB-month linear above 1 TB). **Total ≈ $720–870** | By this scale, the self-hosted advantage is ~6×. diff --git a/docs/architecture/09-tenant-isolation.md b/docs/architecture/09-tenant-isolation.md index 6de34e7..35b003c 100644 --- a/docs/architecture/09-tenant-isolation.md +++ b/docs/architecture/09-tenant-isolation.md @@ -195,7 +195,7 @@ Tenant + org isolation applies outside PostgreSQL too: ### Storage (SeaweedFS) -``` +```text tenants/{tenant_id}/organizations/{org_id}/courses/{course_id}/... ← org-scoped tenants/{tenant_id}/brand/... ← tenant-wide ``` diff --git a/docs/architecture/29-dapr-integration.md b/docs/architecture/29-dapr-integration.md index fe53a39..8de0750 100644 --- a/docs/architecture/29-dapr-integration.md +++ b/docs/architecture/29-dapr-integration.md @@ -82,6 +82,15 @@ Topics follow the convention `learnstack.{module}.{aggregate}`. Examples: ### `statestore.yaml` — Valkey state store +> The component below uses `spec.type: state.redis` and `redisHost` metadata — +> these are **Dapr provider-type / RESP-protocol identifiers**, NOT vendor +> brand markers. The actual backend is Valkey 8.x per +> [ADR-0030](../decisions/0030-redis-compatible-store-valkey.md); Valkey is +> drop-in compatible on the RESP wire protocol so the Dapr `state.redis` +> adapter consumes it unchanged. Operators read `redisHost` as "where to +> reach the RESP-compatible store"; in dev compose the value points at the +> `valkey` service (`infra/dapr/components/statestore-redis.yaml`). + ```yaml apiVersion: dapr.io/v1alpha1 kind: Component diff --git a/docs/decisions/0002-initial-architecture.md b/docs/decisions/0002-initial-architecture.md index 5fe29b2..c1921f2 100644 --- a/docs/decisions/0002-initial-architecture.md +++ b/docs/decisions/0002-initial-architecture.md @@ -3,7 +3,7 @@ ## Status Accepted with two amendments — see the bottom of this document for the dated -amendment block. Each amendment supersedes a single backend row of the +amendment blocks. Each amendment supersedes a single backend row of the original Decision section without rewriting the rest of the ADR: - **Amendment 1 (2026-05-19):** storage backend MinIO → SeaweedFS per @@ -13,12 +13,15 @@ original Decision section without rewriting the rest of the ADR: version pinned to 18.x per [ADR-0031](0031-postgresql-major-version.md). -Every other choice in this ADR — .NET 10, ASP.NET Core, EF Core, modular -monolith, Next.js — stands. +Every other choice in the Decision section below — .NET 10, ASP.NET Core, +EF Core, modular monolith, Next.js — stands. The Decision + Consequences +text is the original, immutable form per CLAUDE.md's "never edit an +Accepted ADR's decision section" rule; read it together with the +Amendment blocks at the bottom for the current backend choices. ## Decision -LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL 18.x (see Amendment 2), Valkey (see Amendment 2), SeaweedFS (see Amendment 1), and Next.js. +LearnStack starts as a modular monolith using .NET 10, ASP.NET Core, Entity Framework Core, PostgreSQL, Redis, MinIO, and Next.js. ## Context @@ -30,9 +33,9 @@ The team has stronger familiarity with .NET, so .NET 10 is preferred over Go for - The first backend implementation should use .NET 10. - EF Core should be the default ORM. -- PostgreSQL 18.x should be the primary database (Amendment 2). -- Valkey should be used for caching and distributed coordination where needed (Amendment 2). -- SeaweedFS should be used locally for S3-compatible object storage (Amendment 1). +- PostgreSQL should be the primary database. +- Redis should be used for caching and distributed coordination where needed. +- MinIO should be used locally for S3-compatible object storage. - Next.js should be used for public rendering, admin studio, and portals initially. --- @@ -73,8 +76,9 @@ pre-implementation, so the migration drag is zero): 2. **PostgreSQL major pinned to 18.x** per [ADR-0031](0031-postgresql-major-version.md). 18 is the longest- runway LTS available (EOL 2030-11), brings native `gen_uuid_v7()` - that the [ADR-0023 draft](README.md) can adopt without an extension, - and async I/O for sequential scans helps the partitioned `audit_log` + that the [ADR-0023 draft](README.md#open-adr-drafts) can adopt + without an extension, and async I/O for sequential scans helps the + partitioned `audit_log` ([ADR-0016](0016-audit-log-subsystem.md)) operator queries. RLS policy syntax + connection-string + role provisioning are unchanged from 16 / 17, so the tenant-isolation defense-in-depth pattern diff --git a/docs/decisions/0014-adopt-dapr.md b/docs/decisions/0014-adopt-dapr.md index 0303e51..890ba41 100644 --- a/docs/decisions/0014-adopt-dapr.md +++ b/docs/decisions/0014-adopt-dapr.md @@ -14,9 +14,9 @@ LearnStack adopts **Dapr** (Distributed Application Runtime) for three building | Building block | Backend (production) | Component file | |----------------|---------------------|----------------| -| Pub/Sub | Apache Kafka | `dapr/components/pubsub.yaml` | -| State store | Valkey | `dapr/components/statestore.yaml` | -| Secret store | HashiCorp Vault | `dapr/components/secretstore-vault.yaml` | +| Pub/Sub | Apache Kafka | `infra/dapr/components/pubsub-kafka.yaml` | +| State store | Valkey (RESP-protocol fork; see ADR-0030) | `infra/dapr/components/statestore-redis.yaml` (file name keeps the `-redis` suffix because `state.redis` is the Dapr provider-type identifier, not the vendor brand) | +| Secret store | HashiCorp Vault | `infra/dapr/components/secretstore-vault.yaml` | Application code interacts with Dapr **exclusively through wrapped abstractions** — `IEventBus`, `ICacheService`, `ISecretProvider` in `LearnStack.SharedKernel` — never via diff --git a/docs/decisions/0031-postgresql-major-version.md b/docs/decisions/0031-postgresql-major-version.md index 2f60b5d..8c95a29 100644 --- a/docs/decisions/0031-postgresql-major-version.md +++ b/docs/decisions/0031-postgresql-major-version.md @@ -19,7 +19,7 @@ major-version choice only; the rest of ADR-0002 stands) versus 16 LTS at 2028-11. Starting on 18 buys an extra two years of upstream patches before any forced major upgrade. - **`gen_uuid_v7()` is native in 18.** LearnStack's - [ADR-0023 (Strongly-typed ID source generator)](README.md) is a + [ADR-0023 (Strongly-typed ID source generator)](README.md#open-adr-drafts) is a pending draft considering UUIDv7 as the canonical id format (time-ordered, index-friendly). Postgres 18 ships a built-in `gen_uuid_v7()` SQL function — DB-side DEFAULT values become trivial, @@ -178,7 +178,7 @@ preview, that preview already runs on 18 — no major upgrade needed. - [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original PostgreSQL major-version row, now partially superseded. - [ADR-0003 Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) — RLS pattern unchanged across 16/17/18. - [ADR-0016 Audit Log Subsystem](0016-audit-log-subsystem.md) — partitioned `audit_log` benefits from async I/O. -- [ADR-0023 Strongly-typed ID source generator](README.md) — draft; UUIDv7 native in 18 widens the design space. +- [ADR-0023 Strongly-typed ID source generator](README.md#open-adr-drafts) — draft (no dedicated file yet — listed in the decisions index); UUIDv7 native in 18 widens the design space. - [Standards 05 — Database](../standards/05-database.md) - [Standards 12 § Database Operations](../standards/12-infrastructure.md) - PostgreSQL 18 release notes: . diff --git a/docs/roadmap/phase-01-repository-tooling.md b/docs/roadmap/phase-01-repository-tooling.md index 05347f5..0d07985 100644 --- a/docs/roadmap/phase-01-repository-tooling.md +++ b/docs/roadmap/phase-01-repository-tooling.md @@ -172,7 +172,7 @@ Docker Compose under `infra/compose/`: - **Kafka** (Dapr pub/sub backend) + kafka-ui. - **Vault** (Dapr secret store, dev mode). - **Dapr sidecar** + placement service. -- **APISIX** (standalone YAML-reload mode) + apisix-dashboard (dev only). +- **APISIX** (file-driven standalone `data_plane` mode per ADR-0015 — no etcd, no Admin API, no dashboard companion). - Optional Jaeger or Tempo (for trace inspection). - Optional `learnstack-hub` compose overlay for local Hub development (depends on the same Keycloak / Postgres / Kafka / Vault / APISIX stack). diff --git a/docs/standards/05-database.md b/docs/standards/05-database.md index d914d4f..70a97e8 100644 --- a/docs/standards/05-database.md +++ b/docs/standards/05-database.md @@ -1,13 +1,15 @@ # 05 — Database Standards **Status:** Active -**Derives from:** [ADR-0002 Initial Architecture](../decisions/0002-initial-architecture.md), +**Derives from:** [ADR-0002 Initial Architecture](../decisions/0002-initial-architecture.md) +(Amendments 1 + 2), [ADR-0003 Tenant Isolation Defense in Depth](../decisions/0003-tenant-isolation-defense-in-depth.md) (Amendment 1: Organization Scope), [ADR-0006 Events and Outbox](../decisions/0006-events-and-outbox.md) (Amendment 1: Dapr pub/sub dispatch transport), [ADR-0014 Adopt Dapr](../decisions/0014-adopt-dapr.md), -[ADR-0017 Tenant + Organization Hierarchy](../decisions/0017-tenant-organization-hierarchy.md). +[ADR-0017 Tenant + Organization Hierarchy](../decisions/0017-tenant-organization-hierarchy.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). PostgreSQL schema, EF Core, and migration conventions. diff --git a/docs/standards/06-testing.md b/docs/standards/06-testing.md index ef333e2..fbe9895 100644 --- a/docs/standards/06-testing.md +++ b/docs/standards/06-testing.md @@ -8,6 +8,9 @@ Test pyramid, conventions, and what every change must cover. ## Test Pyramid ```mermaid +--- +title: LearnStack Test Pyramid +--- flowchart TB e2e[End-to-end / Playwright
handful of golden flows] contract[Contract & API tests
OpenAPI + provider fakes] @@ -18,6 +21,14 @@ flowchart TB unit --> arch --> integration --> contract --> e2e ``` +Text fallback (for renderers without Mermaid support — pyramid base → top): + +- **Unit tests** (base layer, widest) — domain + application + UI logic. +- **Architecture tests** — module boundaries + tenant invariants. +- **Integration tests** — Testcontainers Postgres / Valkey / SeaweedFS. +- **Contract & API tests** — OpenAPI + provider fakes. +- **End-to-end / Playwright** (top, narrowest) — handful of golden flows. + We invest most at **unit + integration**. Architecture tests are zero-flake. E2E covers only what cannot be proven below. ## Backend Test Types diff --git a/docs/standards/12-infrastructure.md b/docs/standards/12-infrastructure.md index 4f01a35..0fce4cb 100644 --- a/docs/standards/12-infrastructure.md +++ b/docs/standards/12-infrastructure.md @@ -50,23 +50,22 @@ adapter table. ## Local Infrastructure (Docker Compose) ``` -postgres -redis -seaweedfs # single dev binary: master + volume + filer + S3 gateway +postgres # PostgreSQL 18.x per ADR-0031 +valkey # Linux-Foundation BSD-3 fork of Redis 7.2.4 per ADR-0030 +seaweedfs # single dev binary: master + volume + filer + S3 gateway per ADR-0029 meilisearch keycloak # two realms: learnstack + learnstack-hub livekit-server livekit-egress coturn -mailhog +mailpit otel-collector dapr-placement # Dapr building blocks dapr-sidecar-api # one sidecar per backend service -kafka + kafka-ui # Dapr pub/sub backend +kafka + kafka-ui # Dapr pub/sub backend (kafka-ui = ghcr.io/kafbat fork) vault # Dapr secrets backend (dev mode) -apisix # gateway in standalone YAML-reload mode -apisix-dashboard # optional, dev only +apisix # gateway in file-driven standalone (data_plane) mode — no etcd, no Admin API, no dashboard companion ``` - Application projects run **outside** containers during active development; the Dapr diff --git a/docs/standards/20-infrastructure-stack.md b/docs/standards/20-infrastructure-stack.md index 304f272..75ef828 100644 --- a/docs/standards/20-infrastructure-stack.md +++ b/docs/standards/20-infrastructure-stack.md @@ -5,7 +5,10 @@ [ADR-0015 API Gateway: APISIX](../decisions/0015-api-gateway-apisix.md), [ADR-0019 LearnStack Hub](../decisions/0019-learnstack-hub.md), [ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md), -[ADR-0021 Feature-Based Entitlement](../decisions/0021-feature-based-entitlement.md). +[ADR-0021 Feature-Based Entitlement](../decisions/0021-feature-based-entitlement.md), +[ADR-0029 Object Storage — SeaweedFS](../decisions/0029-object-storage-seaweedfs.md), +[ADR-0030 Redis-compatible Store — Valkey](../decisions/0030-redis-compatible-store-valkey.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). This standard defines how application code uses the foundation infrastructure introduced in the 2026-05-18 redesign: Dapr building blocks, the APISIX gateway, the Hub HTTPS @@ -191,8 +194,10 @@ Full deep dive: [15-event-and-outbox.md](../architecture/15-event-and-outbox.md) - `IFeatureFlags.IsEnabledAsync(FeatureKey)` is the only sanctioned read path. Direct SQL against `platform_entitlement_cache` outside the Tenancy module's infrastructure is forbidden (architecture test `Modules_Do_Not_Read_Entitlement_Cache_Directly`). -- Cache TTL for the in-process / Valkey layer is 60s. Eager invalidation flows from the - Dapr event; the TTL is the safety net, not the typical refresh window. +- Cache TTLs: **L1 (in-process `IMemoryCache`)** = 60s; **L2 (Dapr state → Valkey)** = + 15-minute upper bound. Eager invalidation flows from the Dapr event + (`learnstack.hub.entitlement` / `learnstack.cache.invalidation`); the TTLs are the + safety net, not the typical refresh window. - For air-gapped deployments, `SignedLicenseKeyEntitlementProvider` reads a signed `.lic` file and runs the same projection write path; the rest of the system is source-agnostic. diff --git a/infra/apisix/README.md b/infra/apisix/README.md index 4c4ab35..378f53a 100644 --- a/infra/apisix/README.md +++ b/infra/apisix/README.md @@ -31,7 +31,7 @@ appears, behind its own ADR. `infra/apisix/config.yaml` declares the universe of plugins; routes pick from it per-request order: -``` +```text real-ip → cors → openid-connect → limit-req → request-id → proxy-rewrite → upstream ↓ prometheus (response) diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index 1dec2ad..cecce6a 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -93,7 +93,7 @@ services: - "9000:8333" # S3 API gateway (host:9000 matches the prior MinIO endpoint) - "9001:8888" # Filer UI (replaces the MinIO console) - "9333:9333" # Master HTTP API - - "8080:8080" # Volume HTTP API + - "8084:8080" # Volume HTTP API (host 8084 to avoid Keycloak's 8080) volumes: - seaweedfs-data:/data # Dev-only S3 identity (matches prior MinIO credentials so callers @@ -274,7 +274,10 @@ services: KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_LOG_DIRS: /var/lib/kafka/data # Stable cluster id so KRaft does not re-format the log dir on restart. - CLUSTER_ID: learnstack-dev-cluster-id-1 + # Format: URL-safe Base64-encoded UUIDv4 (16-byte UUID → 22-char Base64 + # without padding) — the format `kafka-storage.sh random-uuid` emits. + # This value is checked in deliberately for dev reproducibility. + CLUSTER_ID: ofPH0p5rSlK52BxOX3qLPQ ports: - "9092:9092" volumes: diff --git a/infra/dapr/README.md b/infra/dapr/README.md index 5f6c4d6..7c4bdc6 100644 --- a/infra/dapr/README.md +++ b/infra/dapr/README.md @@ -20,7 +20,7 @@ new ADR. The state store's `actorStateStore` flag is therefore pinned to Dev compose runs one sidecar bound to the `learnstack-api` app id: -``` +```text ┌──────────────────────────┐ ┌──────────────────────────────────────────┐ │ dotnet run │ │ daprd │ │ → host:5080 │ │ ./daprd -app-id learnstack-api \ │ diff --git a/infra/keycloak/realms/learnstack-hub.json b/infra/keycloak/realms/learnstack-hub.json index bbe7a31..74d9ca8 100644 --- a/infra/keycloak/realms/learnstack-hub.json +++ b/infra/keycloak/realms/learnstack-hub.json @@ -60,7 +60,7 @@ { "clientId": "learnstack-hub-web", "name": "LearnStack Hub Web (public PKCE)", - "description": "learnstack-hub-web operator portal — public client, PKCE-enforced.", + "description": "learnstack-hub-web operator portal — public client, PKCE-enforced. `fullScopeAllowed: true` kept for dev simplicity so demo-operator's token carries the hub-operator realm role without explicit scopeMappings; production realm provisioning should tighten this with explicit `defaultClientScopes` listing only the hub-* roles.", "enabled": true, "publicClient": true, "standardFlowEnabled": true, diff --git a/infra/keycloak/realms/learnstack.json b/infra/keycloak/realms/learnstack.json index 220a5b2..fd88af5 100644 --- a/infra/keycloak/realms/learnstack.json +++ b/infra/keycloak/realms/learnstack.json @@ -59,30 +59,31 @@ { "clientId": "learnstack-api", "name": "LearnStack API (confidential service)", - "description": "Backend .NET API — confidential client with service-account + standard + direct grants for dev.", + "description": "Backend .NET API — confidential client. Standard flow + client_credentials (service account). Direct Access Grants (deprecated OIDC ROPC) disabled. webOrigins empty: the backend is not a browser-CORS surface (CORS is handled at apps/web + APISIX).", "enabled": true, "publicClient": false, "secret": "learnstack-api-dev-secret", "standardFlowEnabled": true, "implicitFlowEnabled": false, - "directAccessGrantsEnabled": true, + "directAccessGrantsEnabled": false, "serviceAccountsEnabled": true, "redirectUris": [ "http://localhost:5080/*", "http://localhost:3000/*" ], - "webOrigins": ["+"], + "webOrigins": [], "protocol": "openid-connect", "attributes": { "access.token.lifespan": "3600", "pkce.code.challenge.method": "S256" }, + "_devNote_fullScopeAllowed": "Kept true for dev simplicity so the service-account token carries every realm role automatically. Production realm provisioning (separate flow, not from this JSON) should set fullScopeAllowed=false + explicit scopeMappings per principle of least privilege.", "fullScopeAllowed": true }, { "clientId": "learnstack-web", "name": "LearnStack Web (public PKCE)", - "description": "Next.js apps/web — public client, PKCE-enforced, no client secret.", + "description": "Next.js apps/web — public client, PKCE-enforced, no client secret. `fullScopeAllowed: true` kept for dev simplicity so demo users' tokens carry their realm roles without explicit scopeMappings; production realm provisioning should tighten this with explicit `defaultClientScopes`.", "enabled": true, "publicClient": true, "standardFlowEnabled": true, diff --git a/infra/seaweedfs/README.md b/infra/seaweedfs/README.md index d537014..43e25fc 100644 --- a/infra/seaweedfs/README.md +++ b/infra/seaweedfs/README.md @@ -15,7 +15,7 @@ production topology splits them per ADR-0029 § Implementation Notes. | S3 API gateway | `http://localhost:9000` | The endpoint `IStorageProvider` talks to (drop-in port-map for the previous MinIO S3 endpoint) | | Filer / volume UI | `http://localhost:9001` | Bucket browser, replaces the MinIO console | | Master HTTP API | `http://localhost:9333` | Cluster topology + health (`/cluster/healthz`) | -| Volume HTTP API | `http://localhost:8080` | Internal — read / write blob ops | +| Volume HTTP API | `http://localhost:8084` | Internal — read / write blob ops. Host port 8084 (container 8080) avoids Keycloak's 8080 binding. | ## Dev credentials From 5ddc9708060886931dc08d92ded053d7bbcf7610 Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Tue, 19 May 2026 23:46:28 +0300 Subject: [PATCH 10/11] =?UTF-8?q?fix:=20PR=20#1=20review=20round=202=20?= =?UTF-8?q?=E2=80=94=20live-smoke=20blockers=20+=20standards=20drift?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-2 review surfaced four runtime blockers and a handful of doc / config drifts. Round-1 fixes already covered most static issues; these are the live-smoke and post-bump alignment items. Live-smoke blockers (compose now reaches "all healthy"): - Postgres 18 refuses any mount at /var/lib/postgresql/data — the modern layout mounts the parent /var/lib/postgresql and lets the image place data at $PGDATA. Old `postgres-data` volumes must be wiped (`docker volume rm`); compose note added. - LiveKit's built-in TURN restart-looped on `domain: localhost`. Turn it off in dev — ADR-0005 uses standalone Coturn anyway. - APISIX 3.16-debian ships neither curl nor wget. Probe via bash /dev/tcp; raise start_period to 30s. - SeaweedFS + Meilisearch healthchecks failed under `localhost` (image resolvers prefer ::1). Pin probes to 127.0.0.1. Image pin policy: - kafka-ui was the lone `:latest` in the stack. Pin to v1.5.0 (current kafbat release). Identity-realm posture: - learnstack accessTokenLifespan 3600 → 1800 to match learnstack-hub (dev fixtures internally consistent; production provisioning sets 900 anyway). - README documents the tenant-realm dev-permissive defaults (registrationAllowed=true, verifyEmail=false, 1800s) so production provisioning flips each deliberately. - Long inline client descriptions broke realm import (Keycloak CLIENT.DESCRIPTION is varchar(255)); short descriptions + central README posture section instead. Standards / docs drift: - Standards 11 / 12 / 20 + phase-02a-kernel-tenancy described APISIX mTLS as a route plugin. APISIX 3.x has no `mtls` plugin; mTLS is SSL-object config (client.ca / client.depth) plus a route-level ip-restriction. Rewritten to match the canonical stub in infra/apisix/apisix.yaml. - Standards 12 § Local Infrastructure listed livekit-egress and otel-collector as present; they are Phase 08c / Phase 11. Split into "Shipped in Phase 01" and "Deferred" lists. - Standards 12 Derives-from now cites ADR-0029 / 0030 / 0031. - ADR-0029 image sentence said `chrislusf/seaweedfs:latest (pinned)` — contradiction with the rule and with dev.yml's 3.94. Reworded. - ADR index "ADR-0002's Valkey row" was always Redis. Fixed. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../0029-object-storage-seaweedfs.md | 7 ++-- docs/decisions/README.md | 2 +- docs/roadmap/phase-02a-kernel-tenancy.md | 7 +++- docs/standards/11-security.md | 8 +++- docs/standards/12-infrastructure.md | 35 +++++++++++----- docs/standards/20-infrastructure-stack.md | 9 ++-- infra/compose/README.md | 2 +- infra/compose/dev.yml | 42 ++++++++++++------- infra/coturn/turnserver.conf | 8 ++-- infra/dapr/components/secretstore-vault.yaml | 8 ++++ infra/keycloak/README.md | 40 ++++++++++++++++++ infra/keycloak/realms/learnstack-hub.json | 2 +- infra/keycloak/realms/learnstack.json | 9 ++-- infra/livekit/livekit.yaml | 23 ++++++---- 14 files changed, 149 insertions(+), 53 deletions(-) diff --git a/docs/decisions/0029-object-storage-seaweedfs.md b/docs/decisions/0029-object-storage-seaweedfs.md index 2c2895d..2dd7120 100644 --- a/docs/decisions/0029-object-storage-seaweedfs.md +++ b/docs/decisions/0029-object-storage-seaweedfs.md @@ -73,9 +73,10 @@ SaaS / Dedicated / SelfHosted). The S3 gateway exposes the same S3 API surface the MinIO-based adapter relied on, so adapter code remains unchanged in signature. -Image: `chrislusf/seaweedfs:latest` (pinned to a specific tag per -[Standards 12 § Image Conventions](../standards/12-infrastructure.md); -the dev compose pins the current stable tag). +Image: `chrislusf/seaweedfs:3.94` in dev (the current pinned tag in +`infra/compose/dev.yml`); every environment carries an explicit tag per +[Standards 12 § Image Conventions](../standards/12-infrastructure.md) — +no environment uses `:latest`. This ADR **supersedes the storage choice in ADR-0002 only** — the rest of ADR-0002 (Postgres, Valkey, modular monolith) is unchanged. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index fc56028..113f3f0 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -38,7 +38,7 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | | 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | -| 0030 | [Valkey-compatible Store — Valkey](0030-redis-compatible-store-valkey.md) | Valkey (Linux Foundation, BSD-3-Clause) for the cache + Dapr state-store backend; RESP-protocol drop-in; partially supersedes ADR-0002's Valkey row | +| 0030 | [Redis-compatible Store — Valkey](0030-redis-compatible-store-valkey.md) | Valkey (Linux Foundation, BSD-3-Clause) for the cache + Dapr state-store backend; RESP-protocol drop-in; partially supersedes ADR-0002's Redis row | | 0031 | [PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) | Pin primary RDBMS major version to PostgreSQL 18; native `gen_uuid_v7()` + async I/O + longest LTS runway; partially supersedes ADR-0002's PostgreSQL row | ## Superseded ADRs diff --git a/docs/roadmap/phase-02a-kernel-tenancy.md b/docs/roadmap/phase-02a-kernel-tenancy.md index 5f23284..8374300 100644 --- a/docs/roadmap/phase-02a-kernel-tenancy.md +++ b/docs/roadmap/phase-02a-kernel-tenancy.md @@ -74,8 +74,11 @@ Per [ADR-0015](../decisions/0015-api-gateway-apisix.md): - APISIX runs in standalone YAML-reload mode in dev `docker-compose.yml`. - `infra/apisix/config.yaml` ships with the plugin chain wired: `cors` → `jwt-auth` → `limit-req` → `proxy-rewrite` → `prometheus`. -- A second route set guarded by `mtls` for the future `/api/internal/*` endpoints - (the endpoints themselves arrive in 02c but the gateway slot is reserved Day 1). +- A second route set bound to a dedicated SSL object (mTLS in APISIX is SSL-object + config — `client.ca` / `client.depth` — not a route plugin) plus an `ip-restriction` + on the Hub egress range, reserved Day 1 for the future `/api/internal/*` endpoints + (the endpoints themselves arrive in 02c). The commented stub at the bottom of + `infra/apisix/apisix.yaml` documents the canonical shape. ### Tenancy Schema Foundations diff --git a/docs/standards/11-security.md b/docs/standards/11-security.md index eb3f0e3..da8f25b 100644 --- a/docs/standards/11-security.md +++ b/docs/standards/11-security.md @@ -127,8 +127,12 @@ layer, not the sole control — the API re-verifies everything. - `cors` plugin handles preflight; authenticated cross-origin traffic is allow-listed per environment. - `limit-req` / `limit-count` plugins enforce the rate-limit policy below. -- `mtls` plugin guards the `/api/internal/*` route set; the client certificate must be - signed by the LearnStack-internal CA. +- The `/api/internal/*` route set is gated by **mTLS configured on the APISIX SSL + object** (`client.ca` + `client.depth` per APISIX 3.x SSL-config; mTLS is not a + route plugin) plus an `ip-restriction` (and, when applicable, `consumer-restriction`) + route plugin that only admits the documented Hub egress; the client certificate + must be signed by the LearnStack-internal CA. The route-level pattern is shown in + the commented `/api/internal/*` stub in `infra/apisix/apisix.yaml`. - Gateway config lives in `infra/apisix/` as YAML, version-controlled. No live edits. Direct ingress to backend pods (bypassing APISIX) is blocked at the network policy diff --git a/docs/standards/12-infrastructure.md b/docs/standards/12-infrastructure.md index 0fce4cb..efa5173 100644 --- a/docs/standards/12-infrastructure.md +++ b/docs/standards/12-infrastructure.md @@ -6,7 +6,10 @@ [ADR-0014 Adopt Dapr](../decisions/0014-adopt-dapr.md), [ADR-0015 API Gateway: APISIX](../decisions/0015-api-gateway-apisix.md), [ADR-0019 LearnStack Hub](../decisions/0019-learnstack-hub.md), -[ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md). +[ADR-0020 Triple Deployment + Hybrid License](../decisions/0020-triple-deployment-hybrid-license.md), +[ADR-0029 Object Storage — SeaweedFS](../decisions/0029-object-storage-seaweedfs.md), +[ADR-0030 Redis-Compatible Store — Valkey](../decisions/0030-redis-compatible-store-valkey.md), +[ADR-0031 PostgreSQL — Start on 18.x](../decisions/0031-postgresql-major-version.md). How LearnStack is built, packaged, deployed, configured, and observed at the infrastructure level. The application-code rules for the foundation building blocks @@ -49,25 +52,32 @@ adapter table. ## Local Infrastructure (Docker Compose) +Shipped in Phase 01 packets 1-6 (`infra/compose/dev.yml`): + ``` postgres # PostgreSQL 18.x per ADR-0031 valkey # Linux-Foundation BSD-3 fork of Redis 7.2.4 per ADR-0030 seaweedfs # single dev binary: master + volume + filer + S3 gateway per ADR-0029 meilisearch -keycloak # two realms: learnstack + learnstack-hub -livekit-server -livekit-egress -coturn mailpit -otel-collector - +keycloak # two realms: learnstack + learnstack-hub +livekit # SFU (livekit-server image) +coturn # TURN/STUN +kafka # KRaft mode (no ZooKeeper) — Dapr pub/sub backend +kafka-ui # ghcr.io/kafbat fork (dev only) +vault # Dapr secrets backend (dev mode) dapr-placement # Dapr building blocks dapr-sidecar-api # one sidecar per backend service -kafka + kafka-ui # Dapr pub/sub backend (kafka-ui = ghcr.io/kafbat fork) -vault # Dapr secrets backend (dev mode) apisix # gateway in file-driven standalone (data_plane) mode — no etcd, no Admin API, no dashboard companion ``` +Deferred — added by a later phase, not in the Phase 01 stack: + +``` +livekit-egress # Phase 08c (recording / consent / cost model) +otel-collector # Phase 11 (Production hardening — observability stack) +``` + - Application projects run **outside** containers during active development; the Dapr sidecar still runs alongside via `dapr run` or compose. - CI runs the same image tags as developers. @@ -173,8 +183,11 @@ See [10-observability.md](10-observability.md). - TLS everywhere; HTTP → HTTPS redirect at the edge. - **APISIX is the only tenant-facing ingress** ([ADR-0015](../decisions/0015-api-gateway-apisix.md)). Direct ingress to backend pods is blocked by network policy. -- The `/api/internal/*` route set is reachable only through a separate APISIX - route guarded by the `mtls` plugin, with the LearnStack-internal CA pinned. +- The `/api/internal/*` route set is reachable only through an APISIX route bound + to an SSL object that pins the LearnStack-internal CA via `client.ca` / + `client.depth` (mTLS in APISIX is SSL-object config, not a route plugin), plus a + route-level `ip-restriction` constraint on the Hub egress range. See the commented + `/api/internal/*` stub in `infra/apisix/apisix.yaml` for the canonical shape. - Strict ingress rules; only documented ports open. - Private VPC for backend services; database, Valkey, Kafka, Vault not on public internet. diff --git a/docs/standards/20-infrastructure-stack.md b/docs/standards/20-infrastructure-stack.md index 75ef828..3975515 100644 --- a/docs/standards/20-infrastructure-stack.md +++ b/docs/standards/20-infrastructure-stack.md @@ -145,9 +145,12 @@ Rules: 4. `proxy-rewrite` / `request-id` (correlation id injection) 5. `prometheus` (metrics export) - Hub-facing internal routes (`/api/internal/*`) live under a **separate APISIX - instance** (or a separate `route` set with `mtls` plugin) and require the mTLS client - certificate signed by the LearnStack-internal CA per - [ADR-0019](../decisions/0019-learnstack-hub.md). + instance** (or a separate route set bound to a dedicated SSL object that pins + `client.ca` to the LearnStack-internal CA — mTLS in APISIX is SSL-object config, + not a route plugin) plus a route-level `ip-restriction` for the Hub egress range; + the client certificate must be signed by that CA per + [ADR-0019](../decisions/0019-learnstack-hub.md). The commented `/api/internal/*` + stub in `infra/apisix/apisix.yaml` documents the canonical shape. - Gateway config lives in `infra/apisix/` as version-controlled YAML. Hot-reload via `apisix reload` after a config change; no in-place edit of running configs. diff --git a/infra/compose/README.md b/infra/compose/README.md index 18a6fe0..2965e44 100644 --- a/infra/compose/README.md +++ b/infra/compose/README.md @@ -55,7 +55,7 @@ recording / consent / cost-tracking story. | Service | Image | Local endpoint | Default credentials | |---------|-------|----------------|---------------------| | Kafka (KRaft) | `confluentinc/cp-kafka:8.2.1` | `localhost:9092` (in-cluster only — see note below) | none (`PLAINTEXT`, `authType: none`) | -| kafka-ui | `ghcr.io/kafbat/kafka-ui:latest` | `localhost:8081` | open UI (dev only) | +| kafka-ui | `ghcr.io/kafbat/kafka-ui:v1.5.0` | `localhost:8081` | open UI (dev only) | | Vault | `hashicorp/vault:1.21.4` | `localhost:8200` | root token `learnstack-dev-root-token` | | Dapr placement | `daprio/placement:1.17.7` | `localhost:50005` | — | | Dapr sidecar (api) | `daprio/daprd:1.17.7` | `localhost:3500` (HTTP), `localhost:50001` (gRPC) | — | diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index cecce6a..7126945 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -38,7 +38,13 @@ services: ports: - "5432:5432" volumes: - - postgres-data:/var/lib/postgresql/data + # Postgres 18+ refuses any mount at `/var/lib/postgresql/data` — the + # entrypoint pre-check treats that path's existence (even when empty) + # as a legacy 16/17 layout and aborts (see docker-library/postgres + # PR #1259). The modern layout mounts the parent `/var/lib/postgresql` + # and lets the image place data at `/var/lib/postgresql//docker` + # under PGDATA's default, which is `pg_upgrade --link`-compatible. + - postgres-data:/var/lib/postgresql # First-boot init scripts (idempotent SQL). Creates the `keycloak` # database the Keycloak service uses; see the "Identity" section of # ../compose/README.md for the re-seed procedure. @@ -100,9 +106,12 @@ services: # don't rebuild env vars). Production loads identities from Vault. - ../seaweedfs/s3-identities.json:/etc/s3-identities.json:ro healthcheck: - # The master HTTP API exposes /cluster/healthz; alpine-based - # SeaweedFS image ships wget. - test: ["CMD", "wget", "-qO-", "http://localhost:9333/cluster/healthz"] + # `127.0.0.1` (not `localhost`): SeaweedFS image's wget resolves + # `localhost` to `::1` first and does not fall back to IPv4, so a + # `localhost` probe gets connection-refused even though the master + # listens on 127.0.0.1:9333 + the container IP. `/cluster/healthz` + # is the master HTTP API readiness endpoint (SeaweedFS 3.x). + test: ["CMD-SHELL", "wget --spider -q -T 3 -t 1 http://127.0.0.1:9333/cluster/healthz"] interval: 10s timeout: 5s retries: 10 @@ -143,7 +152,10 @@ services: volumes: - meilisearch-data:/meili_data healthcheck: - test: ["CMD", "wget", "-qO-", "http://localhost:7700/health"] + # `127.0.0.1` (not `localhost`): the Meilisearch image resolves `localhost` + # to `::1` while the server listens only on IPv4 0.0.0.0:7700, so a + # `localhost` probe gets connection-refused even when the service is up. + test: ["CMD", "wget", "-qO-", "http://127.0.0.1:7700/health"] interval: 10s timeout: 5s retries: 10 @@ -167,7 +179,6 @@ services: KC_DB_USERNAME: learnstack KC_DB_PASSWORD: learnstack KC_HOSTNAME_STRICT: "false" - KC_HOSTNAME_STRICT_HTTPS: "false" KC_HTTP_ENABLED: "true" KC_HEALTH_ENABLED: "true" KC_METRICS_ENABLED: "true" @@ -291,9 +302,11 @@ services: # Kafka UI — using the active community fork `kafbat/kafka-ui` because the # original `provectuslabs/kafka-ui` was abandoned (no release since - # 2024-04). The fork preserves the same env-var contract. + # 2024-04). The fork preserves the same env-var contract. Pinned per + # Standards 12 § Image Conventions (every dev image carries an explicit + # tag; bump deliberately, never let `:latest` drift). kafka-ui: - image: ghcr.io/kafbat/kafka-ui:latest + image: ghcr.io/kafbat/kafka-ui:v1.5.0 container_name: learnstack-kafka-ui restart: unless-stopped environment: @@ -414,15 +427,16 @@ services: - "9443:9443" # HTTPS gateway (no cert in dev) - "9091:9091" # Prometheus metrics healthcheck: - # The debian-based APISIX image ships curl; we hit the gateway port - # (a 404 is fine — it confirms nginx is listening). A 200 from a real - # route would require the .NET API host to be up, which is out of - # scope for this compose file. - test: ["CMD-SHELL", "curl -fsS -o /dev/null -w '%{http_code}' http://localhost:9080 | grep -E '^(200|404)$' || exit 1"] + # apache/apisix:3.16.0-debian ships NEITHER curl NOR wget (verified by + # inspecting the image). It does ship bash. The /dev/tcp pseudo-device + # is a bash builtin that opens a TCP connection — perfect lightweight + # listener probe. `127.0.0.1` (not `localhost`) avoids the IPv6 `::1` + # detour for the same reason as the seaweedfs / meilisearch probes. + test: ["CMD", "bash", "-c", "echo > /dev/tcp/127.0.0.1/9080"] interval: 10s timeout: 5s retries: 10 - start_period: 15s + start_period: 30s volumes: postgres-data: diff --git a/infra/coturn/turnserver.conf b/infra/coturn/turnserver.conf index e18f7ea..cdeeeda 100644 --- a/infra/coturn/turnserver.conf +++ b/infra/coturn/turnserver.conf @@ -29,9 +29,11 @@ user=devuser:devsecret # Network hygiene. no-multicast-peers mobility -no-cli -no-tlsv1 -no-tlsv1_1 +# Notes for Coturn 4.6+: +# - `no-cli` removed: CLI is off by default; enable explicitly with cli-port+cli-password. +# - `no-tlsv1` / `no-tlsv1_1` removed: TLS 1.0 / 1.1 are already disabled by default +# (and OpenSSL deprecates them platform-wide); listing them only triggers a config +# parse warning. Coturn 4.11 negotiates TLS 1.2 / 1.3 only. # Logging to stdout for `docker compose logs coturn`. log-file=stdout diff --git a/infra/dapr/components/secretstore-vault.yaml b/infra/dapr/components/secretstore-vault.yaml index e7bb549..c72489a 100644 --- a/infra/dapr/components/secretstore-vault.yaml +++ b/infra/dapr/components/secretstore-vault.yaml @@ -6,6 +6,14 @@ # Dev-only token. NEVER deploy a config carrying `vaultToken` literally — # production wires the auth method via Vault's Dapr component metadata # patterns documented at https://docs.dapr.io/. +# +# KV ENGINE NOTE (Phase 02b): Vault -dev mounts `secret/` as KV v2; this +# component reads from that mount with `vaultKVPrefix: secret`. Some Dapr +# Vault-component versions need an explicit `engineType: kv-v2` metadata +# entry to detect v2's `/data/` indirection. Validate `GET +# /v1.0/secrets/secretstore/` against a known dev secret before the +# first ISecretProvider integration test in Phase 02b; if it returns 404 +# while the secret exists, add `engineType: kv-v2` here. apiVersion: dapr.io/v1alpha1 kind: Component diff --git a/infra/keycloak/README.md b/infra/keycloak/README.md index d7014ef..63b01b4 100644 --- a/infra/keycloak/README.md +++ b/infra/keycloak/README.md @@ -43,6 +43,46 @@ The realm separation is a **hard architectural invariant** per The Phase 02b OIDC integration enforces this in code; the dev seed mirrors it in data (no shared users, no shared client IDs). +## Dev-only client posture + +The realm JSONs keep `fullScopeAllowed: true` on every client for dev +simplicity — service-account tokens (for `learnstack-api`) and user tokens +(for `learnstack-web` / `learnstack-hub-web`) carry every realm role +automatically, so demo flows work without per-client `defaultClientScopes` +provisioning. **Production realm provisioning** (Terraform / +keycloak-config-cli, not these JSONs) flips this: + +| Client | Type | Dev | Production target | +|--------|------|-----|-------------------| +| `learnstack-api` | confidential, service-account | `fullScopeAllowed: true`, `directAccessGrantsEnabled: false`, `webOrigins: []` | `fullScopeAllowed: false` + explicit `scopeMappings` (least-privilege) | +| `learnstack-web` | public, PKCE | `fullScopeAllowed: true` | `fullScopeAllowed: false` + explicit `defaultClientScopes` listing the tenant-* roles only | +| `learnstack-hub-web` | public, PKCE | `fullScopeAllowed: true` | `fullScopeAllowed: false` + explicit `defaultClientScopes` listing the hub-* roles only | + +These notes used to sit inline in each client's `description` field, but +Keycloak's `CLIENT.DESCRIPTION` column is `varchar(255)` and the long +inline notes broke realm import. The descriptions in the JSON now point +back to this section. + +## Dev-only realm posture (tenant realm) + +The `learnstack` realm JSON carries permissive defaults so the demo flows +work without operator intervention. Each of these defaults is **flipped** +by the production realm-provisioning flow (Terraform / keycloak-config-cli), +not by editing the dev JSON: + +| Attribute | Dev (`learnstack.json`) | Production target | Why dev value | +|-----------|------------------------|-------------------|---------------| +| `registrationAllowed` | `true` | `false` (invite-only) | Lets reviewers exercise the tenant sign-up flow against `localhost:3000` without seeding a user first. | +| `verifyEmail` | `false` | `true` | Dev SMTP is Mailpit; flipping verification on would gate every demo on visiting `localhost:8025`. | +| `accessTokenLifespan` | `1800` (matches `learnstack-hub`) | `900` recommended (≤ 15 min, refresh-token-driven) | Halves the round-trip churn of debug sessions; still short enough to surface refresh bugs. | +| `bruteForceProtected` | `true` (same as prod) | `true` | No dev relaxation. | + +The `learnstack-hub` realm is **already** at the stricter posture +(`registrationAllowed: false`, `verifyEmail: true`, `accessTokenLifespan: 1800`) +because operators are seeded out-of-band per +[ADR-0019](../../docs/decisions/0019-learnstack-hub.md). The tenant-realm +fields above are the only intentional dev/prod deltas. + ## MFA - `learnstack-hub` realm: `CONFIGURE_TOTP` declared as a required action so the diff --git a/infra/keycloak/realms/learnstack-hub.json b/infra/keycloak/realms/learnstack-hub.json index 74d9ca8..6a2d4d2 100644 --- a/infra/keycloak/realms/learnstack-hub.json +++ b/infra/keycloak/realms/learnstack-hub.json @@ -60,7 +60,7 @@ { "clientId": "learnstack-hub-web", "name": "LearnStack Hub Web (public PKCE)", - "description": "learnstack-hub-web operator portal — public client, PKCE-enforced. `fullScopeAllowed: true` kept for dev simplicity so demo-operator's token carries the hub-operator realm role without explicit scopeMappings; production realm provisioning should tighten this with explicit `defaultClientScopes` listing only the hub-* roles.", + "description": "Hub operator portal. Public client, PKCE-enforced. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", "enabled": true, "publicClient": true, "standardFlowEnabled": true, diff --git a/infra/keycloak/realms/learnstack.json b/infra/keycloak/realms/learnstack.json index fd88af5..1a4f474 100644 --- a/infra/keycloak/realms/learnstack.json +++ b/infra/keycloak/realms/learnstack.json @@ -19,7 +19,7 @@ "quickLoginCheckMilliSeconds": 1000, "maxDeltaTimeSeconds": 43200, "failureFactor": 30, - "accessTokenLifespan": 3600, + "accessTokenLifespan": 1800, "accessTokenLifespanForImplicitFlow": 900, "ssoSessionIdleTimeout": 1800, "ssoSessionMaxLifespan": 36000, @@ -59,7 +59,7 @@ { "clientId": "learnstack-api", "name": "LearnStack API (confidential service)", - "description": "Backend .NET API — confidential client. Standard flow + client_credentials (service account). Direct Access Grants (deprecated OIDC ROPC) disabled. webOrigins empty: the backend is not a browser-CORS surface (CORS is handled at apps/web + APISIX).", + "description": "Backend .NET API. Confidential client; standard flow + service-account. ROPC disabled, webOrigins empty. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", "enabled": true, "publicClient": false, "secret": "learnstack-api-dev-secret", @@ -74,16 +74,15 @@ "webOrigins": [], "protocol": "openid-connect", "attributes": { - "access.token.lifespan": "3600", + "access.token.lifespan": "1800", "pkce.code.challenge.method": "S256" }, - "_devNote_fullScopeAllowed": "Kept true for dev simplicity so the service-account token carries every realm role automatically. Production realm provisioning (separate flow, not from this JSON) should set fullScopeAllowed=false + explicit scopeMappings per principle of least privilege.", "fullScopeAllowed": true }, { "clientId": "learnstack-web", "name": "LearnStack Web (public PKCE)", - "description": "Next.js apps/web — public client, PKCE-enforced, no client secret. `fullScopeAllowed: true` kept for dev simplicity so demo users' tokens carry their realm roles without explicit scopeMappings; production realm provisioning should tighten this with explicit `defaultClientScopes`.", + "description": "Next.js apps/web. Public client, PKCE-enforced, no secret. Dev posture (fullScopeAllowed=true) documented in infra/keycloak/README.md.", "enabled": true, "publicClient": true, "standardFlowEnabled": true, diff --git a/infra/livekit/livekit.yaml b/infra/livekit/livekit.yaml index e4f22ed..4227e53 100644 --- a/infra/livekit/livekit.yaml +++ b/infra/livekit/livekit.yaml @@ -25,13 +25,22 @@ keys: devkey: devsecret-32-byte-min-length-padding-xyz turn: - enabled: true - domain: localhost - tls_port: 7882 - udp_port: 3478 - # No TLS cert in dev — LiveKit OSS supports plain TURN listeners on the - # configured ports without a cert pair. Production wires Let's Encrypt - # through the same provider adapter family APISIX uses (ADR-0022). + # LiveKit's built-in TURN is OFF in dev — ADR-0005 uses standalone Coturn + # (see the `coturn` service in dev.yml) for STUN/TURN. Built-in TURN also + # rejects `domain: localhost` ("TURN domain is not correct"), which would + # otherwise restart-loop the container. Production keeps the standalone- + # Coturn split for the same reason: rotating credentials and TLS termination + # live on the TURN side, not in the SFU. + enabled: false + +# ICE servers (Phase 08c wiring): The `coturn` service runs at host:3478 but +# THIS file does not yet declare `rtc.turn_servers` — meaning LiveKit room +# tokens issued from a `Room.CreateAt`-flavoured call carry no TURN reference +# and browser clients fall back to direct ICE only. The wiring lands with the +# `ILiveClassProvider` LiveKit adapter in Phase 08c (per ADR-0005), which +# issues per-session credentials via the shared-secret pattern; until then, +# dev classroom flows that need symmetric-NAT relay must be tested with +# `--node-ip` set to the workstation's LAN IP, not 127.0.0.1. room: auto_create: true From ec882350470b036c1a2a5fe6a2e8a694d6a5b7fe Mon Sep 17 00:00:00 2001 From: Cemil ILIK Date: Wed, 20 May 2026 00:34:22 +0300 Subject: [PATCH 11/11] feat(infra): enhance healthchecks and add service labels for OrbStack routing --- infra/compose/dev.yml | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/infra/compose/dev.yml b/infra/compose/dev.yml index 7126945..8b556b7 100644 --- a/infra/compose/dev.yml +++ b/infra/compose/dev.yml @@ -50,10 +50,15 @@ services: # ../compose/README.md for the re-seed procedure. - ./postgres-init:/docker-entrypoint-initdb.d:ro healthcheck: + # `start_period: 20s` gives the first boot enough room for initdb + + # the `postgres-init/*.sql` scripts to complete before pg_isready + # starts counting failures against the retry budget. Re-boots + # (data already in the volume) skip this — Postgres is ready in <2s. test: ["CMD-SHELL", "pg_isready -U learnstack -d learnstack"] interval: 5s timeout: 5s retries: 10 + start_period: 20s # Valkey replaces Redis per ADR-0030. RESP-protocol drop-in (Dapr # `state.redis` component, StackExchange.Redis library, ICacheService @@ -85,6 +90,12 @@ services: image: chrislusf/seaweedfs:3.94 container_name: learnstack-seaweedfs restart: unless-stopped + labels: + # OrbStack hint: route `seaweedfs.learnstack-dev.orb.local` to the Filer + # UI (8888) rather than picking the lowest EXPOSE'd port (8080 = Volume + # HTTP API root, which returns 400). Use explicit `:9333` / `:8333` for + # master / S3 when needed. + dev.orbstack.http-port: "8888" command: - server - -dir=/data @@ -120,6 +131,10 @@ services: image: axllent/mailpit:v1.29.7 container_name: learnstack-mailpit restart: unless-stopped + labels: + # OrbStack hint: route `mailpit.learnstack-dev.orb.local` to the Web UI + # (8025) rather than the SMTP listener (1025), which is not HTTP. + dev.orbstack.http-port: "8025" ports: - "1025:1025" # SMTP - "8025:8025" # web UI @@ -192,6 +207,13 @@ services: depends_on: postgres: condition: service_healthy + # Realm SMTP server points at `mailpit:1025`. If Mailpit hasn't bound + # the SMTP listener yet when Keycloak sends its first email (password + # reset, verification), the send silently fails — TCP connect refused + # bubbles up as a generic SMTP timeout. Gating Keycloak start on + # mailpit-healthy closes that window. + mailpit: + condition: service_healthy healthcheck: # The Keycloak 26 image is UBI-9-minimal based + openjdk-21-runtime — # neither curl nor wget is in the image. Bash IS available, so we use @@ -235,8 +257,11 @@ services: healthcheck: # `--spider` issues a HEAD without downloading the body; tolerates # LiveKit's WebSocket-upgrade contract changing across minor versions - # since we only care that the listener is bound. - test: ["CMD", "wget", "--spider", "-q", "http://localhost:7880"] + # since we only care that the listener is bound. `127.0.0.1` (not + # `localhost`) for consistency with seaweedfs / meilisearch probes — + # the IPv6 `::1` fallback failure has bitten us on other Alpine-based + # minimal images, and the LiveKit image is built on the same family. + test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:7880"] interval: 10s timeout: 5s retries: 10 @@ -318,6 +343,16 @@ services: depends_on: kafka: condition: service_healthy + healthcheck: + # Spring Boot Actuator (`/actuator/health` returns 200 + `{"status":"UP"}` + # when the Kafka client has connected). Spring Boot startup with JDK 21 + # + Netty native init + Kafka client discovery is slow — `start_period` + # buys the boot 30s before the healthcheck starts counting failures. + test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:8080/actuator/health"] + interval: 10s + timeout: 5s + retries: 12 + start_period: 30s # ---- Secrets (Phase 01 packet 6) ----------------------------------------- # HashiCorp Vault in -dev mode. Production runs Vault in HA mode with