feat(cli): cache the shadow database baseline as a PGDATA snapshot (CLI-2191) by avallete · Pull Request #6184 · supabase/cli

avallete · 2026-08-13T15:40:16Z

db diff / db pull / declarative sync rebuild the shadow's platform baseline (initdb + init SQL + the PG15+ one-shot realtime/storage/auth jobs, ~15s) on every run, before user migrations even start. This PR makes that a once-per-config cost. Measured on a default-services PG17 project: ~3.3s warm provision vs ~15.5s cold.

The cache is opt-in for soak (SUPABASE_SHADOW_CACHE=1 / true; unset is off). After soak it can flip to default-on.

What changed

Readiness gate. The shadow's docker healthcheck cannot report before t+10s (10s interval, no start_period), so legacyWaitForShadowReady polls container state plus a short authenticated connect (500ms spacing, 2s connect timeout). Same timeout budget, error shape, and log dump. The healthcheck config and the long-running db container's wait are untouched.
exec'd postgres entrypoint. PID 1 was sh, which swallowed SIGTERM, so every docker stop burned the 10s grace period. All three entrypoint variants now exec; stops take ~1s (also benefits supabase stop).
Baseline snapshot cache (off unless SUPABASE_SHADOW_CACHE is set). When enabled, the cold path stops the shadow after the platform baseline and streams PGDATA to ~/.supabase/cache/shadow-baseline/shadow-baseline-<key>.tar (~90MB; SUPABASE_HOME overrides the root). A matching later run unpacks that tar into a fresh container before docker start — initdb and the baseline are skipped, migrations apply directly.

Cache contract

Pristine by construction. Every run gets a new container restored from the baseline's exact bytes. No reset protocol, metadata, lock file, or kept container — concurrency is the atomic rename.
Artifact is a tar stream through docker cp -. The directory form of docker cp resets ownership to root and Postgres refuses the data dir.
Shared across worktrees with the same settings. Retention is LRU keep-3 + 2-day mtime TTL; the current key is never evicted; abandoned .partial files older than 5 minutes are swept. Warm hits refresh mtime.
The key hashes every input baked into the cluster (images including enabled-service job tags, jwtSecret, rootKey, db password/settings, jwtExpiry, auto_expose, roles.sql, vault names+values, shadowPort, major_version, JWKS when realtime is on PG≥15, effective webhooks/pg_net policy).
OrioleDB and PG ≤ 14 are cache-ineligible. sync --no-cache bypasses the cache for that invocation.
Cold export drops --rm (Docker destroys an AutoRemove container on docker stop). Release still removes every shadow; a SIGKILLed CLI leaves at most one stopped project-labeled container, swept by supabase stop.
A cache miss or anomaly never fails the run, except when the shadow does not come back after a cold export. Warm anomalies delete the suspect tar and cold-provision; a failed export warns and continues uncached.

Still uncached: db diff --use-pgadmin, db pull --declarative's raw shadow, migration squash.

…lta-next # Conflicts: # apps/cli-go/cmd/db.go # apps/cli/docs/go-cli-porting-status.md # apps/cli/src/legacy/commands/db/diff/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/diff/diff.handler.ts # apps/cli/src/legacy/commands/db/diff/diff.integration.test.ts # apps/cli/src/legacy/commands/db/diff/diff.layers.ts # apps/cli/src/legacy/commands/db/pull/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/pull/pull.handler.ts # apps/cli/src/legacy/commands/db/pull/pull.integration.test.ts # apps/cli/src/legacy/commands/db/pull/pull.layers.ts # apps/cli/src/legacy/commands/db/push/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts # apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.ts # apps/cli/src/legacy/commands/db/schema/declarative/generate/generate.handler.ts # apps/cli/src/legacy/commands/db/schema/declarative/generate/generate.integration.test.ts # apps/cli/src/legacy/commands/db/schema/declarative/sync/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/schema/declarative/sync/sync.handler.ts # apps/cli/src/legacy/commands/db/schema/declarative/sync/sync.integration.test.ts # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.seam.layer.ts # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.seam.service.ts # apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts # apps/cli/src/legacy/shared/legacy-db-connection.sql-pg.layer.ts # apps/cli/src/legacy/shared/legacy-http-errors.ts # apps/cli/src/legacy/shared/legacy-migration-apply.ts # apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts # apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts # packages/api/src/effect.ts # packages/api/src/internal/client.ts # packages/api/src/internal/client.unit.test.ts

…lta-next # Conflicts: # apps/cli/src/legacy/commands/db/diff/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/diff/diff.handler.ts # apps/cli/src/legacy/commands/db/diff/diff.integration.test.ts # apps/cli/src/legacy/commands/db/pull/SIDE_EFFECTS.md # apps/cli/src/legacy/commands/db/shared/legacy-shadow-source.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:5ca12af032

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

One runSupabaseLive golden path: init + minimal start, then db diff --local --use-pg-delta twice against the same SUPABASE_HOME — the cold run must export the tar (shadow-debug: baseline-export), the warm run must restore it (baseline-restore, refreshed mtime, identical stdout). Replaces the in-process legacyAcquireShadowDatabase calls, which could stay green while the command wiring or env propagation broke; mechanics coverage lives in the integration suite. Shadow port comes from SUPABASE_DB_SHADOW_PORT with a bounded retry on bind conflicts — true reservation is impossible since Docker must bind the port itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:4ba00ac249

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…n's pid Eight candidates from a pid-seeded base in the IANA dynamic range replace the fixed 54987/54988 pair, so independently concurrent runs start from different bases and an occupied port costs one retry step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:9d277539bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

avallete

We should have SUPABASE_SHADOW_CACHE false by default.

So we can have a soaking / testing / opt-in period before releasing to all consumers.

Gate SUPABASE_SHADOW_CACHE through the viper helper, tighten tar retention, drop the one-off debug channel, and reuse existing helpers.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:7fd00b925a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

The platform image still installs pgjwt, so RESTRICT drop of pgcrypto fails during CLI-owned shadow prep.

…erge-conflicts-848c2e # Conflicts: # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:bb21b82611

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

The next engine ignores schema_paths when building its migrations baseline, so the declared-schema drift diffed to nothing. Also fix the invalid 'analytics' exclude (logflare). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:3c8ac90d5e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- Revalidate roles.sql before publishing a cold baseline tar; a mid-provision edit skips the export instead of publishing under a stale key. - Share one JWKS memo across provisionPlan's two shadows so their snapshot keys hash identical bytes. - Scope legacyWaitForShadowReady's lastFailure per evaluation (Effect.suspend). - Narrow the platform-error reason with Predicate.isTagged. - Relocate the shadow-cache/session-semantics notes from the frozen go-cli-divergences.md into the SIDE_EFFECTS.md files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…omments Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:e31bb57d1b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

jgoux

Reviewed the current head (ff17c99) against develop. I found no blocking correctness, lifecycle, data-safety, or command-routing issues.

The cache key covers the baseline-writing inputs; warm/cold fallback and container cleanup are coherent; targeted unit and integration coverage and the current CI checks are green.

Two non-blocking follow-ups worth considering before making this default-on:

add full-archive integrity verification beyond PG_VERSION plus the keyed marker;
replace legacyMemoizeSuccess with an Effect-owned single-flight primitive if shadow provisioning becomes concurrent.

Approving for the opt-in soak.

…I-2192) (supabase#6203) `supabase stop --no-backup` was gated by Docker's 10s SIGTERM grace: Kong, Vector, Edge Runtime, and Logflare kept `sh` as PID 1 (or hung in their own shutdown), and stops run in parallel so wall time = slowest container. Postgres `exec` and postgres-meta v0.98.0 already landed on develop with supabase#6184 / supabase#6207. This PR is only the remaining wrappers, rebased onto that tip. ## What changed **`exec` in every remaining CLI-authored wrapper** so the real process is PID 1 (deliberate divergence from Go; stop timing is outside the parity surface, ADR 0016). Command paths, output, exit codes, telemetry, and documented side effects unchanged; no `-t`, no `docker rm -f`, no grace-period env: - **Kong** - **Vector** — wget wait-loop kept; TERM during that wait now exits instead of burning the 10s grace - **Edge Runtime** — long-lived `start` / `functions serve` container and the ephemeral script-runner ## `supabase stop --no-backup` wall time | CLI | run 1 | run 2 | run 3 | min | avg | max | |---|---:|---:|---:|---:|---:|---:| | `@beta` 2.116.0-beta.6 | 12.665s | 12.743s | 12.502s | 12.502s | **12.64s** | 12.743s | | PR 6203 @ 28c39f1 | 4.918s | 4.885s | 5.623s | 4.885s | **5.14s** | 5.623s | **~2.5× faster** (avg **−7.5s / −59%**). Start time is unchanged (same ~30–45s after images are warm). ## Per-container `docker stop -t 10` | Container | PID 1 `@beta` | PID 1 PR | Stop `@beta` | Stop PR | |---|---|---|---:|---:| | kong | `sh -c` … `docker-entrypoint.sh kong docker-start` | `nginx: master process` | 10.241s | **0.167s** | | vector | `sh -c` … `vector --config` | `vector --config /etc/vector/vector.yaml` | 10.478s | **0.288s** | | edge_runtime | `sh -c edge-runtime start …` | `edge-runtime start …` | 10.405s | **0.296s** | | analytics (logflare) | `sh -c` … `sh run.sh` | supervised `sh run.sh` (TERM → 3s → KILL) | 10.274s | **3.227s** | | db | `postgres` (already exec) | `postgres` (unchanged) | 0.571s | 3.4–4.5s (see below) | | pg_meta | `node dist/server/server.js` | `node dist/server/server.js` | 0.257s | 0.297s | Stops run in parallel, so wall time ≈ slowest container + CLI prune (`container prune` → optional volume prune → network prune). `@beta` is gated by the ~10.4s wrappers. This PR is gated by **Logflare’s ~3s supervisor** and **Postgres shutdown (~4s on a full local db)**, which matches the ~5s `stop --no-backup` times. ## Options to gain more speed (defered) Just levers. After this PR, wall time is ~`max(logflare ≈ 3s, postgres ≈ 4s)` + ~1s prune. Cutting only Logflare leaves Postgres as the gate (~4s + prune ≈ 5s, i.e. little visible win). Beating ~3s wall needs **both** plus less prune. ### Logflare (~3s supervisor) 1. **Upstream:** image whose SIGTERM handler actually finishes (already tracked as the remaining nicety on CLI-2192). Then the 3s window can win without KILL. 2. **Shorter supervisor grace** (3s → 1s). Same TERM-then-KILL shape, ~2s less. More likely to KILL a BEAM that would have exited at 2.5s. 3. **KILL immediately** on TERM. Same end state as today’s post-timeout kill, ~3s sooner. Least graceful. ### Postgres (~4s smart shutdown) 4. **Fast shutdown** — trap SIGTERM and `pg_ctl stop -m fast` / SIGINT. Drops the wait-for-clients phase; still checkpoints. Helps most when Auth/REST/Realtime/Storage are still connected. 5. **Immediate shutdown** — `-m immediate` / SIGQUIT. Skips checkpoint; next `start` crash-recovers. Reasonable for `stop --no-backup` (volumes die anyway); bad if the volume is kept (`stop` without `--no-backup`). --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…an shadows Overlap the two next-engine plan shadows when a baseline can be shared, and keep progress lines ordered. Rebased onto develop after #6102/#6184/#6203 landed.

avalleteand others added 30 commits August 6, 2026 09:04

feat(cli): bundle pg-delta next engine
1baf8ea

chore: merge develop into pg-delta next
8078b53

fix(cli): allow pg-topo parser build script
4b697a2

fix(cli): embed libpg-query wasm in compiled binary
c52cf53

feat(cli): isolate pg-delta next shadow databases
1f82bf9

fix(cli): correct diff and migration execution contracts
5bffecb

chore: merge develop into pg-delta next
f12b1b9

test(cli): update transactional Go mocks
958d36c

chore(cli): remove unused diff helper
d486195

fix(cli): isolate pg-delta-next baseline behavior
394ecf8

chore: merge develop into pg-delta next
9f66d41

fix(cli): warn about manifestless declarative removals
438a9ac

fix(cli): warn on pg-delta coverage gaps
0ebe58c

fix(cli): format pg-delta SQL by default
dcdf7e0

fix(cli): clarify declarative diff baselines
16028da

test(cli): use malformed branch response type
b7ad988

chore(cli): update pg-delta next preview
7f6fb32

feat(cli): improve declarative schema upgrade flow
f9bd289

fix(cli): address pg-delta next review findings
46a1614

chore(cli): use published pg-delta alpha
ca55d8a

fix(cli): address pg-delta next rollout blockers
9f264e9

fix(cli): add pg-net webhook remediation
0879ced

fix(cli): scope pg-delta next schema filters
3f0428f

fix(cli): diff pg-delta next against live local database
1bc9e6b

fix(cli): preserve migration transaction metadata
be38f7b

fix(cli): converge database webhooks configuration
0139811

fix(cli): harden pg-delta schema workflows
ab8423b

Merge branch 'develop' into feat/upgrade-pg-delta-next
db04525

chatgpt-codex-connectorBot reviewed Aug 18, 2026
View reviewed changes

Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.live.test.ts Outdated
Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.live.test.ts Outdated

chatgpt-codex-connectorBot reviewed Aug 18, 2026
View reviewed changes

Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.live.test.ts Outdated

chatgpt-codex-connectorBot reviewed Aug 18, 2026
View reviewed changes

Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.ts

avalleteand others added 2 commits August 18, 2026 15:59

docs(cli): log the image-tag-vs-digest cache-key accepted risk
f3dc609
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Merge branch 'develop' into avallete/shadow-db-perf-6ad622
3dd25f3

avallete commented Aug 20, 2026
View reviewed changes

fix(cli): default the shadow baseline cache off for soak
7fd00b9
Gate SUPABASE_SHADOW_CACHE through the viper helper, tighten tar retention, drop the one-off debug channel, and reuse existing helpers.

chatgpt-codex-connectorBot reviewed Aug 20, 2026
View reviewed changes

avallete added 2 commits August 21, 2026 11:31

fix(cli): drop pgjwt before pgcrypto on PG15+ declarative shadows
5f80832
The platform image still installs pgjwt, so RESTRICT drop of pgcrypto fails during CLI-owned shadow prep.

Merge remote-tracking branch 'origin/develop' into avallete/pr-6184-m…
29858c6
…erge-conflicts-848c2e # Conflicts: # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts # apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts

avallete commented Aug 24, 2026
View reviewed changes

Comment threadapps/cli/docs/go-cli-divergences.md Outdated
Comment threadapps/cli/src/legacy/commands/db/diff/diff.handler.ts Outdated
Comment threadapps/cli/src/legacy/commands/db/diff/diff.handler.ts Outdated

docs(cli): trim shadow-cache rationale comments per review
bb21b82
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connectorBot reviewed Aug 24, 2026
View reviewed changes

test(cli): create shadow-cache e2e drift via db query
3c8ac90
The next engine ignores schema_paths when building its migrations baseline, so the declared-schema drift diffed to nothing. Also fix the invalid 'analytics' exclude (logflare). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connectorBot reviewed Aug 24, 2026
View reviewed changes

Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.ts
Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.ts Outdated
Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-database.ts Outdated
Comment threadapps/cli/src/legacy/shared/legacy-pgdelta.paths.ts

avalleteand others added 2 commits August 24, 2026 14:04

chore(cli): match archive problems exhaustively and reframe session c…
e31bb57
…omments Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chatgpt-codex-connectorBot reviewed Aug 24, 2026
View reviewed changes

Comment threadapps/cli/src/legacy/shared/db-bootstrap/shadow-cache.ts
Comment threadapps/cli/src/legacy/shared/db-bootstrap/container-lifecycle.ts Outdated

chore(cli): reword lifecycle cleanup comment in its own terms
ff17c99
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

jgoux approved these changes Aug 24, 2026
View reviewed changes

avallete added this pull request to the merge queue Aug 24, 2026

Merged via the queue into develop with commit ad0ad23Aug 24, 2026
40 checks passed

avallete deleted the avallete/shadow-db-perf-6ad622 branch August 24, 2026 14:48

supabase-cli-releaserBot mentioned this pull request Aug 25, 2026
chore: production deploy #6321
Merged

chatgpt-codex-connectorBot mentioned this pull request Aug 26, 2026
docs(release): notes for v2.116.0 #6346
Closed

Uh oh!

Conversation

avallete commented Aug 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Cache contract

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

avallete left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connectorBot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

jgoux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

avallete commented Aug 13, 2026•
edited
Loading