improvement(db): opt-in read-replica client + migration runner hardening

[waleedlatif1]

Summary

Migration runner hardening (packages/db/scripts/migrate.ts)

• lock_timeout = '5s' asserted at the top of every migration attempt — DDL that can't get its table lock fails fast (SQLSTATE 55P03) instead of queueing all table traffic behind a pending AccessExclusiveLock. Applied via client.unsafe (Postgres rejects parameterized SET)
• Retry on lock timeout: up to 8 attempts with jittered exponential backoff (backoffWithJitter), 55P03 detected via getPostgresErrorCode (walks drizzle's wrapped cause chains)
• Advisory-lock acquisition is now a bounded pg_try_advisory_lock loop (30-min deadline) — a wedged runner fails its peers visibly instead of silently hanging concurrent migration runners
• warnOnInvalidIndexes() after every run — surfaces INVALID indexes from failed CONCURRENTLY builds that IF NOT EXISTS would silently skip forever
• CONCURRENTLY convention updated: post-COMMIT files must clear AND restore lock_timeout, with corrected batch-transaction semantics (drizzle wraps all pending files in ONE transaction) and idempotency requirements
• Migration session pinned for the run: max_lifetime: null (postgres-js otherwise recycles the connection after a randomized 30–60 min, silently dropping the advisory lock and session SETs) and a backend-pid guard before every attempt (an externally dropped/replaced connection aborts loudly instead of running unlocked)
• Optional MIGRATION_DATABASE_URL: migrations prefer a direct (non-pooled) DSN — session advisory locks and session SETs are unsupported through PgBouncer transaction pooling. Falls back to DATABASE_URL (direct-connection setups need no change)

Read replica (opt-in, not withReplicas)

• dbReplica export in @sim/db: separate pool from optional DATABASE_REPLICA_URL, falls back to the primary when unset (self-hosted setups unaffected), fail-fast DSN validation at boot
• Flipped heavy, staleness-tolerant reads: logs listing (list-logs.ts), logs stats + export routes, v1 audit-log page queries, the display-only reads in getUserUsageLogs, and data-drain source page scans
• Billing display reads threaded to the replica via an optional executor: DbOrTx = db param on getBillingPeriodUsageCost / getBillingPeriodUsageCostByUser / computeDailyRefreshConsumed / getUserUsageData / getEffectiveCurrentPeriodCost / getSimplifiedBillingSummary / getOrganizationBillingData / getOrgMemberLedgerByUser. Only display routes (billing summary, usage-limits, org member ledgers, admin billing views) pass the replica; every enforcement caller (usage gates, overage/invoice calculation, threshold billing) keeps the primary default
• Deliberately kept on the primary: billing enforcement, auth, workflow state, audit-log org scoping and data-drain workspace scoping (authz/cursor correctness), log detail fetch, table service
• Test mocks (@sim/testing + local factories) export dbReplica as the same instance as db

Misc

• table-grid column rename returns mutateAsync per useInlineRename's contract (isSaving now spans the request)

Deliberate trade (reviewed)

The per-row permissions join inside the logs list/stats/export queries now evaluates on the replica: a user whose workspace access was just revoked can still list/export that workspace's logs for up to the replication-lag window (typically sub-second). Accepted because the exposure is transient and bounded, there is no monotonic cursor to cause permanent damage, and pinning the join to the primary would defeat the offload. Tenant-boundary scoping reads that gate external API access (v1 audit logs org scope, data-drain workspace scope) deliberately stay on the primary.

Type of Change

• Bug fix / improvement

Testing

• Typecheck + 318 tests passing across touched areas; check:api-validation passes
• Smoke tests against a real Postgres: parameterized-SET failure reproduced and fix verified; real lock contention → 55P03 → retry path verified through wrapped errors
• End-to-end: two concurrent runners against a fresh database — winner applied all 231 migrations under the 5s lock_timeout (incl. both CONCURRENTLY files), loser waited on the advisory lock and exited 0, zero invalid indexes; MIGRATION_DATABASE_URL override verified

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jun 11, 2026 1:11am

[cursor]

PR Summary

Medium Risk
Billing enforcement remains on the primary, but replica-backed logs/permissions joins and display billing can be briefly stale; migration locking/DDL retry changes affect deploy-time schema applies.

Overview
Adds an opt-in dbReplica client (DATABASE_REPLICA_URL, primary fallback) and routes heavy, display-only reads through it: logs list/stats/export, audit-log pages, usage-log listing/summaries, data-drain scans, and billing summary APIs via an optional executor on billing helpers. Enforcement paths (usage gates, writes, auth scoping for drains/audit org boundaries) stay on the primary; usage-log cursor resolution stays on primary to avoid broken pagination.

Migration runner is hardened: optional MIGRATION_DATABASE_URL for direct DSNs, bounded pg_try_advisory_lock, session pinning + backend-pid guard, lock_timeout with retries on 55P03, and post-run invalid-index warnings. CI wires migration secrets; env examples document replica and migration URLs.

Replica pagination safety: data drains apply a 5-minute timeCursorStabilityBound; /users/me/usage-limits derives currentPeriodCost from the same checkServerSideUsageLimits result as limit so display cannot disagree under lag.

UI: table column rename returns mutateAsync so useInlineRename tracks in-flight saves.

^{Reviewed by Cursor Bugbot for commit f078233. Configure here.}

[greptile-apps]

Greptile Summary

This PR introduces two independent but related improvements: migration runner hardening and an opt-in read-replica client.

• Migration hardening (migrate.ts): pins the session with max_lifetime: null, converts the advisory lock acquisition to a bounded try-lock loop (30-min deadline), asserts lock_timeout = '5s' before every attempt so blocking DDL fails fast instead of stalling the table, retries on 55P03 with jittered exponential backoff, guards against silent connection recycling with a backend-PID check on direct connections, and surfaces INVALID indexes left by failed CONCURRENTLY builds.
• Read-replica routing (packages/db/db.ts + 15+ call sites): adds a dbReplica export that falls back to the primary when DATABASE_REPLICA_URL is unset; routes logs listing/export/stats, billing display queries, and data-drain page scans to the replica while keeping all enforcement, auth, workflow state, and authz-boundary reads on the primary.
• Data-drain stability (cursor.ts): timeCursorStabilityBound excludes rows newer than 5 minutes from cursor-based page scans, preventing replica-lag-induced cursor skips from permanently dropping rows.

Confidence Score: 5/5

Safe to merge. Billing enforcement, auth, and authz-boundary reads deliberately stay on the primary; the replica is only used for display and export paths where bounded staleness is acceptable.

The enforcement/replica boundary is carefully drawn and consistently applied across all 15+ touched call sites. Migration hardening is additive and backward-compatible. The one minor inconsistency (userStats baseline in getEffectiveCurrentPeriodCost ignoring executor) has no current caller that would trigger it.

apps/sim/lib/billing/core/usage.ts — getEffectiveCurrentPeriodCost hardcodes db for the userStats baseline query while the rest of the function respects executor.

Important Files Changed

Filename	Overview
packages/db/scripts/migrate.ts	Major hardening: bounded advisory-lock acquisition loop, lock-timeout (55P03) retry with jittered backoff, session-pinned connection, PID guard for direct connections, and invalid-index warnings after each run.
packages/db/db.ts	Adds opt-in dbReplica export with fail-fast DSN validation at module load; falls back to primary when DATABASE_REPLICA_URL is unset, preserving self-hosted compatibility.
apps/sim/lib/billing/core/usage.ts	Adds executor: DbOrTx param to getUserUsageData and getEffectiveCurrentPeriodCost; correctly threads replica to getBillingPeriodUsageCost and computeDailyRefreshConsumed calls, but the userStats.currentPeriodCost baseline query in getEffectiveCurrentPeriodCost hardcodes db rather than using executor.
apps/sim/lib/billing/core/usage-log.ts	Adds executor: DbOrTx param to getBillingPeriodUsageCost/getBillingPeriodUsageCostByUser; cursor resolution for getUserUsageLogs correctly pins to primary, page and summary reads flip to dbReplica.
apps/sim/lib/billing/credits/daily-refresh.ts	Adds executor: DbOrTx param to computeDailyRefreshConsumed; the heavy usageLog aggregation query correctly uses executor while function signature is threaded from all call sites.
apps/sim/lib/data-drains/sources/cursor.ts	Adds timeCursorStabilityBound helper that excludes rows < 5 min old, guarding against replica lag causing cursor-skipped rows in data drain pagination.
apps/sim/app/api/v1/audit-logs/query.ts	getOrgWorkspaceIds keeps the org-scope boundary fetch on primary db; only the data page query (queryAuditLogs) switches to dbReplica, consistent with the PR's stated tenant-boundary guarantee.
apps/sim/app/api/users/me/usage-limits/route.ts	Removes the redundant parallel getEffectiveCurrentPeriodCost call and reuses usageCheck.currentUsage from checkServerSideUsageLimits, which already reads from the primary DB via the enforcement path.
packages/testing/src/mocks/database.mock.ts	Correctly exports dbReplica as the same object instance as db in both dbChainMock and databaseMock, ensuring per-test chain overrides on db also cover replica call sites.
apps/sim/lib/billing/core/organization.ts	Adds executor: DbOrTx to getOrgMemberLedgerByUser and getOrganizationBillingData; correctly threads replica from display routes only.
.github/workflows/migrations.yml	Adds MIGRATION_DATABASE_URL secret plumbing for production and staging; omits dev (no direct DSN secret exists for dev), and falls back to DATABASE_URL when unset.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[API Request] --> B{Read type?}
B -->|Enforcement: usage gates, thresholds| C[primary db]
B -->|Auth / workflow state| C
B -->|Billing display: summary, usage-limits| D{DATABASE_REPLICA_URL set?}
B -->|Log listing / stats / export| D
B -->|Data drain page scans| D
B -->|Audit-log API page queries| D
D -->|Yes| E[dbReplica pool - max: 10 connections]
D -->|No| C
C --> F[(Primary Postgres)]
E --> G[(Read Replica)]
G -.->|async replication| F
subgraph boundary[Boundary reads - always primary]
H[org workspace scope IDs - getOrgWorkspaceIds]
I[cursor resolution - usageLog cursor row]
J[auth / session]
end
H --> F
I --> F
J --> F

Loading

_{Reviews (5): Last reviewed commit: "fix(billing): usage-limits returns cost ..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f078233. Configure here.}