Skip to content

enhancement: add postgres support - #1

Merged
jasonmccallister merged 4 commits into
mainfrom
add-postgres-support
Apr 27, 2026
Merged

enhancement: add postgres support#1
jasonmccallister merged 4 commits into
mainfrom
add-postgres-support

Conversation

@jasonmccallister

Copy link
Copy Markdown
Member

Adds postgres support, as default, with a new createQueue factory helper. PostgreSQL driver run migrations

Signed-off-by: Jason McCallister <jason@mccallister.dev>
@jasonmccallisterjasonmccallister self-assigned this Apr 27, 2026
@jasonmccallisterjasonmccallister added the enhancement New feature or request label Apr 27, 2026

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a pluggable driver architecture for @snaapi/queue and adds a new Postgres-backed implementation (now the default when using the new createQueue() factory), with accompanying docs, tests, and CI updates.

Changes:

  • Introduces a QueueDriver interface and refactors the queue/worker/dispatcher/middleware pipeline to be backing-store agnostic (Deno KV and Postgres).
  • Adds a Postgres driver (schema/migrations, LISTEN/NOTIFY + polling consumer, locks/counters/failed-job storage).
  • Adds Postgres integration tests, Docker compose setup, and CI job configuration to run both KV and Postgres test suites.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file
FileDescription
tests/queue_test.tsUpdates job context assertions to validate locks/counters instead of kv.
tests/postgres_test.tsAdds Postgres integration test suite covering dispatch, delay, context, middleware, chains, uniqueness, failed jobs, retries, and migrations.
src/worker.tsRefactors worker execution to use QueueDriver and populates JobContext with locks/counters.
src/types.tsUpdates JobContext to expose locks/counters primitives; updates comments around backing store.
src/queue.tsMakes Queue accept Deno.Kv or QueueDriver, adds listener lifecycle management, and exposes driver.
src/middleware.tsMigrates rateLimit / withoutOverlapping from direct KV usage to ctx.counters / ctx.locks.
src/failed.tsRefactors FailedJobStore into a driver-backed facade.
src/factory.tsAdds createQueue() factory selecting driver via env/options (defaults to Postgres).
src/drivers/types.tsIntroduces driver interfaces: enqueue/listen/unique, failed store, locks, counters.
src/drivers/postgres.tsAdds Postgres driver: schema/migrate, enqueue, LISTEN/NOTIFY listener loop, locks/counters/failed store.
src/drivers/deno-kv.tsAdds Deno KV driver implementing the new driver interfaces.
src/dispatcher.tsRefactors dispatcher to enqueue through QueueDriver (including unique dispatch).
src/chain.tsRefactors chain dispatch/advance to use QueueDriver.
scripts/migrate.tsAdds a script to run Postgres schema migration from the CLI.
mod.tsExports the new factory and drivers/types.
deno.lockAdds npm:pg dependency.
deno.jsonAdds imports mapping for pg and adds tasks for Postgres testing and DB lifecycle/migration.
compose.yamlAdds local Postgres service definition for development/testing.
README.mdUpdates docs to describe backing stores, createQueue(), Postgres usage, and migration notes.
.github/workflows/ci.ymlAdds Postgres service and runs both KV and Postgres test tasks in CI.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadsrc/worker.ts
Comment threadsrc/drivers/postgres.ts Outdated
Comment threadsrc/drivers/postgres.ts
Comment threadsrc/drivers/postgres.ts Outdated
Comment threadsrc/drivers/postgres.ts Outdated
Comment threadsrc/drivers/postgres.ts
jasonmccallisterand others added 3 commits April 27, 2026 18:54
Signed-off-by: Jason McCallister <jason@mccallister.dev>
Signed-off-by: Jason McCallister <jason@mccallister.dev>
Reservations now hold the original row until the handler returns, so a
mid-flight retry (which preserves envelope.id) collided on the jobs
primary key. Generating a fresh UUID for the row keeps the dispatch id
on the envelope (and ctx.id) stable while letting retries coexist with
the reserved original.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jasonmccallister
jasonmccallister merged commit 6d5eff6 into mainApr 27, 2026
1 check passed
@jasonmccallister
jasonmccallister deleted the add-postgres-support branch April 27, 2026 23:26
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jasonmccallister