Shared Postgres world: queue namespaces don't isolate claiming, and startup recovery re-enqueues other applications' active runs

[mechiland]

Summary

When multiple independent applications share one @workflow/world-postgres database, they execute each other's workflow runs. Queue namespaces (WORKFLOW_QUEUE_NAMESPACE) do not prevent this, for two independent reasons:

1. Claiming ignores namespaces. The graphile-worker task identifiers are ${jobPrefix}flows / ${jobPrefix}steps — the namespace only prefixes the topic string inside the message. Any consumer polling the same database claims any namespace's jobs, and createTaskHandler then reconstructs the queue name using the consumer's own prefix (const queueName = ${queue}${messageData.id}``), so the foreign job is executed locally instead of being skipped.
2. Startup recovery is unscoped. reenqueueActiveRuns lists all pending/running runs with no namespace/deployment filter and re-enqueues each under the booting application's namespace prefix. So every application boot takes over every other application's in-flight runs. Fix namespaced active run recovery #2888 fixed which namespace the recovered runs are enqueued under, and [world-local] Scope untagged startup recovery to untagged runs #2667 scoped world-local recovery by tag — but the Postgres world still passes its raw, unfiltered storage.runs into recovery (packages/world-postgres's start()), so cross-application takeover remains. Notably, workflow_runs.deployment_id exists in the schema but world-postgres hardcodes getDeploymentId() to 'postgres', so it can't be used for scoping today.

Code references (checked against @workflow/world@5.0.0-beta.21 and @workflow/world-postgres@5.0.0-beta.27, the latest betas as of 2026-07-17):

• packages/world/src/recovery.ts — runs.list({ status, resolveData: 'none', ... }) with no scoping parameter.
• packages/world-postgres/src/queue.ts — getJobQueueName() derives the task identifier from jobPrefix only; createTaskHandler() rebuilds the queue name from the local prefix without validating the producer's namespace.
• packages/world-postgres/src/index.ts — start() → reenqueueActiveRuns(storage.runs, queue.queue, 'world-postgres', config.namespace).

How we hit this

We run a self-hosted platform that hosts many independent Eve agents (each is its own app/process), and initially pointed all of them at one WORKFLOW_POSTGRES_URL. Eve already derives a distinct WORKFLOW_QUEUE_NAMESPACE per agent, so topics were namespaced — yet agents intermittently executed each other's turns with the wrong application's code and credentials (surfacing as flaky provider/API-key errors, depending on which runner won the claim). Tracing it led to the two mechanisms above.

Repro sketch

1. Two apps A and B (any two workflow apps with same-named workflows — e.g. two instances of the same framework), same WORKFLOW_POSTGRES_URL, distinct WORKFLOW_QUEUE_NAMESPACE.
2. Start a long-running workflow in A.
3. While it is running, boot B. B logs [world-postgres] Re-enqueued N active run(s) on startup including A's run, under B's namespace.
4. B's runner claims and executes A's run (same workflow_flows task id; handler reconstructs the topic with B's prefix). With distinct WORKFLOW_POSTGRES_JOB_PREFIX values the takeover becomes deterministic: the re-enqueued foreign run can only be claimed by B.

Suggested directions

Either would resolve this class of issue:

• Scope claiming and recovery to the world's identity — e.g. include the namespace in the graphile task identifier (or validate the producer namespace in the handler and re-enqueue/skip on mismatch), and give runs.list a filter that reenqueueActiveRuns passes through (the deployment_id column looks like a natural hook, similar in spirit to the world-local tag scoping in [world-local] Scope untagged startup recovery to untagged runs #2667).
• Or, if a Postgres world database is intended to be exclusive to a single application, state that explicitly in the world-postgres docs — today nothing fails loudly when two applications share one database; they just silently cross-execute.

Workaround

We moved to one Postgres database per application (derived from a base URL, created + workflow-postgres-setup bootstrapped on demand). That fully isolates storage, queue, and recovery, and has been working well — sharing it here for other self-hosters who hit the same symptom.