supabase test db silently reports Files=0, Tests=0 on Linux regardless of path argument

[edwinquick]

Bug report

supabase test db on the Linux release build (tested: standalone Go binary from supabase_linux_amd64.tar.gz, and the .deb package -- both v2.100.1) reports:

Connecting to local database...
Files=0, Tests=0, 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
Result: NOTESTS

and exits 0 (success), regardless of what path is given -- even when the target files unambiguously exist and are readable.

Tried, all producing the identical Files=0 result:

• Bare supabase test db (no path argument)
• supabase test db supabase/tests/database (directory)
• supabase test db supabase/tests/database/ (trailing slash)
• supabase test db ./supabase/tests/database
• supabase test db supabase/tests/database/*.sql (shell-expanded individual files)
• supabase test db supabase/tests/database/one_specific_file.test.sql (single, definitely-existing file)
• With and without --local

Reproduction (self-contained)

# Any Linux host with Docker
git clone <any repo with supabase/tests/database/*.test.sql pgTAP files>
cd <repo>
curl -fsSL -o sb.tar.gz https://github.com/supabase/cli/releases/download/v2.100.1/supabase_linux_amd64.tar.gz
mkdir sbx && tar xzf sb.tar.gz -C sbx
./sbx/supabase-go start
ls supabase/tests/database/*.sql # confirms files exist
./sbx/supabase-go test db supabase/tests/database
# Files=0, Tests=0, Result: NOTESTS -- despite the ls above showing real files

Reproduced in three independent environments: a custom self-hosted GitHub Actions runner container, a completely unrelated freshly-created ubuntu:24.04 container (Docker socket bind-mounted, no other shared state), and via docker exec directly. Confirmed not an issue with the reproducing environment's file layout, working directory, or Docker mount configuration -- pwd, ls, and file content are all correct at the exact point test db is invoked.

What I ruled out

• Not a missing-dependency issue: installed postgresql-client (psql) and libtap-parser-sourcehandler-pgtap-perl (pg_prove) -- no change in behavior.
• Not a subprocess-shelling issue: supabase test db ... --debug shows zero trace of ever invoking pg_prove/psql/prove as a subprocess (no exec/cmd/prove mentions anywhere in the debug log). The Go binary appears to have its own native TAP-summary-formatted output (mimicking Test::Harness's classic Files=N, Tests=N, ... wallclock secs line) without ever calling out to Perl's prove/pg_prove at all.
• Workaround confirmed: calling pg_prove directly against the same local Postgres connection (pg_prove --host 127.0.0.1 --port 54322 --username postgres --dbname postgres supabase/tests/database/*.sql) works correctly and finds/runs all files as expected. So the underlying Postgres instance, pgTAP extension, and test files are all fine -- this is specifically test db's own internal file-argument handling that's broken on the Linux build.
• Not a stale/cached-state issue: reproduced on a database freshly created via supabase start moments earlier, in a container with no prior Supabase state at all.

Version

supabase --version
2.100.1

(also reproduced via the raw supabase-go binary from the same release tarball, so this isn't specific to the Bun-wrapped supabase entrypoint either)

Impact

Silent -- test db exits 0 with Result: NOTESTS, which most CI setups (including ours) don't treat as a failure. This means a CI job built around supabase test db on Linux can run zero tests indefinitely while reporting green, with no error surfaced anywhere. Given how easy this is to reproduce and how silent the failure mode is, this seems worth a loud warning/non-zero exit at minimum, even before the root cause is found.

[hamodywe]

Traced this through the current implementation. I think there are two separate things here, and only one of them is Linux.

The Files=0 itself is almost certainly the Docker socket, not Linux. All three environments you reproduced in are containers with the host's Docker socket bind-mounted. test db runs pg_prove in a sibling container and passes your test path as a bind mount — apps/cli/src/legacy/shared/legacy-test-db.pg-prove-args.ts builds `${hostMount}:${dockerMount}:ro` and hands the same path to pg_prove as an argument. Those paths are resolved against the filesystem of the process running the CLI, but the daemon that has to satisfy the bind is the one on the host. Inside a container, /…/supabase/tests/database exists for you and does not exist for the daemon, so Docker creates an empty directory at that path on the host and mounts that. pg_prove -r then walks an empty directory and correctly reports Files=0.

That also explains the two things that looked contradictory in your report: ls showing the files (your container's filesystem) and --debug showing no pg_prove/psql subprocess (there is none — it runs in a container, legacy-test-db.handler.ts calls docker.run(...)).

Worth testing to confirm: run the same command directly on the Docker host rather than inside a container. If it passes there, it is the socket, and the fix on your side is to make the path identical inside and out (bind the project at the same absolute path in your runner container).

The part that is a real bug regardless: this exits 0. The handler passes the container's exit code straight through —

// Non-zero pg_prove exit → fail (exit 1), matching Go's cobra error return.
if (exitCode !== 0) { return yield* Effect.fail(new LegacyTestDbRunError(...)); }

— and pg_prove exits 0 for Result: NOTESTS. So "I found no tests to run" is reported as success. In CI that is the worst possible failure mode: a misconfigured path, a typo, or exactly the socket situation above all turn a test suite into a green build that ran nothing. That is how this went unnoticed across three environments.

A host-side guard before the container starts would catch most of it cheaply: resolve the paths (they already are, in buildLegacyPgProveArgs), and fail with a real message when none exists or none contains a .sql/.pg file, instead of letting pg_prove shrug. It would not catch the socket case — the files do exist for the CLI there — so treating NOTESTS as a failure is the stronger fix, though that means looking at the TAP stream, which is currently written straight to stdout by the subprocess and never inspected.

Happy to put up the host-side validation if that is the direction you want; the NOTESTS handling is a design call about the output stream that I would rather not make for you.

[hamodywe]

Following up on my note above — I've got the host-side guard written and tested locally (fails with an actionable error before the container starts, instead of letting pg_prove's NOTESTS exit 0 slip through as a green build; an explicitly-passed file is never blocked). Happy to open a PR against it if this is open for an outside contribution — I saw it's assigned, so didn't want to step on anything or trip the contribution gate. Just let me know if you'd like the PR (and the open-for-contribution label), or if you'd rather take it in-house.