feat(cli): port supabase storage ls/cp/mv/rm to native TypeScript by Coly010 · Pull Request #5667 · supabase/cli

Coly010 · 2026-06-23T16:30:35Z

What changed

Replaces the Phase 0 LegacyGoProxy shims for storage ls/cp/mv/rm with native Effect implementations that talk to the Storage gateway directly (apps/cli-go/internal/storage as the reference), and adds TS-only --output-format json|stream-json on top of byte-faithful Go text output.

ls — pagination + recursive BFS over buckets/objects.
cp — 4-way scheme branch (download/upload, single/recursive), dst-key resolution, bucket auto-create on Bucket not found, content-type sniff/refine, --cache-control/--content-type/--jobs.
mv — single move with recursive per-object BFS fallback on not_found.
rm — group-by-bucket, confirm prompt, chunked deletes (1000), recursive directory + bucket removal.

Shared-infra promotion (hoist-before-duplicate)

The Storage gateway, credential derivation, bucket-config helpers, content-type sniffing, the gateway runtime layer, and the yes/no prompt are promoted out of seed buckets into legacy/shared/. seed buckets is refactored to consume them in the same change (its existing suite covers the regression).

Reviewer notes

--linked/--local are per-leaf, not group scoped globals. Effect CLI requires global-flag names to be unique tree-wide and seed already owns linked/local with opposite Go defaults, so two groups can't both declare them (caught by the bundled-binary smoke test). Cost vs Go's persistent flags: the flags must follow the subcommand token (storage ls --local, not storage --local ls) — the same shape the db family uses. Documented in storage.flags.ts + each SIDE_EFFECTS.md.
Parity over hardening: a multi-perspective review flagged a path-traversal write primitive in recursive download (remote object names → local paths). Go's filepath.Join has no boundary guard, so adding one would diverge from the strict 1:1 port; left identical to Go (noted for a future, explicit hardening).
Porting tracker flipped to ported; the bogus --copy-metadata cp note removed (no such Go flag).

Replaces the Phase 0 Go-proxy shims for `storage ls/cp/mv/rm` with native Effect implementations that talk to the Storage gateway directly, mirroring `apps/cli-go/internal/storage` byte-for-byte in text mode and adding TS-only `--output-format json|stream-json` structured output. Per the hoist-before-duplicate policy, the storage gateway, credential derivation, bucket-config helpers, content-type sniffing, the gateway runtime layer, and the yes/no prompt are promoted out of `seed buckets` into `legacy/shared/`; `seed buckets` is refactored to consume them (regression covered by its existing suite). `--linked`/`--local` are declared per-leaf rather than as `storage`-group scoped globals: Effect CLI requires global-flag names to be unique tree-wide and `seed` already owns `linked`/`local` with opposite defaults. The only behavioural cost vs Go's persistent flags is that they must follow the subcommand token (`storage ls --local`), the same shape the `db` family uses. Flips `storage ls/cp/mv/rm` to `ported` in the porting-status tracker and removes the bogus `--copy-metadata` cp note.

github-actions · 2026-06-23T20:08:50Z

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@cd6ba61cb24551c3f2b90efd75b852362a76035a

Preview package for commit cd6ba61.

`storage cp -r ss://… local` returned the walk error before running the downloads it had already queued. Go runs `errors.Join(walkErr, jq.Collect())` (cp.go:96): a walk that fails partway still runs the queued jobs, then surfaces the error. Sequence the `iterError` check after the download pass, keeping the `count == 0 → "Object not found"` early-return ahead of it (cp.go:93-95). Adds a regression test covering a mid-walk list failure with a download already queued. Also reproduce Go's cobra `(default …)` help tokens for `--cache-control`, `--content-type`, and `--jobs` inline in their descriptions, since Effect CLI renders no defaults. `--content-type`'s runtime default stays empty (auto-detect via sniffing); only its displayed default reads `auto-detect`, matching Go's `DefValue` override (storage.go:106).

…UPABASE_EXPERIMENTAL Two Go-parity gaps surfaced by review of the storage port: - `storage ls/cp/mv/rm` ran without `--experimental`. Go puts `storageCmd` in its experimental slice and rejects it in `PersistentPreRunE` (root.go:63,91-96). Add a shared `legacyRequireExperimental` gate (byte-exact "must set the --experimental flag to run this command") and wire it into each storage leaf after the `--linked/--local` mutex check and before instrumentation, matching Go's ValidateFlagGroups -> PersistentPreRunE order so a closed gate emits no telemetry/linked-project writes. - `SUPABASE_YES` was documented but never read. Go binds every persistent flag to viper with AutomaticEnv + the SUPABASE prefix (root.go:318-320,334), so `viper.GetBool("YES")`/`("EXPERIMENTAL")` honor the env vars. Add `legacyViperEnvBool` (strconv.ParseBool parity) and `legacyResolveYes` / `legacyResolveExperimental` accessors, and route all `--yes` prompt consumers (storage rm, seed buckets, secrets unset, config push, bootstrap) and the experimental gate through them. Also add the `storage cp` value-consuming flags (content-type, cache-control, jobs, -j) to the shared `--linked/--local` argv scanner so their values can't be mistaken for a target selector. Adds unit coverage for the env parser, the gate (flag + env paths), the scanner edge cases, and the rm SUPABASE_YES flow; updates storage SIDE_EFFECTS docs and the e2e surface.

jgoux

Reviewed the storage ls/cp/mv/rm port end-to-end against the Go reference (parity, shared-infra hoist, Effect/TS correctness, security, tests). Overall this is strong, faithful work — pagination, BFS ordering, the cp 4-way scheme matrix, dst-key resolution, bucket auto-create, mv recursive fallback, and all success/error strings match Go, and the seed buckets hoist is behavior-preserving. No behavior-breaking parity divergence found.

Leaving inline notes on two things worth addressing before merge — (1) the path-traversal write primitive in recursive download, and (2) a test-coverage asymmetry where cp/mv/rm lack the linked-path, HTTP-error, and stream-json tests that ls already has — plus a couple of minor clarity nits. Everything else verified correct.

jgoux · 2026-06-24T12:30:40Z

+ for (const name of names) {
+ const abs = path.join(dir, name);
+ // afero.Walk uses Lstat (no-follow); a symlink is not regular → skipped.
+ const isSymlink = yield* fs.readLink(abs).pipe(


Minor: this treats anyreadLink failure as "not a symlink." For a regular file the expected EINVAL is correct, but it also swallows genuine IO/permission errors that Go's Lstat-based afero.Walk would surface. Low impact, but slightly less faithful than Go — consider only treating EINVAL as not-a-symlink and propagating other errno values.

…Go deviation

…cepted Go-parity risk

… --jobs

… recursive moved count

… object-delete failure

Coly010 requested a review from a team as a code owner June 23, 2026 16:30

Coly010 force-pushed the cli/port-storage-commands branch from 9f57951 to 3fe148cCompare June 23, 2026 20:03

Coly010 self-assigned this Jun 24, 2026

Coly010 added 2 commits June 24, 2026 10:26

jgoux reviewed Jun 24, 2026
View reviewed changes

Coly010 added 5 commits June 24, 2026 14:16

docs(cli): document the storage cp --jobs <1 clamp as an intentional …
5db60ed
…Go deviation

docs(cli): note storage cp recursive-download path traversal as an ac…
938bc37
…cepted Go-parity risk

test(cli): cover storage cp linked-path, HTTP-error, stream-json, and…
ae539bb
… --jobs

test(cli): cover storage mv linked-path, HTTP-error, stream-json, and…
3a33a96
… recursive moved count

test(cli): cover storage rm linked-path, HTTP-error, stream-json, and…
cd6ba61
… object-delete failure

jgoux approved these changes Jun 24, 2026
View reviewed changes

Coly010 added this pull request to the merge queue Jun 24, 2026

Merged via the queue into develop with commit 6ff1a86Jun 24, 2026
54 checks passed

Coly010 deleted the cli/port-storage-commands branch June 24, 2026 14:24

Navigation Menu

Uh oh!

feat(cli): port supabase storage ls/cp/mv/rm to native TypeScript - #5667

feat(cli): port supabase storage ls/cp/mv/rm to native TypeScript#5667
Coly010 merged 8 commits into
developfrom
cli/port-storage-commands

Coly010 commented Jun 23, 2026•
edited
Loading

Uh oh!

chatgpt-codex-connectorBot commented Jun 23, 2026

Uh oh!

github-actionsBot commented Jun 23, 2026•
edited
Loading

Uh oh!

jgoux left a comment

Uh oh!

Uh oh!

Uh oh!

jgouxJun 24, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Coly010 commented Jun 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Shared-infra promotion (hoist-before-duplicate)

Reviewer notes

Related

Uh oh!

chatgpt-codex-connectorBot commented Jun 23, 2026

Uh oh!

github-actionsBot commented Jun 23, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Supabase CLI preview

Uh oh!

jgoux left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jgouxJun 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Coly010 commented Jun 23, 2026•
edited
Loading

github-actionsBot commented Jun 23, 2026•
edited
Loading