Uh oh!
There was an error while loading. Please reload this page.
Add interactive pagination for JSON list output - #5016
Closed
simonfaltum wants to merge 3 commits into
Closed
Conversation
When stdin, stdout, and stderr are all TTYs and a list command has no row template (so the CLI falls back to the JSON renderer), dumping hundreds of items at once scrolls everything past the user before they can read any of it. Introduce a simple interactive pager that streams 50 items at a time and asks the user what to do next on stderr: [space] more [enter] all [q|esc] quit Piped output and redirected stdout keep the existing non-paged behavior — the capability check requires all three streams to be TTYs. The accumulated output is always a syntactically valid JSON array, even if the user quits early, so readers that capture stdout still get parseable JSON. New in libs/cmdio: - capabilities.go: `SupportsPager()` — stdin + stdout + stderr TTYs, not Git Bash. - pager.go: shared plumbing for any interactive pager we add later — raw-mode stdin setup with a key-reader goroutine, `pagerNextKey` / `pagerShouldQuit`, `crlfWriter` to compensate for the terminal's cleared OPOST flag while raw mode is active, and the shared prompt/key constants. - paged_json.go: the JSON pager. Defers writing the opening bracket until the first item is encoded, so empty iterators and iterators that error before yielding produce a valid `[]` instead of a half-open array. - render.go: `RenderIterator` routes to the JSON pager when the capability check passes and no row template is registered. Test coverage: - crlfWriter newline translation (6 cases). - `pagerShouldQuit` / `pagerNextKey` behavior on quit keys, non-quit keys, and closed channels. - JSON pager: fits in one page, SPACE one/two more pages, ENTER drains, q/esc/Ctrl+C quit, Ctrl+C interrupts a drain, empty iterator, `--limit` respected, fetch errors preserve valid JSON, prompt goes to the prompts stream only. Co-authored-by: Isaac
Contributor
Approval status: pending
|
5 tasks
simonfaltum added a commit
that referenced
this pull request
Apr 17, 2026
Depends on #5016. Extends the interactive pager introduced in #5016 to commands that register a row template (jobs, clusters, apps, pipelines, etc.). Reuses the shared plumbing from that PR — raw-mode key reader, crlfWriter, prompt helpers, SupportsPager capability — and adds only the template-specific rendering on top. Shape of the new code: - paged_template.go: the template pager. Executes the header + row templates into an intermediate buffer per batch, splits by tab, locks visual column widths from the first batch, and pads every subsequent batch to those widths. The output matches the non-paged tabwriter path byte-for-byte for single-page results and stays aligned across pages for longer ones. - render.go: `RenderIterator` now routes to the template pager when a row template is set, and to the JSON pager otherwise. Covers the subtle rendering bugs that come up when you drop into raw mode and page output: - `term.MakeRaw` clears OPOST, disabling '\n'→'\r\n' translation; the already-shared crlfWriter fixes the staircase effect. - Header and row templates must parse into independent *template.Template instances so the second Parse doesn't overwrite the first (otherwise every row flush re-emits the header text). - An empty iterator still flushes the header. - Column widths are locked from the first batch so a short final batch doesn't visibly compress vs the wider batches above it. Co-authored-by: Isaac
go mod tidy promoted golang.org/x/term from an indirect dependency to a direct one (the JSON pager uses it for raw-mode stdin). The repo's TestRequireSPDXLicenseComment in internal/build rejects direct dependencies that don't carry an SPDX identifier in a comment, which tripped `make test` on every platform. Move the dependency into the main require block alongside the other golang.org/x packages and add the `// BSD-3-Clause` comment that matches its upstream license. Co-authored-by: Isaac
simonfaltum added a commit
that referenced
this pull request
Apr 17, 2026
Mirrors the fix in the base PR (#5016). go mod tidy promoted golang.org/x/term from indirect to a direct dependency, and the repo's TestRequireSPDXLicenseComment in internal/build rejects direct dependencies without an SPDX identifier comment — failing `make test` on every platform. Move the dependency into the main require block with the correct `// BSD-3-Clause` comment. This commit is independent from #5016 so 5015 can land on top of main; once the base PR merges, git will resolve this trivially on rebase. Co-authored-by: Isaac
TestNoticeFileCompleteness cross-checks the BSD-3-Clause section of NOTICE against the go.mod require block. Adding golang.org/x/term as a direct dependency (for raw-mode stdin) also requires adding its attribution to NOTICE. Mirror the existing entries for golang.org/x/sys and golang.org/x/text. Co-authored-by: Isaac
simonfaltum
commented
Apr 21, 2026
MemberAuthor
Closing. The shared pager infrastructure from this PR (capabilities, raw-mode key reader, |
simonfaltum added a commit
that referenced
this pull request
Apr 24, 2026
## Why List commands with a row template (`jobs list`, `clusters list`, `apps list`, `pipelines list`, `workspace list`, etc.) drain the full iterator and render every row at once. In workspaces with hundreds of resources, the output scrolls past before you can read it. An interactive terminal should get a chance to step through the output. This PR is an alternative to #4729 (the Bubble Tea TUI). It only solves pagination, nothing else. Smaller diff, no new public API, no override file changes. Interactive I/O is consolidated on bubbletea (already a direct dep via the spinner), so no new dependency is added. ## Changes **Before:** `databricks <resource> list` drained the full iterator through the existing template + tabwriter pipeline before showing anything. **Now:** when stdin, stdout, and stderr are all TTYs and the command has a row template, the CLI streams 50 rows at a time. While a batch is being fetched, the view shows a loading spinner: ``` ⣾ loading… ``` Between batches the prompt takes over: ``` [space] more [enter] all [q|esc] quit ``` `SPACE` fetches the next page. `ENTER` drains the rest (still interruptible by `q`/`esc`/`Ctrl+C` between pages, with the spinner staying up while fetching). `q`/`esc`/`Ctrl+C` stop immediately. Piped output and `--output json` keep the existing non-paged behavior. Rendering reuses the existing `Annotations["template"]` and `Annotations["headerTemplate"]`: colors, alignment, and row format come from the same code path as today's non-paged `jobs list`. No new `TableConfig`, no new `ColumnDef`, no changes to any override files. Files under `libs/cmdio/`: - `capabilities.go`: `SupportsPager()` (stdin + stdout + stderr all TTYs, not Git Bash). - `pager.go`: `pagerModel` (a `tea.Model`) that drives the paged render loop. Captures keys as `tea.KeyMsg`, emits rendered rows with `tea.Println` (which prints above the TUI area), and in `View()` shows either a spinner (while fetching) or the prompt (between pages). Same braille frames + green color as `cmdio.NewSpinner`. Only one fetch runs at a time; SPACE during an in-flight fetch is dropped, ENTER flips `drainAll` and lets the pending `batchMsg` chain the next fetch, so the iterator is never read from two goroutines. - `paged_template.go`: the template pager. Executes the header + row templates into an intermediate buffer per batch, splits by tab, computes visual column widths (stripping ANSI SGR so colors don't inflate), locks those widths from the first page, and pads every subsequent page to the same widths. Single-page output matches tabwriter's alignment; columns stay aligned across pages for longer lists. - `render.go`: `RenderIterator` routes to the template pager when the capability check passes and a row template is set. No `cmd/` changes. No new public API beyond `Capabilities.SupportsPager`. Notes on the bubbletea approach: - Using `tea.Model` + `tea.Println` means we don't call `term.MakeRaw` ourselves: tea enters and restores raw mode on its own, so the earlier `crlfWriter` workaround for the cleared `OPOST` flag is gone. - The header and row templates parse into independent `*template.Template` instances. Sharing one receiver causes the second `Parse` to overwrite the first, which made `apps list` render the header in place of every data row. - Empty iterators still flush their header: the first fetch returns `done=true` with header lines, and the pager prints them before quitting. - Tabwriter computes column widths per-flush and resets them. The pager does the padding itself with widths locked from the first batch, so a short final batch does not compress visually against wider pages above it. History: this consolidates #5016 (shared pager infrastructure) and drops an earlier JSON-output pager. JSON output is mostly consumed by scripts, so paging it adds complexity without a clear win. ## Test plan - [x] `go test ./libs/cmdio/...` passes. Coverage: state-machine unit tests on `pagerModel.Update` (init, batch handling, drain chaining, error propagation, every key path, in-flight-fetch serialization, spinner visibility) plus end-to-end tests via `tea.Program` for full drain, `--limit` integration, header-once, empty iterator, header + rows, cross-batch column stability, and content parity with the non-paged path for single-page lists. - [x] `make checks` passes. - [x] `make lint` passes (0 issues). - [ ] Manual smoke in a TTY: `apps list`, `jobs list`, `clusters list`, `workspace list /`. First page renders after a brief spinner, SPACE fetches next (spinner reappears), ENTER drains (spinner stays up), `Ctrl+C`/`esc`/`q` quit (and interrupt a drain). - [ ] Manual smoke with piped stdout and `--output json`: output unchanged from `main`.
denik pushed a commit
that referenced
this pull request
May 20, 2026
## Why List commands with a row template (`jobs list`, `clusters list`, `apps list`, `pipelines list`, `workspace list`, etc.) drain the full iterator and render every row at once. In workspaces with hundreds of resources, the output scrolls past before you can read it. An interactive terminal should get a chance to step through the output. This PR is an alternative to #4729 (the Bubble Tea TUI). It only solves pagination, nothing else. Smaller diff, no new public API, no override file changes. Interactive I/O is consolidated on bubbletea (already a direct dep via the spinner), so no new dependency is added. ## Changes **Before:** `databricks <resource> list` drained the full iterator through the existing template + tabwriter pipeline before showing anything. **Now:** when stdin, stdout, and stderr are all TTYs and the command has a row template, the CLI streams 50 rows at a time. While a batch is being fetched, the view shows a loading spinner: ``` ⣾ loading… ``` Between batches the prompt takes over: ``` [space] more [enter] all [q|esc] quit ``` `SPACE` fetches the next page. `ENTER` drains the rest (still interruptible by `q`/`esc`/`Ctrl+C` between pages, with the spinner staying up while fetching). `q`/`esc`/`Ctrl+C` stop immediately. Piped output and `--output json` keep the existing non-paged behavior. Rendering reuses the existing `Annotations["template"]` and `Annotations["headerTemplate"]`: colors, alignment, and row format come from the same code path as today's non-paged `jobs list`. No new `TableConfig`, no new `ColumnDef`, no changes to any override files. Files under `libs/cmdio/`: - `capabilities.go`: `SupportsPager()` (stdin + stdout + stderr all TTYs, not Git Bash). - `pager.go`: `pagerModel` (a `tea.Model`) that drives the paged render loop. Captures keys as `tea.KeyMsg`, emits rendered rows with `tea.Println` (which prints above the TUI area), and in `View()` shows either a spinner (while fetching) or the prompt (between pages). Same braille frames + green color as `cmdio.NewSpinner`. Only one fetch runs at a time; SPACE during an in-flight fetch is dropped, ENTER flips `drainAll` and lets the pending `batchMsg` chain the next fetch, so the iterator is never read from two goroutines. - `paged_template.go`: the template pager. Executes the header + row templates into an intermediate buffer per batch, splits by tab, computes visual column widths (stripping ANSI SGR so colors don't inflate), locks those widths from the first page, and pads every subsequent page to the same widths. Single-page output matches tabwriter's alignment; columns stay aligned across pages for longer lists. - `render.go`: `RenderIterator` routes to the template pager when the capability check passes and a row template is set. No `cmd/` changes. No new public API beyond `Capabilities.SupportsPager`. Notes on the bubbletea approach: - Using `tea.Model` + `tea.Println` means we don't call `term.MakeRaw` ourselves: tea enters and restores raw mode on its own, so the earlier `crlfWriter` workaround for the cleared `OPOST` flag is gone. - The header and row templates parse into independent `*template.Template` instances. Sharing one receiver causes the second `Parse` to overwrite the first, which made `apps list` render the header in place of every data row. - Empty iterators still flush their header: the first fetch returns `done=true` with header lines, and the pager prints them before quitting. - Tabwriter computes column widths per-flush and resets them. The pager does the padding itself with widths locked from the first batch, so a short final batch does not compress visually against wider pages above it. History: this consolidates #5016 (shared pager infrastructure) and drops an earlier JSON-output pager. JSON output is mostly consumed by scripts, so paging it adds complexity without a clear win. ## Test plan - [x] `go test ./libs/cmdio/...` passes. Coverage: state-machine unit tests on `pagerModel.Update` (init, batch handling, drain chaining, error propagation, every key path, in-flight-fetch serialization, spinner visibility) plus end-to-end tests via `tea.Program` for full drain, `--limit` integration, header-once, empty iterator, header + rows, cross-batch column stability, and content parity with the non-paged path for single-page lists. - [x] `make checks` passes. - [x] `make lint` passes (0 issues). - [ ] Manual smoke in a TTY: `apps list`, `jobs list`, `clusters list`, `workspace list /`. First page renders after a brief spinner, SPACE fetches next (spinner reappears), ENTER drains (spinner stays up), `Ctrl+C`/`esc`/`q` quit (and interrupt a drain). - [ ] Manual smoke with piped stdout and `--output json`: output unchanged from `main`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
List commands without a row template fall through to the JSON renderer, which today dumps the entire array in one go. For workspaces with hundreds of apps, jobs, pipelines, or files, the output scrolls past before you can read any of it. An interactive terminal should get a chance to step through the output.
This PR adds that pager for the JSON case. A follow-up (#5015) adds the same interaction for commands that do have a row template, reusing the shared infrastructure introduced here.
Changes
Before:
RenderIteratoralways emitted the entire JSON array at once.Now: when stdin, stdout, and stderr are all TTYs and the command has no row template,
databricks <resource> liststreams 50 JSON items at a time and prompts on stderr:SPACEfetches and renders the next page.ENTERdrains the remaining iterator (still interruptible byq/esc/Ctrl+Cbetween pages).q/esc/Ctrl+Cstop immediately. The emitted JSON is always a syntactically valid array — even on early quit — so anything that redirects stdout still gets parseable output.Piped output and redirected stdout keep the existing non-paged behavior: the capability check requires all three streams to be TTYs.
New files under
libs/cmdio/:capabilities.go—SupportsPager()(stdin + stdout + stderr all TTYs, not Git Bash).pager.go— shared plumbing: raw-mode stdin setup with a key-reader goroutine,pagerNextKey/pagerShouldQuit, acrlfWriterto compensate for the terminal's clearedOPOSTflag while raw mode is active, and the prompt/key constants.paged_json.go— the JSON pager itself. Defers writing the opening[until the first item is encoded, so empty iterators and iterators that error before yielding produce valid[].render.go—RenderIteratorroutes to the JSON pager when the capability check passes and no row template is set.No
cmd/changes. No new public API beyondCapabilities.SupportsPager.Test plan
go test ./libs/cmdio/...(all passing, new coverage includescrlfWriter, the key helpers, and every pager control path for JSON output).make checkspasses.make lintfullpasses (0 issues).spacefetches pages,enterdrains,q/esc/Ctrl+Cquit, the on-screen JSON remains valid after any of these.| jq) — output matchesmain.