From e65c5bc05812004479d8ff72ec2edeba52cbf36e Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Tue, 21 Jul 2026 20:45:45 +0200 Subject: [PATCH 01/14] Emoji welcome + a screen copy catalog (golden) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two changes from Lukas's review of the real `tb` output. 1. Emoji are welcome. Reverses the "no emoji" rule โ€” the ๐Ÿ‘‹ greeting, ๐Ÿ’š sign-off, ๐Ÿš€ sent, and the traffic-light/โš  status glyphs are intentional brand warmth. The guard's emoji check is removed (it now enforces only hardcoded brand colour + the 'workspace' term); STYLE.md documents emoji as welcome. 2. testdata/screens.golden โ€” a copy catalog. TestScreensGolden renders every home-view state (+ data list) through the real renderers, colour off, into one committed file, so the exact wording + spacing can be reviewed WITHOUT deploying โ€” read the file, or the diff on any PR that changes copy. The test fails on drift; regenerate with TB_UPDATE_GOLDEN=1 go test -run TestScreensGolden. (It already surfaces one inconsistency: `data list` empty-state says `tracebloc data ingest` while the rest says `tb` โ€” a separate fix.) Co-Authored-By: Claude Opus 4.8 --- STYLE.md | 8 +- internal/cli/screens_golden_test.go | 103 ++++++++++++ internal/cli/testdata/screens.golden | 230 +++++++++++++++++++++++++++ scripts/check-style.sh | 21 +-- 4 files changed, 346 insertions(+), 16 deletions(-) create mode 100644 internal/cli/screens_golden_test.go create mode 100644 internal/cli/testdata/screens.golden diff --git a/STYLE.md b/STYLE.md index 19069087..d544c880 100644 --- a/STYLE.md +++ b/STYLE.md @@ -30,7 +30,9 @@ or hex elsewhere. The tone table (`internal/ui/ui.go`) maps each role: | Error โœ– | `Errorf` (`toneErr`) | red `#f64c4c` | bold glyph | | Label : value | `Field`, `Stat` (`toneLabel`) | dim neutral | โ€” | -**No emoji.** The lime `โ—` is the online indicator (not ๐ŸŸข). +**Emoji are welcome** โ€” used with intent, for warmth (๐Ÿ‘‹ greeting, ๐Ÿ’š sign-off, +๐Ÿš€ sent) and for status (๐ŸŸข online, ๐ŸŸก starting, ๐Ÿ”ด offline, โš  caution). They're a +brand touch, not policed by the guard โ€” just don't overuse them. The engine renders exact 24-bit hex on truecolor terminals, the **deep shade** (`#01637a` / `#578c2b`) on light backgrounds, the nearest ANSI-16 otherwise, and @@ -58,8 +60,8 @@ word in output text only. ## What's enforced vs reviewed `scripts/check-style.sh` (CI Lint job, blocking) catches the **mechanical** -violations: hardcoded brand colour outside `internal/ui`, status emoji, and -`workspace` in user-facing text. Run it locally with `make check-style` (also part +violations: hardcoded brand colour outside `internal/ui`, and `workspace` in +user-facing text. Run it locally with `make check-style` (also part of `make ci`) or directly: `bash scripts/check-style.sh`. It can't police **judgement** โ€” using the right *role* for a token (a command in diff --git a/internal/cli/screens_golden_test.go b/internal/cli/screens_golden_test.go new file mode 100644 index 00000000..a01f4710 --- /dev/null +++ b/internal/cli/screens_golden_test.go @@ -0,0 +1,103 @@ +package cli + +import ( + "bytes" + "os" + "strings" + "testing" + + "github.com/tracebloc/cli/internal/ui" +) + +// TestScreensGolden renders the CLI's user-facing screens through the REAL +// renderers (plain, colour off โ€” so wording and spacing are what you review) and +// pins them in testdata/screens.golden. That file is the one place to read all +// the copy without deploying: open it, or read the diff on any PR that changes it. +// +// Regenerate after an intentional copy change: +// +// TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden +// +// Colour off is deliberate: the palette lives in STYLE.md + the ui tone table +// (pinned by brand_tones_test.go); this catalog is about words and whitespace. +func TestScreensGolden(t *testing.T) { + const goldenPath = "testdata/screens.golden" + + render := func(f func(*ui.Printer)) string { + var b bytes.Buffer + f(ui.New(&b, ui.WithColor(false))) + return b.String() + } + + // The home view in every state resolveHomeModel can produce. compute is shown + // only when Online; inv=tb means a `tb` launcher is installed beside the CLI. + online := homeModel{ + state: homeOnline, email: "lukas@tracebloc.io", name: "Lukas", envName: "hello-world", + compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, + inv: binTB, fullMenu: true, hasResources: true, + } + base := online + base.state, base.hasCompute, base.compute = homeRunning, false, computeInfo{} + + running := base + runningNotOnline := base + runningNotOnline.confirmedNotOnline = true + starting := base + starting.state = homeStarting + offline := base + offline.state = homeOffline + noEnv := base + noEnv.state, noEnv.fullMenu, noEnv.envName = homeNoEnv, false, "" + notSignedIn := homeModel{state: homeNotSignedIn, inv: binTB} + + type screen struct { + title string + render func(*ui.Printer) + } + screens := []screen{ + {"tb โ€” home ยท Online", func(p *ui.Printer) { renderHome(p, online) }}, + {"tb โ€” home ยท running (couldn't confirm connection)", func(p *ui.Printer) { renderHome(p, running) }}, + {"tb โ€” home ยท running (backend reports not online)", func(p *ui.Printer) { renderHome(p, runningNotOnline) }}, + {"tb โ€” home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }}, + {"tb โ€” home ยท offline (can't reach it)", func(p *ui.Printer) { renderHome(p, offline) }}, + {"tb โ€” home ยท no secure environment yet", func(p *ui.Printer) { renderHome(p, noEnv) }}, + {"tb โ€” home ยท not signed in", func(p *ui.Printer) { renderHome(p, notSignedIn) }}, + {"tb data list โ€” empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }}, + } + + var out strings.Builder + out.WriteString("tracebloc CLI โ€” screen copy catalog\n") + out.WriteString("โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•\n") + out.WriteString("The exact wording + spacing of every screen, rendered from the real\n") + out.WriteString("renderers (colour off). Read this instead of deploying to see copy.\n") + out.WriteString("Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") + for _, s := range screens { + dashes := 60 - len(s.title) + if dashes < 0 { + dashes = 0 + } + out.WriteString("\n\nโ”Œโ”€ " + s.title + " " + strings.Repeat("โ”€", dashes) + "\n") + out.WriteString(render(s.render)) + out.WriteString("โ””" + strings.Repeat("โ”€", 62) + "\n") + } + got := out.String() + + if os.Getenv("TB_UPDATE_GOLDEN") != "" { + if err := os.MkdirAll("testdata", 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(goldenPath, []byte(got), 0o644); err != nil { + t.Fatal(err) + } + t.Logf("wrote %s", goldenPath) + return + } + + want, err := os.ReadFile(goldenPath) + if err != nil { + t.Fatalf("read %s (regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden): %v", goldenPath, err) + } + if got != string(want) { + t.Errorf("screen copy drifted from %s.\nRegenerate + review the diff:\n TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden", goldenPath) + } +} diff --git a/internal/cli/testdata/screens.golden b/internal/cli/testdata/screens.golden new file mode 100644 index 00000000..14bbfd67 --- /dev/null +++ b/internal/cli/testdata/screens.golden @@ -0,0 +1,230 @@ +tracebloc CLI โ€” screen copy catalog +โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• +The exact wording + spacing of every screen, rendered from the real +renderers (colour off). Read this instead of deploying to see copy. +Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden + + +โ”Œโ”€ tb โ€” home ยท Online โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โœ“ Secure environment "hello-world" ยท Online (12 CPU ยท 23 GiB) + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท running (couldn't confirm connection) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท running (backend reports not online) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท running, but tracebloc hasn't heard from it โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท starting up โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท starting up, not ready yet โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท offline (can't reach it) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โœ— Secure environment "hello-world" ยท can't reach it from here โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท no secure environment yet โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  No secure environment on this machine yet โ€” run the installer to set one up. + + + Your secure environment + + ยท tb doctor check the connection & diagnose issues + + + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb โ€” home ยท not signed in โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + + Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ— Not signed in yet. + + + Start here + + ยท tb login sign in to tracebloc + + + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ”Œโ”€ tb data list โ€” empty โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + Datasets in hello-world (0) + + No datasets yet โ€” ingest one with `tracebloc data ingest`. +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ diff --git a/scripts/check-style.sh b/scripts/check-style.sh index e767f564..c21f6f13 100755 --- a/scripts/check-style.sh +++ b/scripts/check-style.sh @@ -7,11 +7,12 @@ # locally: make check-style (or: bash scripts/check-style.sh) # Exit 0 = clean, 1 = violations found, 2 = the guard itself errored (fail-closed). # -# Three mechanical checks (semantic calls โ€” role misuse, judgement-y wording โ€” -# stay with CODEOWNERS review + STYLE.md; a grep can't police those): -# 1. No hardcoded brand colour outside the colour engine (internal/ui). -# 2. No status / traffic-light emoji โ€” the lime dot is the online indicator. -# 3. No 'workspace' in user-facing text โ€” the term is "secure environment". +# Two mechanical checks (semantic calls โ€” role misuse, judgement-y wording โ€” +# stay with CODEOWNERS review + STYLE.md; a grep can't police those). Emoji are +# intentionally NOT policed โ€” they're welcome (see STYLE.md): +# 1. No hardcoded brand colour outside the colour engine (internal/ui). New +# output must go through the Printer tones, never a re-hardcoded hex/RGB. +# 2. No 'workspace' in user-facing text โ€” the term is "secure environment". # Matched as a whole word, so the exitNoWorkspace code identifier is exempt; # comments and _test.go files are exempt too. # @@ -35,9 +36,9 @@ hits='' # + opt-out lines removed). grep exit 2+ (bad regex/flags/tree) โ†’ fail closed. scan() { local re="$1" flags="${2:-}" out rc - # shellcheck disable=SC2086 # No 2>/dev/null: let a real grep error surface on stderr โ€” rc>=2 below turns # it into a fail-closed exit, so the error is visible AND fatal, never a silent pass. + # shellcheck disable=SC2086 out="$(grep -rnE $flags --include='*.go' "$re" internal/)" rc=$? if [[ "$rc" -ge 2 ]]; then @@ -64,13 +65,7 @@ scan "$brand" '-i' report "hardcoded brand colour โ€” use the Printer tones in ${ENGINE}, don't re-hardcode hex/RGB" \ "$(printf '%s' "$hits" | grep -vE "^${ENGINE}" || true)" -# 2) Status / traffic-light emoji. Pattern built from bytes so this source stays -# emoji-free (green/red/yellow/orange circles). -emoji="$(printf '\360\237\237\242|\360\237\224\264|\360\237\237\241|\360\237\237\240')" -scan "$emoji" -report "status emoji โ€” use the lime online dot (see STYLE.md), not traffic-light emoji" "$hits" - -# 3) Banned terminology in user-facing text: 'workspace' -> 'secure environment'. +# 2) Banned terminology in user-facing text: 'workspace' -> 'secure environment'. # -w matches whole words only (exitNoWorkspace is exempt); skip comment lines # (content starts with //, anchored to the file:line: prefix). scan 'workspace' '-iw' From 5ed4f88b38411d78bd9259f953ed22ed1db98a0f Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Tue, 21 Jul 2026 21:06:22 +0200 Subject: [PATCH 02/14] =?UTF-8?q?Make=20the=20copy=20catalog=20complete=20?= =?UTF-8?q?=E2=80=94=20commands=20+=20screens=20+=20every=20message?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lukas: the home-only catalog wasn't enough โ€” needs every possible string a user could see. The catalog now has three parts, all in testdata/screens.golden: A. COMMANDS โ€” walks NewRootCmd and captures the `--help` of every command (24 of them): all Short/Long descriptions + flag help, exactly as printed. B. SCREENS โ€” the stateful views rendered plain (13): home in all 7 states, data list empty/populated/--all, client-create review, offboard summaries. C. MESSAGES โ€” a harvested, deduped, sorted index of every user-facing string literal in internal/cli, submit, push, doctor, cluster (501 of them). This catches error paths + the flows not rendered above (ingest validation, login, resources) so nothing a user can see is missing. 1497 lines total. Read the file (or the PR diff) to review any copy without deploying; the test fails on drift, regenerate with TB_UPDATE_GOLDEN=1. Co-Authored-By: Claude Opus 4.8 --- internal/cli/screens_golden_test.go | 188 ++-- internal/cli/testdata/screens.golden | 1321 +++++++++++++++++++++++++- 2 files changed, 1427 insertions(+), 82 deletions(-) diff --git a/internal/cli/screens_golden_test.go b/internal/cli/screens_golden_test.go index a01f4710..c23f3f78 100644 --- a/internal/cli/screens_golden_test.go +++ b/internal/cli/screens_golden_test.go @@ -3,85 +3,117 @@ package cli import ( "bytes" "os" + "path/filepath" + "regexp" + "sort" "strings" "testing" + "github.com/spf13/cobra" + + "github.com/tracebloc/cli/internal/push" "github.com/tracebloc/cli/internal/ui" ) -// TestScreensGolden renders the CLI's user-facing screens through the REAL -// renderers (plain, colour off โ€” so wording and spacing are what you review) and -// pins them in testdata/screens.golden. That file is the one place to read all -// the copy without deploying: open it, or read the diff on any PR that changes it. +// TestScreensGolden pins EVERY piece of user-facing copy in one committed file, +// testdata/screens.golden, so wording + spacing can be reviewed without deploying +// โ€” read the file, or the diff on any PR that changes copy. Three parts: // -// Regenerate after an intentional copy change: +// A. Commands โ€” the `--help` of every command (all Short/Long/flag copy, exact) +// B. Screens โ€” the stateful views rendered plain (home, data list, review, โ€ฆ) +// C. Messages โ€” a harvested, deduped index of every user-facing string in the +// source, so error paths + flows not rendered above are still here. // -// TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden +// The test fails on drift; regenerate after an intentional copy change: // -// Colour off is deliberate: the palette lives in STYLE.md + the ui tone table -// (pinned by brand_tones_test.go); this catalog is about words and whitespace. +// TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden func TestScreensGolden(t *testing.T) { const goldenPath = "testdata/screens.golden" + var cat strings.Builder + + cat.WriteString("tracebloc CLI โ€” complete copy catalog\n") + cat.WriteString("โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•\n") + cat.WriteString("Every user-facing string, generated from the source. Read this (or the\n") + cat.WriteString("diff on any PR that changes it) instead of deploying to review copy.\n") + cat.WriteString("Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") + + // โ”€โ”€ PART A: every command's --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " A. COMMANDS โ€” every `--help` " + strings.Repeat("โ–ˆ", 30) + "\n") + root := NewRootCmd(BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"}) + var walk func(c *cobra.Command) + walk = func(c *cobra.Command) { + var b bytes.Buffer + c.SetOut(&b) + c.SetErr(&b) + c.InitDefaultHelpFlag() + _ = c.Help() + cat.WriteString("\nโ”Œโ”€ " + c.CommandPath() + " --help " + strings.Repeat("โ”€", 40) + "\n") + cat.WriteString(b.String()) + subs := append([]*cobra.Command(nil), c.Commands()...) + sort.Slice(subs, func(i, j int) bool { return subs[i].Name() < subs[j].Name() }) + for _, s := range subs { + if s.Name() != "help" && s.Name() != "completion" { + walk(s) + } + } + } + walk(root) - render := func(f func(*ui.Printer)) string { + // โ”€โ”€ PART B: rendered screens (plain) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " B. SCREENS โ€” rendered plain " + strings.Repeat("โ–ˆ", 31) + "\n") + screen := func(title string, f func(*ui.Printer)) { var b bytes.Buffer f(ui.New(&b, ui.WithColor(false))) - return b.String() + cat.WriteString("\nโ”Œโ”€ " + title + " " + strings.Repeat("โ”€", maxi(0, 55-len(title))) + "\n") + cat.WriteString(b.String()) + cat.WriteString("โ””" + strings.Repeat("โ”€", 60) + "\n") } - // The home view in every state resolveHomeModel can produce. compute is shown - // only when Online; inv=tb means a `tb` launcher is installed beside the CLI. online := homeModel{ state: homeOnline, email: "lukas@tracebloc.io", name: "Lukas", envName: "hello-world", - compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, - inv: binTB, fullMenu: true, hasResources: true, + compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, inv: binTB, fullMenu: true, hasResources: true, } - base := online - base.state, base.hasCompute, base.compute = homeRunning, false, computeInfo{} - - running := base - runningNotOnline := base - runningNotOnline.confirmedNotOnline = true - starting := base + noComp := online + noComp.state, noComp.hasCompute, noComp.compute = homeRunning, false, computeInfo{} + notOnline := noComp + notOnline.confirmedNotOnline = true + starting := noComp starting.state = homeStarting - offline := base + offline := noComp offline.state = homeOffline - noEnv := base + noEnv := noComp noEnv.state, noEnv.fullMenu, noEnv.envName = homeNoEnv, false, "" - notSignedIn := homeModel{state: homeNotSignedIn, inv: binTB} + signedOut := homeModel{state: homeNotSignedIn, inv: binTB} - type screen struct { - title string - render func(*ui.Printer) - } - screens := []screen{ - {"tb โ€” home ยท Online", func(p *ui.Printer) { renderHome(p, online) }}, - {"tb โ€” home ยท running (couldn't confirm connection)", func(p *ui.Printer) { renderHome(p, running) }}, - {"tb โ€” home ยท running (backend reports not online)", func(p *ui.Printer) { renderHome(p, runningNotOnline) }}, - {"tb โ€” home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }}, - {"tb โ€” home ยท offline (can't reach it)", func(p *ui.Printer) { renderHome(p, offline) }}, - {"tb โ€” home ยท no secure environment yet", func(p *ui.Printer) { renderHome(p, noEnv) }}, - {"tb โ€” home ยท not signed in", func(p *ui.Printer) { renderHome(p, notSignedIn) }}, - {"tb data list โ€” empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }}, + screen("tb โ€” home ยท Online", func(p *ui.Printer) { renderHome(p, online) }) + screen("tb โ€” home ยท running (couldn't confirm)", func(p *ui.Printer) { renderHome(p, noComp) }) + screen("tb โ€” home ยท running (backend not online)", func(p *ui.Printer) { renderHome(p, notOnline) }) + screen("tb โ€” home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }) + screen("tb โ€” home ยท offline", func(p *ui.Printer) { renderHome(p, offline) }) + screen("tb โ€” home ยท no secure environment", func(p *ui.Printer) { renderHome(p, noEnv) }) + screen("tb โ€” home ยท not signed in", func(p *ui.Printer) { renderHome(p, signedOut) }) + + sample := []push.DatasetInfo{ + {Name: "xray_train", Intent: "train", Task: "image_classification", Records: 12000, Classes: 2, Extension: "jpg", SizeBytes: 1 << 30}, + {Name: "xray_test", Intent: "test", Task: "image_classification", Records: 3000, Classes: 2, Extension: "jpg", SizeBytes: 256 << 20}, + {Name: "ingest_run_journal", System: true}, } + screen("tb data list โ€” empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }) + screen("tb data list โ€” populated", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) }) + screen("tb data list --all โ€” with system tables", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) }) + screen("tb client create โ€” review", func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") }) + screen("tb delete โ€” offboard summary (keep data)", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) }) + screen("tb delete โ€” offboard summary (remove data)", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) }) - var out strings.Builder - out.WriteString("tracebloc CLI โ€” screen copy catalog\n") - out.WriteString("โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•\n") - out.WriteString("The exact wording + spacing of every screen, rendered from the real\n") - out.WriteString("renderers (colour off). Read this instead of deploying to see copy.\n") - out.WriteString("Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") - for _, s := range screens { - dashes := 60 - len(s.title) - if dashes < 0 { - dashes = 0 - } - out.WriteString("\n\nโ”Œโ”€ " + s.title + " " + strings.Repeat("โ”€", dashes) + "\n") - out.WriteString(render(s.render)) - out.WriteString("โ””" + strings.Repeat("โ”€", 62) + "\n") + // โ”€โ”€ PART C: harvested message index โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " C. MESSAGES โ€” every user-facing string in the source " + strings.Repeat("โ–ˆ", 5) + "\n") + cat.WriteString("(Deduped, sorted. Catches errors, hints, warnings, and flow copy โ€” ingest,\n") + cat.WriteString("login, resources โ€” that isn't a rendered screen above. `%โ€ฆ` are placeholders.)\n\n") + for _, m := range harvestMessages(t) { + cat.WriteString(" " + m + "\n") } - got := out.String() + got := cat.String() if os.Getenv("TB_UPDATE_GOLDEN") != "" { if err := os.MkdirAll("testdata", 0o755); err != nil { t.Fatal(err) @@ -89,15 +121,61 @@ func TestScreensGolden(t *testing.T) { if err := os.WriteFile(goldenPath, []byte(got), 0o644); err != nil { t.Fatal(err) } - t.Logf("wrote %s", goldenPath) + t.Logf("wrote %s (%d bytes)", goldenPath, len(got)) return } - want, err := os.ReadFile(goldenPath) if err != nil { t.Fatalf("read %s (regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden): %v", goldenPath, err) } if got != string(want) { - t.Errorf("screen copy drifted from %s.\nRegenerate + review the diff:\n TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden", goldenPath) + t.Errorf("copy catalog drifted from %s.\nRegenerate + review the diff:\n TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden", goldenPath) + } +} + +func maxi(a, b int) int { + if a > b { + return a + } + return b +} + +// harvestMessages reads the user-facing packages and extracts every string +// literal passed to a Printer method or an error constructor โ€” a complete, +// deduped index of user-facing copy, independent of whether a screen renders it. +func harvestMessages(t *testing.T) []string { + t.Helper() + // Printer method call with a double-quoted first arg, or errors.New / fmt.Errorf. + printer := regexp.MustCompile(`\.(?:Successf|Warnf|Errorf|Infof|Hintf|Detailf|Para|Section|PromptHint|PromptHeader|WarnLine|CrossLine|CheckLine|Step|Action|Stat|Field)\(\s*"((?:[^"\\]|\\.)*)"`) + errs := regexp.MustCompile(`(?:errors\.New|fmt\.Errorf)\(\s*"((?:[^"\\]|\\.)*)"`) + seen := map[string]struct{}{} + // Relative to internal/cli (the test's working dir). + for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { + _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { + if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { + return nil + } + src, err := os.ReadFile(path) + if err != nil { + return nil + } + for _, re := range []*regexp.Regexp{printer, errs} { + for _, m := range re.FindAllStringSubmatch(string(src), -1) { + s := strings.TrimSpace(m[1]) + // Skip empties and format-only fragments (" ", "%s") โ€” no real words. + if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { + continue + } + seen[s] = struct{}{} + } + } + return nil + }) + } + out := make([]string, 0, len(seen)) + for s := range seen { + out = append(out, s) } + sort.Strings(out) + return out } diff --git a/internal/cli/testdata/screens.golden b/internal/cli/testdata/screens.golden index 14bbfd67..ce697375 100644 --- a/internal/cli/testdata/screens.golden +++ b/internal/cli/testdata/screens.golden @@ -1,11 +1,715 @@ -tracebloc CLI โ€” screen copy catalog -โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -The exact wording + spacing of every screen, rendered from the real -renderers (colour off). Read this instead of deploying to see copy. +tracebloc CLI โ€” complete copy catalog +โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• +Every user-facing string, generated from the source. Read this (or the +diff on any PR that changes it) instead of deploying to review copy. Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden -โ”Œโ”€ tb โ€” home ยท Online โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ–ˆโ–ˆโ–ˆ A. COMMANDS โ€” every `--help` โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ + +โ”Œโ”€ tracebloc --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +The tracebloc CLI connects machines to tracebloc as clients and +manages the datasets that models train on. Your data stays on your +infrastructure โ€” models from other collaborators come to it, once you +approve them. + +Two kinds of commands: + + Your account (sign in first): login, logout, auth, client + This machine's client: data, cluster + +A typical first session: + + tracebloc login # sign in or create your account (browser) + tracebloc data ingest ./my-data # stage a dataset into your client + tracebloc data list # see what's in the cluster + +The CLI finds your cluster through your kubeconfig, stages data onto +the cluster's shared storage, and reports progress as it goes. No +Helm, no YAML, no kubectl needed. + +Usage: + tracebloc [flags] + tracebloc [command] + +Available Commands: + auth Inspect tracebloc authentication state + client Provision this machine's tracebloc client + cluster Inspect the cluster the CLI is currently targeting + data Manage the datasets in your secure environment + delete Offboard this machine from tracebloc (revoke, uninstall, reclaim disk) + doctor Check your secure environment is connected and ready to run training + login Sign in to tracebloc in your browser (device flow) + logout Sign out (revoke the token server-side and clear it locally) + resources Show how much of this machine tracebloc may use + version Print the tracebloc CLI version, git SHA, and build date + +Flags: + -h, --help help for tracebloc + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc auth --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Inspect tracebloc authentication state + +Usage: + tracebloc auth [flags] + tracebloc auth [command] + +Available Commands: + status Show whether you're signed in, and to which backend + +Flags: + -h, --help help for auth + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc auth [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc auth status --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Show whether you're signed in, and to which backend + +Usage: + tracebloc auth status [flags] + +Flags: + --check exit 0 only if signed in with a backend-valid token, else 1; silent unless --verbose + --env string backend environment the check targets: dev|stg|prod (default: $CLIENT_ENV, then prod) + -h, --help help for status + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc client --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Provision a tracebloc client for this machine. Requires sign-in first +(`tracebloc login`). To remove tracebloc from this machine, use +`tracebloc delete`. + +Usage: + tracebloc client [flags] + tracebloc client [command] + +Available Commands: + status Show whether tracebloc can see this machine's client (online) + +Flags: + -h, --help help for client + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc client [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc client create --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Provision a tracebloc client for this machine (auto-named; no flags required) + +Usage: + tracebloc client create [flags] + +Flags: + --context string kubeconfig context for the target cluster (default: current-context) + --credential-file string write the machine credential to this path (mode 0600, sourceable env) instead of printing it โ€” for the installer to feed the chart (never shown on the terminal) + -h, --help help for create + --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) โ€” read to anchor the client to this cluster + --location string optional location zone for carbon reporting, e.g. DE (default: $TRACEBLOC_CLIENT_LOCATION; omitted if unset) + --name string client name (default: $TRACEBLOC_CLIENT_NAME, else auto-generated -NN; shown on your dashboard + carbon reports) + --yes skip the confirmation prompt + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc client list --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +List the clients in your account + +Usage: + tracebloc client list [flags] + +Aliases: + list, ls + +Flags: + -h, --help help for list + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc client status --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Report tracebloc's view of this machine's active client โ€” online, offline, +or pending. With --wait, poll until tracebloc reports it online (exit 0) or the +timeout elapses (non-zero), to confirm the client connected after setup. + +Usage: + tracebloc client status [flags] + +Flags: + -h, --help help for status + --timeout duration with --wait, give up after this long (default 2m0s) + --wait poll until tracebloc reports this client online + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc cluster --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Commands for inspecting the Kubernetes cluster the CLI is +configured to talk to. + +Use `cluster info` to verify which cluster, namespace, and +client the next `data ingest` will target. Useful as a +pre-flight before doing anything destructive (e.g. ingesting into +the wrong cluster). + +Usage: + tracebloc cluster [flags] + tracebloc cluster [command] + +Available Commands: + info Show the cluster, namespace, client install, and ingestor token state + +Flags: + -h, --help help for cluster + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc cluster [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc cluster doctor --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Checks, in plain terms, whether your secure environment is connected to +tracebloc and ready to run training โ€” and if something's wrong, exactly what to +do about it. + + --verbose the full technical breakdown (for support) + --diagnose write a redacted support bundle to email to tracebloc + +Exit codes: + 0 healthy + 2 a problem was found + 3 couldn't read your local config + +Usage: + tracebloc cluster doctor [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --diagnose write a redacted support bundle for tracebloc support and exit + -h, --help help for doctor + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your secure environment is installed (default: your active client's) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc cluster info --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Discovers the tracebloc client installed in the configured +cluster + namespace and prints: + + โ€ข Which kubeconfig context the CLI used + โ€ข The namespace it resolved to + โ€ข The client's release name + chart version + appVersion + โ€ข The jobs-manager Service the next data ingest would POST to + โ€ข The ingestor ServiceAccount the post-install hook would auth as + โ€ข The cluster's configured INGESTOR_IMAGE_DIGEST default + โ€ข Whether the user's kubeconfig can mint short-lived SA tokens + via TokenRequest, or has to fall back to a static + service-account-token Secret + +The actual token bytes are never printed; the diagnostic shows +SHA256(token)[:8] so the customer can verify "yes, that's the +token I expect" without leaking it to terminal scrollback. + +Exit codes: + 0 cluster discovered + token mintable; CLI is ready + 4 cluster reachable but no tracebloc client found + 5 cluster reachable + release found but no usable SA token + +Usage: + tracebloc cluster info [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for info + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + --token-expiry-seconds int requested SA token expiration in seconds (default 600 = 10 min; ignored for static-secret fallback) (default 600) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc data --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Commands for ingesting and managing the datasets your secure environment holds โ€” +the data models train on. It stays on your infrastructure. + +`data ingest` ingests a local dataset into your secure environment's storage, +submits the ingestion run, and watches it to completion (streaming +logs + the final summary). `data validate` checks an ingest.yaml +locally first. + +What a dataset looks like depends on the task: + tabular / time-series โ€” a .csv file, or a folder with one .csv + image โ€” a folder with labels.csv + images/ + text โ€” a folder with labels.csv + texts/ + +`tracebloc cluster info` is the pre-flight you'd typically run +before the first ingest. + +Usage: + tracebloc data [flags] + tracebloc data [command] + +Aliases: + data, dataset + +Available Commands: + delete Delete an ingested dataset's in-cluster artifacts (table + PVC files) + ingest Ingest a local dataset into your secure environment + list List datasets ingested in the cluster, with size / records / format + validate Validate an ingest.yaml against the embedded v1 schema, locally + +Flags: + -h, --help help for data + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc data [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc data delete --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Removes the in-cluster artifacts a previous `data ingest` created +for a table: the MySQL table in training_test_datasets and the dataset's +directories on the shared PVC. Destructive and not undoable. + +The dataset's catalog metadata on the tracebloc backend is never removed โ€” it +is kept as a record on tracebloc, marked unavailable, so a collaborator's run +that referenced it still has its history. + +Exit codes: + 0 artifacts removed (or --dry-run, or the user declined) + 2 invalid table name + 3 kubeconfig error, or refused (no confirmation off a terminal) + 4 cluster reachable but no tracebloc client / shared storage missing, + or the client's dataset list couldn't be read (can't confirm the target) + 5 no dataset by that name on this client (nothing to delete) + 7 teardown failed mid-flight (table drop or PVC rm errored) + +With --output-json, stdout carries exactly one JSON result object per run +(human output goes to stderr) and the exit codes above are unchanged; see +docs/json-output.md for the shape and the stability promise. + +Usage: + tracebloc data delete [flags] + +Aliases: + delete, rm + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --dry-run show what would be deleted without deleting anything + -h, --help help for delete + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed + --output-json emit the delete result as JSON on stdout (human output โ†’ stderr; never prompts โ€” pass --yes to delete, or --dry-run) + -y, --yes skip the confirmation prompt (required when not on a terminal) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc data ingest --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Ingests a local dataset into your secure environment's storage, +submits the ingestion run, and follows it to completion (streaming +progress + the final summary). Your data never leaves your own +infrastructure. Supports 16 tasks across the image, text, and +tabular / time-series families; pick one with --task. + + is the data itself. What it looks like depends on the task: + + tabular / time-series โ€” the dataset is a single CSV. Pass the .csv + file directly, or a folder holding exactly one .csv: + + churn.csv (the .csv file itself) + or + churn/ + data.csv (the one .csv in the folder) + + image (classification, object/keypoint detection) โ€” a folder with + labels.csv + an images/ subfolder: + + cats_dogs/ + labels.csv (required) + images/ (required) + 001.jpg + ... + + text (classification, masked language modeling) โ€” a folder with + labels.csv + a texts/ subfolder (masked language modeling uses sequences/): + + reviews/ + labels.csv (required) + texts/ (required โ€” sequences/ for masked language modeling) + 001.txt + ... + +A bare .csv file is accepted only for the tabular / time-series family; +image and text datasets must be a folder. + +Accepted image extensions: .jpg, .jpeg, or .png (case-insensitive). +All images in one dataset must share a single type โ€” the cluster +validates the type it was told to expect. + +v0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger +datasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€” +see tracebloc/client#147 non-goals. + +Exit codes: + 0 files staged + ingested successfully (or --detach: just staged + submitted) + 2 schema validation failed (synthesized spec rejected) or + v0.1-unsupported task passed + 3 local-layout or kubeconfig error + 4 cluster reachable but no tracebloc client / shared storage missing + 5 ingestor SA token couldn't be obtained, or jobs-manager + rejected the token (401/403) + 6 destination table already exists (re-run with --overwrite to + replace it, or pick a different --name) + 7 pre-flight succeeded but staging the files failed + (Pod creation, image pull, exec stream, or remote tar error) โ€” + or, with --overwrite, removing the old table failed + 8 jobs-manager rejected the submit (4xx/5xx other than auth) + 9 ingestion Job exited non-zero, or completed with row-level + failures the summary panel reports + +Usage: + tracebloc data ingest [flags] + +Aliases: + ingest, push + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --detach kubectl logs -f -n job/ exit immediately after jobs-manager accepts the run (no log streaming, no summary panel). Use for CI scenarios; reconnect later with kubectl logs -f -n job/. + --dry-run validate + discover + walk, but don't create any cluster resources + -h, --help help for ingest + --idempotency-key string reuse this idempotency key across retry attempts (default: fresh per invocation). jobs-manager treats a duplicate key as a replay and attaches to the existing Job rather than spawning a new one โ€” useful for at-most-once-across-attempts semantics. + --image-digest images.ingestor.digest pin the ingestor container image to a specific digest (default: jobs-manager picks the cluster-configured images.ingestor.digest). Format: sha256:. + --intent string is this training or test data? train|test (default train) + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + --label-column string name of the label/target column (in labels.csv for image tasks, in the data CSV for tabular) + --label-policy string regression-class only (tabular_regression, time_series_forecasting, time_to_event_prediction): passthrough|bucket (default bucket โ€” bins the target so the raw value never leaves the cluster) + --min-size string image tasks only: reject images smaller than WxH before the ingest (e.g. 64x64). Set it to the smallest size your model can train on โ€” raise or lower it freely. Default: unset (no local size check). + --name string a name for this dataset โ€” start with a letter or underscore, then letters/digits/underscores โ€” you'll reference it by this name when you start a training run + -n, --namespace string namespace where your tracebloc client is installed + --no-input disable interactive prompts; fail on missing required values (for CI/scripts) + --number-of-keypoints int keypoint_detection only: number of keypoints per sample (required; e.g. 17 for COCO pose) + --output-json emit a machine-readable JSON result on stdout (human output โ†’ stderr; implies --no-input) + --overwrite tracebloc data delete replace the destination table if it already exists: its current table + files are removed first (same as tracebloc data delete), then the new data is ingested. Not combinable with --idempotency-key + --schema string tabular/time-series only: column types as col:TYPE,col:TYPE (e.g. age:INT,price:FLOAT). Default: inferred from the CSV (INT/BIGINT/FLOAT/BOOLEAN/DATE/DATETIME/VARCHAR(n)). + --stage-pod-image string override the ephemeral stage Pod's image (default: digest-pinned alpine 3.20 baked into the CLI). Pin by digest in your override too โ€” tag-only refs drift silently. + --target-size string image tasks only: the resolution your images already are, as WxH (e.g. 512x512). tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Default: read from your first image. + --task string the task this data is for, one of: image_classification, object_detection, keypoint_detection, text_classification, masked_language_modeling, tabular_classification, tabular_regression, time_series_forecasting, time_series_classification, time_to_event_prediction, causal_language_modeling, seq2seq, token_classification, sentence_pair_classification, embeddings, semantic_segmentation. Omit it on a terminal to pick interactively. + --time-column string time_to_event_prediction only: name of the time/duration column (default: a column named "time") + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc data list --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Lists the datasets ingested into your client โ€” the tables in training_test_datasets +on the cluster โ€” grouped by modality, with each dataset's split (train/test), +record count, size, format, and when it was ingested. + +With no flags it uses your current kubeconfig context and its namespace; +the flags below override that, same as `cluster info` and `data ingest`. +Framework tables (the ingest-run journal) are hidden unless you pass --all. +For the full catalog, see the dashboard at https://ai.tracebloc.io/metadata. + +Exit codes: + 0 listed successfully (including an empty list) + 3 kubeconfig error + 4 cluster reachable but no tracebloc client in the namespace + 7 couldn't query the cluster for datasets + +Usage: + tracebloc data list [flags] + +Flags: + --all include framework/system tables (e.g. the ingest-run journal), normally hidden + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for list + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed + --output-json emit the dataset list as JSON on stdout (human output โ†’ stderr) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc data validate --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Reads , parses it as YAML, and validates it against the bundled +ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints +violations in the same JSON-pointer-prefixed format the cluster's +jobs-manager uses, and exits non-zero if any are found. + +Useful as a pre-flight before running `tracebloc data ingest` โ€” +millisecond local feedback instead of a multi-second cluster round +trip. + +Exit codes: + 0 YAML parses and validates cleanly + 2 YAML parses but has schema violations (printed to stderr) + 3 YAML doesn't parse or file isn't readable + +Usage: + tracebloc data validate [flags] + +Flags: + -h, --help help for validate + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc delete --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Removes tracebloc from this machine: revokes the machine credential, +uninstalls the Helm release, deletes the local cluster, reclaims the tracebloc +container images, and clears local state โ€” then removes the CLI itself. + +Your use cases, datasets' catalog entries, and the models trained here are KEPT +on tracebloc as a record (a colleague's model must not vanish because you +reclaimed this box). System software the installer laid down โ€” Docker, kubectl, +k3d, helm, NVIDIA drivers โ€” is left in place; remove it yourself if unused. + +Destructive: on a single-host install the on-prem datasets live on this machine +and are erased. Not undoable. + +Usage: + tracebloc delete [flags] + +Flags: + --context string kubeconfig context for the target cluster (default: current-context) + --force offboard even if tracebloc still reports this client online + -h, --help help for delete + --keep-data uninstall the software but keep ~/.tracebloc (local config + on-host datasets) + --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace of this machine's tracebloc release (default: the active client's namespace) + --yes skip the typed-name confirmation (for automation) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc doctor --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Checks, in plain terms, whether your secure environment is connected to +tracebloc and ready to run training โ€” and if something's wrong, exactly what to +do about it. + + --verbose the full technical breakdown (for support) + --diagnose write a redacted support bundle to email to tracebloc + +Exit codes: + 0 healthy + 2 a problem was found + 3 couldn't read your local config + +Usage: + tracebloc doctor [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --diagnose write a redacted support bundle for tracebloc support and exit + -h, --help help for doctor + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your secure environment is installed (default: your active client's) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc ingest --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Deprecated alias for `tracebloc data validate` + +Usage: + tracebloc ingest [flags] + tracebloc ingest [command] + +Available Commands: + validate Validate an ingest.yaml against the embedded v1 schema, locally + +Flags: + -h, --help help for ingest + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc ingest [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc ingest validate --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Reads , parses it as YAML, and validates it against the bundled +ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints +violations in the same JSON-pointer-prefixed format the cluster's +jobs-manager uses, and exits non-zero if any are found. + +Useful as a pre-flight before running `tracebloc data ingest` โ€” +millisecond local feedback instead of a multi-second cluster round +trip. + +Exit codes: + 0 YAML parses and validates cleanly + 2 YAML parses but has schema violations (printed to stderr) + 3 YAML doesn't parse or file isn't readable + +Usage: + tracebloc ingest validate [flags] + +Flags: + -h, --help help for validate + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc login --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Sign in to tracebloc. The CLI prints a URL + short code; open the URL +on any device (your laptop or phone), sign in the way you already do +(password, Google, or GitHub), and approve the code. The CLI stores a +user token in ~/.tracebloc (mode 0600). + +Works on a headless / SSH box โ€” the browser and the CLI need not share a +machine. Honors HTTP(S)_PROXY / NO_PROXY for corporate-proxy networks. + +Usage: + tracebloc login [flags] + +Flags: + --env string backend environment: dev|stg|prod (default: $CLIENT_ENV, then prod) + -h, --help help for login + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc logout --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Sign out (revoke the token server-side and clear it locally) + +Usage: + tracebloc logout [flags] + +Flags: + -h, --help help for logout + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc resources --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Shows, in plain terms, how much of this machine tracebloc may use: + + โ€ข Your secure environment โ€” the CPU and memory it can schedule + โ€ข Each training run โ€” the per-run ceiling every run may use (cluster-wide) + +No Kubernetes concepts, no YAML โ€” one number for your environment and one for +each training run's share of it. + +Raise the share with `tracebloc resources set`. Run with --verbose for the +per-node breakdown and the raw values. + +Exit codes: + 0 shown + 3 kubeconfig could not be loaded / cluster unreachable + 4 cluster reachable but no tracebloc client found here + +Usage: + tracebloc resources [flags] + tracebloc resources [command] + +Available Commands: + set Raise how much of this machine tracebloc may use + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for resources + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc resources [command] --help" for more information about a command. + +โ”Œโ”€ tracebloc resources set --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Raise the per-training-run ceiling โ€” how much of this machine a single +training run may use. + +Run it on a terminal with no flags for a guided walkthrough: + + tracebloc resources set + +Or set it directly (for scripts / non-interactive shells): + + tracebloc resources set --cores 4 --memory 16Gi an explicit per-run ceiling + tracebloc resources set --cores 4 change CPU only, keep the rest + tracebloc resources set max let a run use the whole machine + +The number you set is what ONE training run may use. tracebloc keeps a small fixed +amount (about 1 core and 3 GiB) for itself on top โ€” you never have to subtract it. +The new ceiling applies to your NEXT training run; a run already going keeps its +size. + +Exit codes: + 0 applied (or nothing to change) + 2 the requested size doesn't fit this machine / bad input + 3 kubeconfig could not be loaded / cluster unreachable + 4 cluster reachable but no tracebloc client found here + +Usage: + tracebloc resources set [max] [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --cores string CPU cores one training run may use (e.g. 4) + --dry-run show exactly what would change and apply nothing + --gpus int whole GPUs one training run may use (only on a GPU machine) + -h, --help help for set + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + --memory string memory one training run may use (e.g. 16 or 16Gi โ€” the number is GiB) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + --yes skip the confirmation prompt (for automation) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +โ”Œโ”€ tracebloc version --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +Print the tracebloc CLI version, git SHA, and build date + +Usage: + tracebloc version [flags] + +Flags: + -h, --help help for version + --output-json emit the version payload as indented JSON instead of a single human-readable line + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + + +โ–ˆโ–ˆโ–ˆ B. SCREENS โ€” rendered plain โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ + +โ”Œโ”€ tb โ€” home ยท Online โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -36,10 +740,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ”Œโ”€ tb โ€” home ยท running (couldn't confirm connection) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท running (couldn't confirm) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -70,10 +773,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท running (backend reports not online) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท running (backend not online) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -104,10 +806,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ”Œโ”€ tb โ€” home ยท starting up โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท starting up โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -138,10 +839,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ”Œโ”€ tb โ€” home ยท offline (can't reach it) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท offline โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -172,10 +872,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท no secure environment yet โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท no secure environment โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ @@ -196,10 +895,9 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ”Œโ”€ tb โ€” home ยท not signed in โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb โ€” home ยท not signed in โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ @@ -219,12 +917,581 @@ Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ”Œโ”€ tb data list โ€” empty โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ”Œโ”€ tb data list โ€” empty โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Datasets in hello-world (0) No datasets yet โ€” ingest one with `tracebloc data ingest`. -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +โ”Œโ”€ tb data list โ€” populated โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + Datasets in hello-world โ€” 2 ยท 1.25 GiB + 1 system table(s) hidden โ€” show with --all. + + Image classification ยท 2 + โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” + โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +โ”Œโ”€ tb data list --all โ€” with system tables โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + Datasets in hello-world โ€” 2 ยท 1.25 GiB + + Image classification ยท 2 + โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” + โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” + + System ยท 1 + ยท ingest_run_journal โ€” +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +โ”Œโ”€ tb client create โ€” review โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + Review + name: lukas-macbook + namespace: lukas-macbook + location: DE + cluster: a1b2c3d4 (anchors this client โ€” re-runs adopt it) +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +โ”Œโ”€ tb delete โ€” offboard summary (keep data) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + This will remove + ยท This machine's credential โ€” so tracebloc can no longer reach it + ยท Your secure environment "lukas-macbook" and everything it runs on this machine + ยท tracebloc's downloaded images + ยท The tracebloc CLI (your local data & config are kept โ€” --keep-data) + + Kept on tracebloc + ยท Your use cases and the models trained here + ยท Your dataset records (marked unavailable, not deleted) + + Left alone + ยท Docker and related tools โ€” remove them yourself if you no longer need them +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +โ”Œโ”€ tb delete โ€” offboard summary (remove data) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + This will remove + ยท This machine's credential โ€” so tracebloc can no longer reach it + ยท Your secure environment "lukas-macbook" and everything it runs on this machine + ยท tracebloc's downloaded images + ยท Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone + + Kept on tracebloc + ยท Your use cases and the models trained here + ยท Your dataset records (marked unavailable, not deleted) + + Left alone + ยท Docker and related tools โ€” remove them yourself if you no longer need them +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +โ–ˆโ–ˆโ–ˆ C. MESSAGES โ€” every user-facing string in the source โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ +(Deduped, sorted. Catches errors, hints, warnings, and flow copy โ€” ingest, +login, resources โ€” that isn't a rendered screen above. `%โ€ฆ` are placeholders.) + + %d image(s) are smaller than the %dx%d minimum you set with --min-size: %s. + %d image(s) can't be ingested: %s. The cluster rejects these after the upload โ€” + %d image(s) don't match the %dx%d resolution: %s. The cluster validates the size, + %d labels.csv row(s) reference images that aren't in images/: %s. Those records + %d mask(s) are smaller than the %dx%d minimum you set with --min-size: %s. + %d mask(s) don't match the %dx%d resolution the images use: %s. Semantic-segmentation + %d mask(s) in masks/ can't be ingested: %s. The cluster reads every mask as a PNG + %d row(s) in %s have an empty %q (e.g. %s). Every row must name its mask file โ€” an + %d sequence(s) grouped by %q change their %q value mid-sequence (first offending + %d sequence(s) grouped by %q have out-of-order %q values (first offending data row(s) + %d system table(s) hidden โ€” show with --all. + %q exists but is not a directory + %q is a directory, not a file + %q is a directory, not a file. labels.csv must be the + %q is a symbolic link, which v0.1 does not allow in the dataset + %q is not a .csv file. Tabular / time-series data is a single CSV โ€” + %q is not a directory; pass the directory containing labels.csv + images/ + %q is not a directory; pass the directory containing labels.csv + the text files + %s %q must be WxH (e.g. 512x512) + %s %q: height is not an integer: %w + %s %q: width and height must both be positive + %s %q: width is not an integer: %w + %s %s โ€” %s + %s contains a NUL byte โ€” the file is corrupt or not really a CSV. The cluster + %s has a header but no data rows (0 ingestable records). Add at least one data row and re-run. + %s has duplicate column name(s): %s. Each column must be unique โ€” the cluster + %s is empty โ€” add a header and at least one data row, then re-run + %s is empty โ€” no header row + %s is image tasks only; it doesn't apply to task %q + %s isn't valid UTF-8 (likely a Latin-1/Windows-1252 export). The cluster rejects + %s requires CLIENT_WRITE permission + %s starts with a UTF-8 byte-order mark (Excel's \"CSV UTF-8\" export adds it), + %s ยท Online%s + %s ยท can't reach it from here โ€” run %s + %s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s + %s ยท running, but tracebloc hasn't heard from it โ€” run %s + %s ยท starting up, not ready yet โ€” run %s + %s โ€” %s ยท %s + %s โ€” %s ยท %s (%s) + %s: %w + %w in namespace %q, but tracebloc clients are running in: %s. + %w in namespace %q. + %w on the cluster your kubeconfig points at โ€” if this machine should + --label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column + --label-policy is regression-class tasks only (tabular_regression, + --number-of-keypoints is keypoint_detection only; it doesn't apply to task %q + --number-of-keypoints must be a positive integer (got %d); + --overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default). + --schema is empty; expected col:TYPE,col:TYPE,... + --schema is tabular/time-series tasks only; it doesn't apply to task %q + --schema names column(s) that aren't in %s: %s. The cluster rejects this after the + --time-column is time_to_event_prediction only; it doesn't apply to task %q + --timeout has no effect without --wait + A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train + A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load). + A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it. + A training run is allocated up to: + Add --help to any command for the flags. + Already signed out. + Applies to your next training run; a run already going keeps its size. + Ask one of these admins (or ask them to grant you access) + Available now: + CSV %s has no columns + Can't reach tracebloc from here. + Cancelled โ€” %q was left as-is; nothing was ingested. + Cancelled โ€” nothing was changed. + Cancelled โ€” nothing was deleted. + Cancelled โ€” nothing was ingested. + Cancelled โ€” the name didn't match. Nothing was removed. + Cancelled. + Chart uninstall reported: %v + Check on it later with: kubectl logs -f -n %s job/%s + Check your network / HTTP(S)_PROXY, then run `%s doctor` again. + Client install + Client status + Clients in your account + Cluster teardown reported: %v + Connecting to your secure environmentโ€ฆ + Correlation id: %s + Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you. + Couldn't clear the stored active-client pointer (%v) โ€” the on-disk config + Couldn't connect to your secure environment โ€” check your kubeconfig/context. + Couldn't determine this client's namespace โ€” skipped the Helm uninstall. + Couldn't locate the CLI binary to remove it (%v) โ€” delete it by hand. + Couldn't read the target cluster's identity โ€” provisioning without a cluster anchor, so re-running won't be idempotent. Point --kubeconfig/--context at the reachable cluster to enable that. + Couldn't read your tracebloc config โ€” run `%s login` to recreate it. + Couldn't reclaim the temporary copy (%v). It's harmless โ€” the next re-ingest of %q or a `tracebloc data delete %s` will clear it. + Couldn't remove local data (%v) โ€” cleared the active-client pointer; + Couldn't remove the CLI (%v) โ€” remove it by hand: rm -f %s + Couldn't remove the CLI (%v). It looks Homebrew-managed โ€” finish with: brew uninstall tracebloc + Couldn't remove the `tb` alias (%v) โ€” remove it by hand: rm -f %s + Couldn't revoke the credential server-side (%v) โ€” continuing with local teardown. + Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client) to set it. + Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client). + Couldn't verify your session with the backend (%v). + Couldn't write the support bundle: %v + Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d. + DB failures + Deleted %s.%s and %d PVC path(s). + Destructive and cannot be undone. + Detached โ€” the ingestion runs in the background on your secure environment. + Details + Details (for support) + Diagnose auth / cluster problems with: tracebloc doctor + Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal). + Docker and related tools โ€” remove them yourself if you no longer need them + Dry run โ€” nothing was changed + Dry-run complete โ€” your data and secure environment check out; nothing was created. + Dry-run โ€” nothing was deleted. + Each training run already uses up to %s โ€” nothing to change. + Each training run may now use up to %s. + Email it to support@tracebloc.io. + Enter + Everything looks good โ€” you're ready to run training. + Follow it later with: kubectl logs -f -n %s job/%s + Full log: %s + GPU access removed โ€” training runs will use CPU only. + How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose + How much of this machine a training run may use + Ingest settings + Ingestion complete โ€” %s + Ingestion complete โ€” showing its logs: + Ingestion completed partially โ€” %s + Ingestion completed with failures โ€” %s + Ingestion completed with skips โ€” %s + Ingestion started โ€” live progress: + Ingestion started โ€” streaming logs: + Ingestion summary + Ingestor SA token + Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data. + Kept on tracebloc + Kubeconfig + Learn more: https://docs.tracebloc.io + Left %s in place โ€” it isn't tracebloc's `tb` alias. + Left alone + Let's set up your data ingest + Local dataset + Machine credential โ€” needed by the installer to connect this client + Memory + No client in namespace %q โ€” using the one in %q (override with --namespace). + No clients yet. Run `tracebloc client create`. + No new credential issued; the existing one stands. This machine is set to enroll as client %d. + No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail. + No secure environment on this machine yet โ€” run the installer to set one up. + No secure environment on this machine yet. + Not signed in yet. + Not signed in โ€” run `%s login`. + Not signed in. Run `tracebloc login`. + Not yet in the CLI: + Offboarded %q. This machine is no longer connected to tracebloc. + Offboarded %q: the machine credential is revoked, so it can no longer connect to tracebloc โ€” + Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted. + Open + Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR + POST %s%s: %w + PVC %s/%s is in phase %q, not Bound. + PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter + Pick this dataset when you set it up. + Press Enter to accept a default; Ctrl-C to cancel. + Provisioned client %q (namespace %s). + Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent): + Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away. + Ready for `tracebloc data ingest`. + Reclaimed tracebloc's downloaded images. + Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values. + Removed local tracebloc data and config. + Removed stray control characters from the name. + Removed the local environment. + Removed the old %q โ€” ingesting the new data. + Removed the tracebloc CLI from this machine. + Removing in-cluster artifactsโ€ฆ + Review + Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record. + Set one up: %s + Sign in to tracebloc + Signed in + Signed in as + Signed in as %s + Signed in as %s. + Signed in to %q, but this run targets %q โ€” run `tracebloc login`. + Signed in. + Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine. + Signed out. + Signed-in token was rejected by the backend โ€” run `tracebloc login`. + Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`. + Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`. + Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own. + Stopped watching โ€” the ingestion keeps running on your secure environment. + Submitted โ€” tracebloc is validating your data and loading it into the table. + Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below. + Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below. + Table %q already exists โ€” replacing it (table + files). + Target + Target cluster + The column holding the duration / time-to-event. e.g. time, tenure_days + The column in your CSV with the answer the model learns to produce. + The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed. + The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn + The ingestion hasn't started yet (usually a slow image pull or a busy cluster). + The name you provided was only control characters โ€” auto-naming this client instead. + The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224 + The tracebloc CLI (your local data & config are kept โ€” --keep-data) + This CLI is out of date โ€” update it: %s + This cluster is already registered as client %q (namespace %s) โ€” adopted it. + This drops the table and removes the files listed above โ€” there's no undo. Pass --yes next time to skip this prompt. + This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later). + This is irreversible. Type the client name to confirm, or leave blank to cancel. + This machine's credential โ€” so tracebloc can no longer reach it + This matches a previous run (same idempotency key) โ€” attaching to the run already in progress. + This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account. + This will remove + To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases + Tore down %q on this machine, but some cleanup above didn't complete and the server-side revoke + Tore down %q on this machine. The server-side revoke didn't complete, so the credential may + Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`. + Uninstalled tracebloc. + We couldn't tell the data type from what's there โ€” which is it? + What's next + Whether this split trains the model or evaluates it. + Will delete + Wrote a support bundle to ./%s + Wrote client id + namespace to %s (no new credential โ€” the existing one stands). + You don't have permission to %s in this account. + Your CPU and memory budget is unchanged โ€” but this machine has no GPU while the cluster still requests one, so I'll clear that stale GPU setting so runs can schedule. + Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata + Your dataset records (marked unavailable, not deleted) + Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you. + Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone + Your secure environment %q and everything it runs on this machine + Your secure environment is equipped with: + Your session expired โ€” run `%s login`. + Your use cases and the models trained here + \"active\" is this machine's selected client; state is its last reported status to tracebloc. + `tracebloc ingest` doesn't stage datasets โ€” did you mean:\n + a dataset path is required + a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually + a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read + account + active client + active client %q runs on another machine โ€” namespace %q isn't on the cluster your kubeconfig points at; + active client %s isn't in your account โ€” run `tracebloc client create` + annotations + app version + authorized โ€” confirming the token with the backend โ€ฆ + backend + backend %s โ€” requesting a device code โ€ฆ + backfilling the cluster anchor onto the existing client: %w + building SPDY transport: %w + building rest config from kubeconfig: %w + building submit request: %w + building tar archive: %w + can't confirm %q exists on this client โ€” refusing to delete without + can't read %q: %w + cancelled by user + chart version + checking %s for a byte-order mark: %w + client + client id + closing tar writer: %w + cluster + column %q isn't all %s: %d value(s) don't match its declared type (e.g. %s). + columns + command + connected: %s โ€” %s + constructing kubernetes clientset: %w + context + couldn't check whether a tracebloc client is already running on this cluster (%w) โ€” + couldn't determine the installed client chart version (the release is missing + couldn't reach the backend to choose a unique client name (%v) โ€” retry, + couldn't read capacity: %v + couldn't read the account's client list to tell whether this cluster is new + couldn't read this machine's capacity: %w + creating SPDY executor for %s/%s: %w + creating credential-file directory: %w + creating port-forwarder: %w + creating stage Pod in namespace %q: %w + creating staging-cleanup pod: %w + creating teardown pod: %w + dashboard id + data CSV + dataset exceeded v0.1 total cap of %s after streaming %s + dataset exceeded v0.1 total cap of %s after streaming %s (reached %s) + dataset exceeded v0.1 total cap of %s during stream + dataset is %s, exceeds v0.1 cap of %s. + dataset is %s, exceeds v0.1 cap of %s. For larger datasets, the + dataset name is required (set --name) + dataset name is required โ€” pass it as an argument: tracebloc data delete + decoding image header %q: %w + decoding submit response (got body %q): %w + deleting stage Pod %s/%s: %w + destination + dropping %s.%s: %w%s + enter a whole number between %d and %d + exec stream against %s/%s: %w + expected %s %s-separated fields (%s), found %d. + expected a single %s record but the file spans multiple lines. + expires + expires in + field %d is empty โ€” every field (%s) must be non-empty + file %q is %s, exceeds v0.1 single-file cap of %s. + file failures + found %d .csv files in %q (%s); the tabular layout expects exactly one. + found %d tracebloc clients in namespace %q (%s); + full log: %s + generating Pod-name random suffix: %w + generating idempotency key: %w + generating staging-dir suffix: %w + images + images/ and annotations/ don't pair up: %s. Every image needs a same-named .xml + images/ and masks/ don't pair up: %s. Every image needs a same-named + inferring schema from CSV: %w + ingestion Job completed but the summary reports failures โ€” see panel above + ingestion Job exited non-zero โ€” see logs above + ingestion Job's final status couldn't be determined within the watch window โ€” + ingestor ID + ingestor SA + ingestor img + inserted + intent + interactive setup: %w + internal: re-parsing synthesized spec: %w\n%s + invalid table name %q: %w + jobs-manager + jobs-manager: %s + keypoint_detection requires --number-of-keypoints (e.g. + keypoints + kube-system namespace has no UID + label column + label column %q isn't in %s's header (columns: %s). Pass --label-column with one of + label policy + labels.csv + labels.csv column %q must be lowercase \"filename\": the cluster reads + labels.csv has no \"filename\" column (columns: %s) โ€” image tasks match each + labels.csv has no \"filename\" column (columns: %s) โ€” the ingestor matches each + listing Pods for service %s/%s: %w + listing chart-managed deployments in namespace %s: %w + listing client deployments to check for an existing client: %w + listing service-account-token secrets in %s: %w + listing stage Pods in %s: %w + loading embedded schema: %w + loading kubeconfig: %w + local dataset path is required โ€” pass it as an argument, or run + locating mysql pod: %w + location + login timed out โ€” re-run `tracebloc login` + love from tracebloc ๐Ÿ’š + marshaling submit request: %w + marshaling synthesized spec: %w + masks + min size + minting token for ServiceAccount %s/%s via TokenRequest: %w + missing %s/ subdirectory in %q + missing images/ subdirectory in %q. The CLI expects + missing labels.csv in %q. Text categories expect + missing labels.csv in %q. The CLI expects + must be a positive integer + must be between %d and %d + mysql table + name + namespace + no .csv file found in %q. Tabular / time-series categories expect a + no .png mask files found in %q. semantic_segmentation expects + no .txt files found in %q. Text categories expect + no .xml annotation files found in %q. object_detection expects + no CLI-supported tasks for %s data yet + no PersistentVolumeClaim named %q found in namespace %q. + no Ready node on this machine to size a training run against + no Running Pod backing service %s/%s (found %d Pod(s); + no Running pod with name containing %q in namespace %q + no Secret of type kubernetes.io/service-account-token bound to + no active client on this machine โ€” nothing to offboard + no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first + no dataset named %q on this client%s + no image files found in %q. Expected .jpg, .jpeg, or .png; + no image files to detect a type from + no such file or directory: %q โ€” check the path to your dataset + no tracebloc client found + no usable image files in %q โ€” found %s, but the ingestor + no usable ingestor token. TokenRequest failed: %v. + not signed in โ€” run `tracebloc login` first + outcome: early exit before the cluster was probed + outcome: early exit โ€” no roll-up verdict (granular checks below) + overwrite prompt: %w + packaging %s: %w + packaging labels.csv: %w + password + path + port-forward allocated zero ports + port-forward to %s/%s failed during startup: %w + pvc path + querying datasets: %w%s + reading %q: %w + reading %s header: %w + reading %s/: %w + reading %s: %w + reading CSV header from %s: %w + reading CSV row from %s: %w + reading PVC %s/%s: %w + reading allocated port: %w + reading dataset directory %q: %w + reading dataset path %q: %w + reading final Job status for %s/%s: %w + reading images/: %w + reading kube-system namespace UID: %w + reading labels.csv: %w + reading raw kubeconfig: %w + reading service %s/%s: %w + reading submit response body: %w + reading the existing client's identity in namespace %q: %w + ready: %s โ€” %s + refusing to change the ceiling without confirmation: pass --yes, or run on a terminal + refusing to delete without confirmation: pass --yes or run on a terminal + refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name + refusing to provision non-interactively without confirmation โ€” pass --yes to + refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover) + release + release: %s (chart %s) + removing PVC paths: %w%s + removing staged copy %s: %w%s + replacing table %q failed partway โ€” its removal may be incomplete, and a plain re-run + resolution + resolving %q: %w + resolving Service %s/%s to a Pod: %w + resolving namespace from kubeconfig: %w + resource env + root + scanning the cluster for tracebloc clients: %w + schema + schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT) + schema type %q for column %q isn't a supported SQL type โ€” the ingestor would + semantic_segmentation needs a %q column in %s (columns: %s) linking each image to its + semantic_segmentation needs a %q column in %s, but found %q (wrong case). + sent to API + server + service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding + session: %s + setting up jobs-manager port-forward: %w + sha256[:8] + shared PVC + shared PVC: %s (%s) + sign-in was denied in the browser + skipped + source + stage Pod %s/%s did not become Ready within %s%s + stage Pod %s/%s did not reach Ready state: %w%s + stage Pod %s/%s terminated in phase %q before becoming Ready%s + stat %q: %w + stat %s/: %w + stat %s: %w + stat images/: %w + stat labels.csv: %w + state + status + stored active client id %q is not numeric: %w + streaming files to %s/%s: %w%s + streaming logs from Pod %s/%s: %w + submit response missing job_name (got body %q) + submit response missing namespace (got body %q) + success rate + synthesized spec failed schema validation; check the flag values above + table %q already exists in this secure environment. Re-ingesting the same table doesn't merge or replace โ€” + table name %q is invalid: must start with a letter or + table name is %d characters; the max is %d + task + task %q isn't a recognized task. Supported tasks: %s. + task %q isn't supported by the CLI yet%s. Supported tasks: %s. + teardown failed: %w + teardown incomplete โ€” the table %s.%s was dropped, but removing its files failed: %w; + the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster + the label column %q has %d distinct value(s) โ€” a classification dataset needs at + the sequence column %q has %d empty/null value(s) (first at data row %d). Every + the sign-in code expired โ€” re-run `tracebloc login` + the time column %q has %d missing/invalid value(s) (first at data row(s) %v%s). + this backend (%s) doesn't support browser login yet โ€” the device-grant + this machine is too small to choose an amount โ€” after tracebloc's ~1 core and 3 GiB + this task's data is sequence-grouped: the schema must declare %q (groups the timestep + time column + timed out after %s before tracebloc could confirm this client โ€” retry, + timed out after %s waiting for tracebloc to report this client online (last state: %s). + timed out after %s waiting for tracebloc to report this client online; + token saved to ~/.tracebloc (0600) + total records + total size + tracebloc auth + tracebloc can see this client. + tracebloc didn't confirm your session (server error). + tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine. + tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry + tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete` + tracebloc's downloaded images + unknown backend environment %q โ€” valid values are dev, stg, prod (default). + values: + waiting for ingestor Pod: %w + waiting for staging-cleanup pod: %w + waiting for teardown pod: %w + watching ingestor Job: %w + which task is this data for? pass --task โ€” one of: %s. + would set each run to + writing credential file %s: %w + your images mix file types (%s) โ€” the ingestor requires one type + your secure environment %q has %d training run(s) active โ€” offboarding would stop them. From 99d252cba81ba9e3b3b222b871b7fbbbea6af4dc Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Tue, 21 Jul 2026 21:22:01 +0200 Subject: [PATCH 03/14] Make the catalog a byte-exact terminal transcript MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lukas: the wording was right but the layout wasn't โ€” the catalog inserted blank lines the terminal doesn't show. Now each entry is `$ ` followed by the BYTE-EXACT output โ€” line breaks, tabs, leading/trailing blanks all as printed: - Help is captured through the real `--help` flag path (SetArgs+Execute, exactly what the binary runs), not c.Help(). - Screens render through the real renderers (same code the binary runs). - No framing, no inserted whitespace: the only blank lines in the file are the outputs' own. `$ tracebloc --help` now starts its output on the very next line, and consecutive entries sit back-to-back like a real terminal session. The MESSAGE INDEX appendix stays (raw templates for search). Regenerate with TB_UPDATE_GOLDEN=1. Co-Authored-By: Claude Opus 4.8 --- internal/cli/screens_golden_test.go | 115 ++++++++++----------- internal/cli/testdata/screens.golden | 147 ++++++++++----------------- 2 files changed, 110 insertions(+), 152 deletions(-) diff --git a/internal/cli/screens_golden_test.go b/internal/cli/screens_golden_test.go index c23f3f78..23082e32 100644 --- a/internal/cli/screens_golden_test.go +++ b/internal/cli/screens_golden_test.go @@ -16,57 +16,70 @@ import ( ) // TestScreensGolden pins EVERY piece of user-facing copy in one committed file, -// testdata/screens.golden, so wording + spacing can be reviewed without deploying -// โ€” read the file, or the diff on any PR that changes copy. Three parts: +// testdata/screens.golden, so wording AND exact layout (line breaks, tabs, +// leading/trailing blanks) can be reviewed without deploying โ€” read the file, or +// the diff on any PR that changes copy. // -// A. Commands โ€” the `--help` of every command (all Short/Long/flag copy, exact) -// B. Screens โ€” the stateful views rendered plain (home, data list, review, โ€ฆ) -// C. Messages โ€” a harvested, deduped index of every user-facing string in the -// source, so error paths + flows not rendered above are still here. +// It's a verbatim terminal transcript: each block is `$ ` followed by +// the BYTE-EXACT output. Help is captured through the real `--help` flag path +// (SetArgs+Execute โ€” exactly what the binary runs), screens through the real +// renderers. Then an appendix indexes every remaining user-facing string. // // The test fails on drift; regenerate after an intentional copy change: // // TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden func TestScreensGolden(t *testing.T) { const goldenPath = "testdata/screens.golden" + bi := BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"} var cat strings.Builder cat.WriteString("tracebloc CLI โ€” complete copy catalog\n") - cat.WriteString("โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•\n") - cat.WriteString("Every user-facing string, generated from the source. Read this (or the\n") - cat.WriteString("diff on any PR that changes it) instead of deploying to review copy.\n") + cat.WriteString("A verbatim transcript: each `$ command` is followed by its byte-exact output\n") + cat.WriteString("(line breaks, tabs, blank lines โ€” all as the terminal prints them).\n") cat.WriteString("Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") - // โ”€โ”€ PART A: every command's --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " A. COMMANDS โ€” every `--help` " + strings.Repeat("โ–ˆ", 30) + "\n") - root := NewRootCmd(BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"}) - var walk func(c *cobra.Command) - walk = func(c *cobra.Command) { - var b bytes.Buffer - c.SetOut(&b) - c.SetErr(&b) - c.InitDefaultHelpFlag() - _ = c.Help() - cat.WriteString("\nโ”Œโ”€ " + c.CommandPath() + " --help " + strings.Repeat("โ”€", 40) + "\n") - cat.WriteString(b.String()) + // block writes `$ \n` then the output VERBATIM โ€” nothing else. The only + // whitespace between entries is each output's own real leading/trailing blank + // lines, so what you read is byte-for-byte what the terminal prints. + block := func(cmd, output string) { + cat.WriteString("$ " + cmd + "\n") + cat.WriteString(output) + } + + // โ”€โ”€ PART A: every command's --help, through the real flag path โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= COMMANDS โ€” every `--help`, byte-exact\n" + strings.Repeat("=", 78) + "\n") + // Enumerate every command path (incl. hidden โ€” a user can still run them). + var paths [][]string + var walk func(c *cobra.Command, prefix []string) + walk = func(c *cobra.Command, prefix []string) { + paths = append(paths, prefix) subs := append([]*cobra.Command(nil), c.Commands()...) sort.Slice(subs, func(i, j int) bool { return subs[i].Name() < subs[j].Name() }) for _, s := range subs { - if s.Name() != "help" && s.Name() != "completion" { - walk(s) + if s.Name() == "help" || s.Name() == "completion" { + continue } + child := append(append([]string{}, prefix...), s.Name()) + walk(s, child) } } - walk(root) + walk(NewRootCmd(bi), nil) + for _, p := range paths { + var b bytes.Buffer + r := NewRootCmd(bi) + r.SetOut(&b) + r.SetErr(&b) + r.SetArgs(append(append([]string{}, p...), "--help")) + _ = r.Execute() + block(strings.TrimSpace("tracebloc "+strings.Join(p, " "))+" --help", b.String()) + } - // โ”€โ”€ PART B: rendered screens (plain) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " B. SCREENS โ€” rendered plain " + strings.Repeat("โ–ˆ", 31) + "\n") - screen := func(title string, f func(*ui.Printer)) { + // โ”€โ”€ PART B: screens, rendered verbatim (same code the binary runs) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= SCREENS โ€” byte-exact renderer output\n" + strings.Repeat("=", 78) + "\n") + render := func(cmd string, f func(*ui.Printer)) { var b bytes.Buffer f(ui.New(&b, ui.WithColor(false))) - cat.WriteString("\nโ”Œโ”€ " + title + " " + strings.Repeat("โ”€", maxi(0, 55-len(title))) + "\n") - cat.WriteString(b.String()) - cat.WriteString("โ””" + strings.Repeat("โ”€", 60) + "\n") + block(cmd, b.String()) } online := homeModel{ @@ -85,30 +98,28 @@ func TestScreensGolden(t *testing.T) { noEnv.state, noEnv.fullMenu, noEnv.envName = homeNoEnv, false, "" signedOut := homeModel{state: homeNotSignedIn, inv: binTB} - screen("tb โ€” home ยท Online", func(p *ui.Printer) { renderHome(p, online) }) - screen("tb โ€” home ยท running (couldn't confirm)", func(p *ui.Printer) { renderHome(p, noComp) }) - screen("tb โ€” home ยท running (backend not online)", func(p *ui.Printer) { renderHome(p, notOnline) }) - screen("tb โ€” home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }) - screen("tb โ€” home ยท offline", func(p *ui.Printer) { renderHome(p, offline) }) - screen("tb โ€” home ยท no secure environment", func(p *ui.Printer) { renderHome(p, noEnv) }) - screen("tb โ€” home ยท not signed in", func(p *ui.Printer) { renderHome(p, signedOut) }) + render("tb # home ยท Online", func(p *ui.Printer) { renderHome(p, online) }) + render("tb # home ยท running (couldn't confirm)", func(p *ui.Printer) { renderHome(p, noComp) }) + render("tb # home ยท running (backend not online)", func(p *ui.Printer) { renderHome(p, notOnline) }) + render("tb # home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }) + render("tb # home ยท offline", func(p *ui.Printer) { renderHome(p, offline) }) + render("tb # home ยท no secure environment", func(p *ui.Printer) { renderHome(p, noEnv) }) + render("tb # home ยท not signed in", func(p *ui.Printer) { renderHome(p, signedOut) }) sample := []push.DatasetInfo{ {Name: "xray_train", Intent: "train", Task: "image_classification", Records: 12000, Classes: 2, Extension: "jpg", SizeBytes: 1 << 30}, {Name: "xray_test", Intent: "test", Task: "image_classification", Records: 3000, Classes: 2, Extension: "jpg", SizeBytes: 256 << 20}, {Name: "ingest_run_journal", System: true}, } - screen("tb data list โ€” empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }) - screen("tb data list โ€” populated", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) }) - screen("tb data list --all โ€” with system tables", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) }) - screen("tb client create โ€” review", func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") }) - screen("tb delete โ€” offboard summary (keep data)", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) }) - screen("tb delete โ€” offboard summary (remove data)", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) }) + render("tb data list # empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }) + render("tb data list # populated", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) }) + render("tb data list --all", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) }) + render("tb client create # review", func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") }) + render("tb delete # keep data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) }) + render("tb delete # remove data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) }) - // โ”€โ”€ PART C: harvested message index โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("โ–ˆ", 3) + " C. MESSAGES โ€” every user-facing string in the source " + strings.Repeat("โ–ˆ", 5) + "\n") - cat.WriteString("(Deduped, sorted. Catches errors, hints, warnings, and flow copy โ€” ingest,\n") - cat.WriteString("login, resources โ€” that isn't a rendered screen above. `%โ€ฆ` are placeholders.)\n\n") + // โ”€โ”€ APPENDIX: every remaining user-facing string โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= MESSAGE INDEX โ€” every user-facing string in the source (templates, not\n= rendered; %s/%d are runtime placeholders). Catches errors, hints, and the\n= flows not rendered above (ingest validation, login, resources).\n" + strings.Repeat("=", 78) + "\n\n") for _, m := range harvestMessages(t) { cat.WriteString(" " + m + "\n") } @@ -133,23 +144,14 @@ func TestScreensGolden(t *testing.T) { } } -func maxi(a, b int) int { - if a > b { - return a - } - return b -} - // harvestMessages reads the user-facing packages and extracts every string // literal passed to a Printer method or an error constructor โ€” a complete, // deduped index of user-facing copy, independent of whether a screen renders it. func harvestMessages(t *testing.T) []string { t.Helper() - // Printer method call with a double-quoted first arg, or errors.New / fmt.Errorf. printer := regexp.MustCompile(`\.(?:Successf|Warnf|Errorf|Infof|Hintf|Detailf|Para|Section|PromptHint|PromptHeader|WarnLine|CrossLine|CheckLine|Step|Action|Stat|Field)\(\s*"((?:[^"\\]|\\.)*)"`) errs := regexp.MustCompile(`(?:errors\.New|fmt\.Errorf)\(\s*"((?:[^"\\]|\\.)*)"`) seen := map[string]struct{}{} - // Relative to internal/cli (the test's working dir). for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { @@ -162,7 +164,6 @@ func harvestMessages(t *testing.T) []string { for _, re := range []*regexp.Regexp{printer, errs} { for _, m := range re.FindAllStringSubmatch(string(src), -1) { s := strings.TrimSpace(m[1]) - // Skip empties and format-only fragments (" ", "%s") โ€” no real words. if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { continue } diff --git a/internal/cli/testdata/screens.golden b/internal/cli/testdata/screens.golden index ce697375..767b75ec 100644 --- a/internal/cli/testdata/screens.golden +++ b/internal/cli/testdata/screens.golden @@ -1,13 +1,13 @@ tracebloc CLI โ€” complete copy catalog -โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ• -Every user-facing string, generated from the source. Read this (or the -diff on any PR that changes it) instead of deploying to review copy. +A verbatim transcript: each `$ command` is followed by its byte-exact output +(line breaks, tabs, blank lines โ€” all as the terminal prints them). Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden -โ–ˆโ–ˆโ–ˆ A. COMMANDS โ€” every `--help` โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ - -โ”Œโ”€ tracebloc --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +============================================================================== += COMMANDS โ€” every `--help`, byte-exact +============================================================================== +$ tracebloc --help The tracebloc CLI connects machines to tracebloc as clients and manages the datasets that models train on. Your data stays on your infrastructure โ€” models from other collaborators come to it, once you @@ -36,9 +36,11 @@ Available Commands: auth Inspect tracebloc authentication state client Provision this machine's tracebloc client cluster Inspect the cluster the CLI is currently targeting + completion Generate the autocompletion script for the specified shell data Manage the datasets in your secure environment delete Offboard this machine from tracebloc (revoke, uninstall, reclaim disk) doctor Check your secure environment is connected and ready to run training + help Help about any command login Sign in to tracebloc in your browser (device flow) logout Sign out (revoke the token server-side and clear it locally) resources Show how much of this machine tracebloc may use @@ -50,8 +52,7 @@ Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc auth --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc auth --help Inspect tracebloc authentication state Usage: @@ -69,8 +70,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc auth [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc auth status --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc auth status --help Show whether you're signed in, and to which backend Usage: @@ -84,8 +84,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc client --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc client --help Provision a tracebloc client for this machine. Requires sign-in first (`tracebloc login`). To remove tracebloc from this machine, use `tracebloc delete`. @@ -105,8 +104,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc client [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc client create --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc client create --help Provision a tracebloc client for this machine (auto-named; no flags required) Usage: @@ -124,8 +122,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc client list --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc client list --help List the clients in your account Usage: @@ -140,8 +137,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc client status --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc client status --help Report tracebloc's view of this machine's active client โ€” online, offline, or pending. With --wait, poll until tracebloc reports it online (exit 0) or the timeout elapses (non-zero), to confirm the client connected after setup. @@ -157,8 +153,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc cluster --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc cluster --help Commands for inspecting the Kubernetes cluster the CLI is configured to talk to. @@ -182,8 +177,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc cluster [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc cluster doctor --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc cluster doctor --help Checks, in plain terms, whether your secure environment is connected to tracebloc and ready to run training โ€” and if something's wrong, exactly what to do about it. @@ -209,8 +203,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc cluster info --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc cluster info --help Discovers the tracebloc client installed in the configured cluster + namespace and prints: @@ -246,8 +239,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc data --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc data --help Commands for ingesting and managing the datasets your secure environment holds โ€” the data models train on. It stays on your infrastructure. @@ -285,8 +277,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc data [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc data delete --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc data delete --help Removes the in-cluster artifacts a previous `data ingest` created for a table: the MySQL table in training_test_datasets and the dataset's directories on the shared PVC. Destructive and not undoable. @@ -326,8 +317,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc data ingest --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc data ingest --help Ingests a local dataset into your secure environment's storage, submits the ingestion run, and follows it to completion (streaming progress + the final summary). Your data never leaves your own @@ -423,8 +413,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc data list --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc data list --help Lists the datasets ingested into your client โ€” the tables in training_test_datasets on the cluster โ€” grouped by modality, with each dataset's split (train/test), record count, size, format, and when it was ingested. @@ -454,8 +443,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc data validate --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc data validate --help Reads , parses it as YAML, and validates it against the bundled ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints violations in the same JSON-pointer-prefixed format the cluster's @@ -479,8 +467,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc delete --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc delete --help Removes tracebloc from this machine: revokes the machine credential, uninstalls the Helm release, deletes the local cluster, reclaims the tracebloc container images, and clears local state โ€” then removes the CLI itself. @@ -508,8 +495,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc doctor --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc doctor --help Checks, in plain terms, whether your secure environment is connected to tracebloc and ready to run training โ€” and if something's wrong, exactly what to do about it. @@ -535,8 +521,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc ingest --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc ingest --help Deprecated alias for `tracebloc data validate` Usage: @@ -554,8 +539,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc ingest [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc ingest validate --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc ingest validate --help Reads , parses it as YAML, and validates it against the bundled ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints violations in the same JSON-pointer-prefixed format the cluster's @@ -579,8 +563,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc login --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc login --help Sign in to tracebloc. The CLI prints a URL + short code; open the URL on any device (your laptop or phone), sign in the way you already do (password, Google, or GitHub), and approve the code. The CLI stores a @@ -599,8 +582,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc logout --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc logout --help Sign out (revoke the token server-side and clear it locally) Usage: @@ -612,8 +594,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc resources --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc resources --help Shows, in plain terms, how much of this machine tracebloc may use: โ€ข Your secure environment โ€” the CPU and memory it can schedule @@ -648,8 +629,7 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) Use "tracebloc resources [command] --help" for more information about a command. - -โ”Œโ”€ tracebloc resources set --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc resources set --help Raise the per-training-run ceiling โ€” how much of this machine a single training run may use. @@ -691,8 +671,7 @@ Flags: Global Flags: --plain disable color and decorative output (also honors $NO_COLOR) --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -โ”Œโ”€ tracebloc version --help โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tracebloc version --help Print the tracebloc CLI version, git SHA, and build date Usage: @@ -707,9 +686,10 @@ Global Flags: --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -โ–ˆโ–ˆโ–ˆ B. SCREENS โ€” rendered plain โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ - -โ”Œโ”€ tb โ€” home ยท Online โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +============================================================================== += SCREENS โ€” byte-exact renderer output +============================================================================== +$ tb # home ยท Online Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -740,9 +720,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท running (couldn't confirm) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท running (couldn't confirm) Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -773,9 +751,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท running (backend not online) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท running (backend not online) Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -806,9 +782,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท starting up โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท starting up Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -839,9 +813,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท offline โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท offline Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ @@ -872,9 +844,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท no secure environment โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท no secure environment Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ @@ -895,9 +865,7 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb โ€” home ยท not signed in โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb # home ยท not signed in Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ @@ -917,16 +885,12 @@ Global Flags: love from tracebloc ๐Ÿ’š -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb data list โ€” empty โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb data list # empty Datasets in hello-world (0) No datasets yet โ€” ingest one with `tracebloc data ingest`. -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb data list โ€” populated โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb data list # populated Datasets in hello-world โ€” 2 ยท 1.25 GiB 1 system table(s) hidden โ€” show with --all. @@ -934,9 +898,7 @@ Global Flags: Image classification ยท 2 โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb data list --all โ€” with system tables โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb data list --all Datasets in hello-world โ€” 2 ยท 1.25 GiB @@ -946,18 +908,14 @@ Global Flags: System ยท 1 ยท ingest_run_journal โ€” -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb client create โ€” review โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb client create # review Review name: lukas-macbook namespace: lukas-macbook location: DE cluster: a1b2c3d4 (anchors this client โ€” re-runs adopt it) -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb delete โ€” offboard summary (keep data) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb delete # keep data This will remove ยท This machine's credential โ€” so tracebloc can no longer reach it @@ -971,9 +929,7 @@ Global Flags: Left alone ยท Docker and related tools โ€” remove them yourself if you no longer need them -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - -โ”Œโ”€ tb delete โ€” offboard summary (remove data) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ +$ tb delete # remove data This will remove ยท This machine's credential โ€” so tracebloc can no longer reach it @@ -987,12 +943,13 @@ Global Flags: Left alone ยท Docker and related tools โ€” remove them yourself if you no longer need them -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ -โ–ˆโ–ˆโ–ˆ C. MESSAGES โ€” every user-facing string in the source โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ -(Deduped, sorted. Catches errors, hints, warnings, and flow copy โ€” ingest, -login, resources โ€” that isn't a rendered screen above. `%โ€ฆ` are placeholders.) +============================================================================== += MESSAGE INDEX โ€” every user-facing string in the source (templates, not += rendered; %s/%d are runtime placeholders). Catches errors, hints, and the += flows not rendered above (ingest validation, login, resources). +============================================================================== %d image(s) are smaller than the %dx%d minimum you set with --min-size: %s. %d image(s) can't be ingested: %s. The cluster rejects these after the upload โ€” From 567b26c38d2a4f3dc4be390131934924fc6da0d9 Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Tue, 21 Jul 2026 21:37:26 +0200 Subject: [PATCH 04/14] Catalog: add the ingest review screen + harvest every string via AST MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lukas: `tb data ingest` output wasn't findable. Two fixes: - Add the ingest pre-flight `Review` screen (renderReview) to the SCREENS section. - Replace the first-arg regex message-harvest with a go/ast walk that captures EVERY string argument to a Printer method or error constructor โ€” so labels that aren't the first arg (Step("โ€ฆ","โ€ฆ") titles, MenuRow descriptions, Field values) are now included. E.g. "Copy into your secure environment" (an ingest Step label) was missing before; it's in the index now. Honest limit: the assembled step-by-step RUN transcript of a live flow (ingest steps + progress, login device flow) still isn't a single rendered screen โ€” those need the flow driven with a mocked cluster. But every STRING they print is now in the MESSAGE INDEX, and the review screen is rendered. Can add driven flow transcripts next if wanted. Co-Authored-By: Claude Opus 4.8 --- internal/cli/screens_golden_test.go | 99 ++- internal/cli/testdata/screens.golden | 936 ++++++++++++--------------- 2 files changed, 501 insertions(+), 534 deletions(-) diff --git a/internal/cli/screens_golden_test.go b/internal/cli/screens_golden_test.go index 23082e32..9bfd2db2 100644 --- a/internal/cli/screens_golden_test.go +++ b/internal/cli/screens_golden_test.go @@ -2,10 +2,13 @@ package cli import ( "bytes" + "go/ast" + "go/parser" + "go/token" "os" "path/filepath" - "regexp" "sort" + "strconv" "strings" "testing" @@ -17,13 +20,14 @@ import ( // TestScreensGolden pins EVERY piece of user-facing copy in one committed file, // testdata/screens.golden, so wording AND exact layout (line breaks, tabs, -// leading/trailing blanks) can be reviewed without deploying โ€” read the file, or -// the diff on any PR that changes copy. +// blank lines) can be reviewed without deploying โ€” read the file, or the diff on +// any PR that changes copy. A verbatim terminal transcript in three parts: // -// It's a verbatim terminal transcript: each block is `$ ` followed by -// the BYTE-EXACT output. Help is captured through the real `--help` flag path -// (SetArgs+Execute โ€” exactly what the binary runs), screens through the real -// renderers. Then an appendix indexes every remaining user-facing string. +// A. Commands โ€” the `--help` of every command, byte-exact (real --help path) +// B. Screens โ€” the stateful views rendered verbatim (home, data list, review, โ€ฆ) +// C. Messages โ€” every user-facing STRING in the source, harvested via AST (so +// error paths + live flows โ€” ingest steps, login, progress โ€” +// that aren't a single rendered screen are still all here). // // The test fails on drift; regenerate after an intentional copy change: // @@ -35,12 +39,10 @@ func TestScreensGolden(t *testing.T) { cat.WriteString("tracebloc CLI โ€” complete copy catalog\n") cat.WriteString("A verbatim transcript: each `$ command` is followed by its byte-exact output\n") - cat.WriteString("(line breaks, tabs, blank lines โ€” all as the terminal prints them).\n") - cat.WriteString("Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") + cat.WriteString("(line breaks, tabs, blank lines โ€” all as the terminal prints them). The final\n") + cat.WriteString("section indexes every user-facing string, incl. multi-step flows not shown as\n") + cat.WriteString("a single screen. Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") - // block writes `$ \n` then the output VERBATIM โ€” nothing else. The only - // whitespace between entries is each output's own real leading/trailing blank - // lines, so what you read is byte-for-byte what the terminal prints. block := func(cmd, output string) { cat.WriteString("$ " + cmd + "\n") cat.WriteString(output) @@ -48,7 +50,6 @@ func TestScreensGolden(t *testing.T) { // โ”€โ”€ PART A: every command's --help, through the real flag path โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= COMMANDS โ€” every `--help`, byte-exact\n" + strings.Repeat("=", 78) + "\n") - // Enumerate every command path (incl. hidden โ€” a user can still run them). var paths [][]string var walk func(c *cobra.Command, prefix []string) walk = func(c *cobra.Command, prefix []string) { @@ -59,8 +60,7 @@ func TestScreensGolden(t *testing.T) { if s.Name() == "help" || s.Name() == "completion" { continue } - child := append(append([]string{}, prefix...), s.Name()) - walk(s, child) + walk(s, append(append([]string{}, prefix...), s.Name())) } } walk(NewRootCmd(bi), nil) @@ -114,14 +114,20 @@ func TestScreensGolden(t *testing.T) { render("tb data list # empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }) render("tb data list # populated", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) }) render("tb data list --all", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) }) + + ingestReview := &runDataIngestArgs{ + LocalPath: "./data", + Spec: push.SpecArgs{Table: "xray_train", Category: "image_classification", Intent: "train"}, + } + render("tb data ingest ./data # pre-flight review", func(p *ui.Printer) { renderReview(p, ingestReview) }) render("tb client create # review", func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") }) render("tb delete # keep data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) }) render("tb delete # remove data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) }) - // โ”€โ”€ APPENDIX: every remaining user-facing string โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= MESSAGE INDEX โ€” every user-facing string in the source (templates, not\n= rendered; %s/%d are runtime placeholders). Catches errors, hints, and the\n= flows not rendered above (ingest validation, login, resources).\n" + strings.Repeat("=", 78) + "\n\n") + // โ”€โ”€ PART C: every user-facing string (AST harvest โ€” catches all args) โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= MESSAGE INDEX โ€” every user-facing string in the source (templates, not\n= rendered; %s/%d are runtime placeholders). Catches the multi-step flows the\n= transcript above can't show whole: ingest steps + progress, login device flow,\n= delete confirmation, and every error/hint.\n" + strings.Repeat("=", 78) + "\n\n") for _, m := range harvestMessages(t) { - cat.WriteString(" " + m + "\n") + cat.WriteString(" " + strconv.Quote(m) + "\n") } got := cat.String() @@ -144,32 +150,67 @@ func TestScreensGolden(t *testing.T) { } } -// harvestMessages reads the user-facing packages and extracts every string -// literal passed to a Printer method or an error constructor โ€” a complete, -// deduped index of user-facing copy, independent of whether a screen renders it. +// harvestMessages parses the user-facing packages and returns every string +// literal passed to a Printer method or an error constructor โ€” ALL arguments +// (so Step labels, MenuRow descriptions, Field values are included), both "โ€ฆ" and +// `โ€ฆ` raw strings. Deduped + sorted. A complete index of user-facing copy, +// independent of whether a screen renders it. func harvestMessages(t *testing.T) []string { t.Helper() - printer := regexp.MustCompile(`\.(?:Successf|Warnf|Errorf|Infof|Hintf|Detailf|Para|Section|PromptHint|PromptHeader|WarnLine|CrossLine|CheckLine|Step|Action|Stat|Field)\(\s*"((?:[^"\\]|\\.)*)"`) - errs := regexp.MustCompile(`(?:errors\.New|fmt\.Errorf)\(\s*"((?:[^"\\]|\\.)*)"`) + methods := map[string]bool{ + "Successf": true, "Warnf": true, "Errorf": true, "Infof": true, "Hintf": true, + "Detailf": true, "Para": true, "Section": true, "PromptHint": true, "PromptHeader": true, + "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, + "Stat": true, "Field": true, "MenuRow": true, "Banner": true, "Command": true, + } + isCopyCall := func(call *ast.CallExpr) bool { + sel, ok := call.Fun.(*ast.SelectorExpr) + if !ok { + return false + } + if methods[sel.Sel.Name] { + return true + } + if x, ok := sel.X.(*ast.Ident); ok { + return (x.Name == "errors" && sel.Sel.Name == "New") || (x.Name == "fmt" && sel.Sel.Name == "Errorf") + } + return false + } + seen := map[string]struct{}{} + fset := token.NewFileSet() for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { return nil } - src, err := os.ReadFile(path) - if err != nil { + f, perr := parser.ParseFile(fset, path, nil, 0) + if perr != nil { return nil } - for _, re := range []*regexp.Regexp{printer, errs} { - for _, m := range re.FindAllStringSubmatch(string(src), -1) { - s := strings.TrimSpace(m[1]) + ast.Inspect(f, func(n ast.Node) bool { + call, ok := n.(*ast.CallExpr) + if !ok || !isCopyCall(call) { + return true + } + for _, arg := range call.Args { + lit, ok := arg.(*ast.BasicLit) + if !ok || lit.Kind != token.STRING { + continue + } + s, uerr := strconv.Unquote(lit.Value) + if uerr != nil { + continue + } + s = strings.TrimSpace(s) + // Skip empties and format-only fragments (e.g. "%s", " ") โ€” no words. if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { continue } seen[s] = struct{}{} } - } + return true + }) return nil }) } diff --git a/internal/cli/testdata/screens.golden b/internal/cli/testdata/screens.golden index 767b75ec..9e19fe08 100644 --- a/internal/cli/testdata/screens.golden +++ b/internal/cli/testdata/screens.golden @@ -1,7 +1,8 @@ tracebloc CLI โ€” complete copy catalog A verbatim transcript: each `$ command` is followed by its byte-exact output -(line breaks, tabs, blank lines โ€” all as the terminal prints them). -Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden +(line breaks, tabs, blank lines โ€” all as the terminal prints them). The final +section indexes every user-facing string, incl. multi-step flows not shown as +a single screen. Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden ============================================================================== @@ -908,6 +909,14 @@ $ tb data list --all System ยท 1 ยท ingest_run_journal โ€” +$ tb data ingest ./data # pre-flight review + + Review + name: xray_train + task: image_classification + intent: train + path: ./data + resolution: auto-detect $ tb client create # review Review @@ -947,508 +956,425 @@ $ tb delete # remove data ============================================================================== = MESSAGE INDEX โ€” every user-facing string in the source (templates, not -= rendered; %s/%d are runtime placeholders). Catches errors, hints, and the -= flows not rendered above (ingest validation, login, resources). += rendered; %s/%d are runtime placeholders). Catches the multi-step flows the += transcript above can't show whole: ingest steps + progress, login device flow, += delete confirmation, and every error/hint. ============================================================================== - %d image(s) are smaller than the %dx%d minimum you set with --min-size: %s. - %d image(s) can't be ingested: %s. The cluster rejects these after the upload โ€” - %d image(s) don't match the %dx%d resolution: %s. The cluster validates the size, - %d labels.csv row(s) reference images that aren't in images/: %s. Those records - %d mask(s) are smaller than the %dx%d minimum you set with --min-size: %s. - %d mask(s) don't match the %dx%d resolution the images use: %s. Semantic-segmentation - %d mask(s) in masks/ can't be ingested: %s. The cluster reads every mask as a PNG - %d row(s) in %s have an empty %q (e.g. %s). Every row must name its mask file โ€” an - %d sequence(s) grouped by %q change their %q value mid-sequence (first offending - %d sequence(s) grouped by %q have out-of-order %q values (first offending data row(s) - %d system table(s) hidden โ€” show with --all. - %q exists but is not a directory - %q is a directory, not a file - %q is a directory, not a file. labels.csv must be the - %q is a symbolic link, which v0.1 does not allow in the dataset - %q is not a .csv file. Tabular / time-series data is a single CSV โ€” - %q is not a directory; pass the directory containing labels.csv + images/ - %q is not a directory; pass the directory containing labels.csv + the text files - %s %q must be WxH (e.g. 512x512) - %s %q: height is not an integer: %w - %s %q: width and height must both be positive - %s %q: width is not an integer: %w - %s %s โ€” %s - %s contains a NUL byte โ€” the file is corrupt or not really a CSV. The cluster - %s has a header but no data rows (0 ingestable records). Add at least one data row and re-run. - %s has duplicate column name(s): %s. Each column must be unique โ€” the cluster - %s is empty โ€” add a header and at least one data row, then re-run - %s is empty โ€” no header row - %s is image tasks only; it doesn't apply to task %q - %s isn't valid UTF-8 (likely a Latin-1/Windows-1252 export). The cluster rejects - %s requires CLIENT_WRITE permission - %s starts with a UTF-8 byte-order mark (Excel's \"CSV UTF-8\" export adds it), - %s ยท Online%s - %s ยท can't reach it from here โ€” run %s - %s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s - %s ยท running, but tracebloc hasn't heard from it โ€” run %s - %s ยท starting up, not ready yet โ€” run %s - %s โ€” %s ยท %s - %s โ€” %s ยท %s (%s) - %s: %w - %w in namespace %q, but tracebloc clients are running in: %s. - %w in namespace %q. - %w on the cluster your kubeconfig points at โ€” if this machine should - --label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column - --label-policy is regression-class tasks only (tabular_regression, - --number-of-keypoints is keypoint_detection only; it doesn't apply to task %q - --number-of-keypoints must be a positive integer (got %d); - --overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default). - --schema is empty; expected col:TYPE,col:TYPE,... - --schema is tabular/time-series tasks only; it doesn't apply to task %q - --schema names column(s) that aren't in %s: %s. The cluster rejects this after the - --time-column is time_to_event_prediction only; it doesn't apply to task %q - --timeout has no effect without --wait - A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train - A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load). - A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it. - A training run is allocated up to: - Add --help to any command for the flags. - Already signed out. - Applies to your next training run; a run already going keeps its size. - Ask one of these admins (or ask them to grant you access) - Available now: - CSV %s has no columns - Can't reach tracebloc from here. - Cancelled โ€” %q was left as-is; nothing was ingested. - Cancelled โ€” nothing was changed. - Cancelled โ€” nothing was deleted. - Cancelled โ€” nothing was ingested. - Cancelled โ€” the name didn't match. Nothing was removed. - Cancelled. - Chart uninstall reported: %v - Check on it later with: kubectl logs -f -n %s job/%s - Check your network / HTTP(S)_PROXY, then run `%s doctor` again. - Client install - Client status - Clients in your account - Cluster teardown reported: %v - Connecting to your secure environmentโ€ฆ - Correlation id: %s - Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you. - Couldn't clear the stored active-client pointer (%v) โ€” the on-disk config - Couldn't connect to your secure environment โ€” check your kubeconfig/context. - Couldn't determine this client's namespace โ€” skipped the Helm uninstall. - Couldn't locate the CLI binary to remove it (%v) โ€” delete it by hand. - Couldn't read the target cluster's identity โ€” provisioning without a cluster anchor, so re-running won't be idempotent. Point --kubeconfig/--context at the reachable cluster to enable that. - Couldn't read your tracebloc config โ€” run `%s login` to recreate it. - Couldn't reclaim the temporary copy (%v). It's harmless โ€” the next re-ingest of %q or a `tracebloc data delete %s` will clear it. - Couldn't remove local data (%v) โ€” cleared the active-client pointer; - Couldn't remove the CLI (%v) โ€” remove it by hand: rm -f %s - Couldn't remove the CLI (%v). It looks Homebrew-managed โ€” finish with: brew uninstall tracebloc - Couldn't remove the `tb` alias (%v) โ€” remove it by hand: rm -f %s - Couldn't revoke the credential server-side (%v) โ€” continuing with local teardown. - Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client) to set it. - Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client). - Couldn't verify your session with the backend (%v). - Couldn't write the support bundle: %v - Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d. - DB failures - Deleted %s.%s and %d PVC path(s). - Destructive and cannot be undone. - Detached โ€” the ingestion runs in the background on your secure environment. - Details - Details (for support) - Diagnose auth / cluster problems with: tracebloc doctor - Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal). - Docker and related tools โ€” remove them yourself if you no longer need them - Dry run โ€” nothing was changed - Dry-run complete โ€” your data and secure environment check out; nothing was created. - Dry-run โ€” nothing was deleted. - Each training run already uses up to %s โ€” nothing to change. - Each training run may now use up to %s. - Email it to support@tracebloc.io. - Enter - Everything looks good โ€” you're ready to run training. - Follow it later with: kubectl logs -f -n %s job/%s - Full log: %s - GPU access removed โ€” training runs will use CPU only. - How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose - How much of this machine a training run may use - Ingest settings - Ingestion complete โ€” %s - Ingestion complete โ€” showing its logs: - Ingestion completed partially โ€” %s - Ingestion completed with failures โ€” %s - Ingestion completed with skips โ€” %s - Ingestion started โ€” live progress: - Ingestion started โ€” streaming logs: - Ingestion summary - Ingestor SA token - Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data. - Kept on tracebloc - Kubeconfig - Learn more: https://docs.tracebloc.io - Left %s in place โ€” it isn't tracebloc's `tb` alias. - Left alone - Let's set up your data ingest - Local dataset - Machine credential โ€” needed by the installer to connect this client - Memory - No client in namespace %q โ€” using the one in %q (override with --namespace). - No clients yet. Run `tracebloc client create`. - No new credential issued; the existing one stands. This machine is set to enroll as client %d. - No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail. - No secure environment on this machine yet โ€” run the installer to set one up. - No secure environment on this machine yet. - Not signed in yet. - Not signed in โ€” run `%s login`. - Not signed in. Run `tracebloc login`. - Not yet in the CLI: - Offboarded %q. This machine is no longer connected to tracebloc. - Offboarded %q: the machine credential is revoked, so it can no longer connect to tracebloc โ€” - Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted. - Open - Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR - POST %s%s: %w - PVC %s/%s is in phase %q, not Bound. - PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter - Pick this dataset when you set it up. - Press Enter to accept a default; Ctrl-C to cancel. - Provisioned client %q (namespace %s). - Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent): - Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away. - Ready for `tracebloc data ingest`. - Reclaimed tracebloc's downloaded images. - Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values. - Removed local tracebloc data and config. - Removed stray control characters from the name. - Removed the local environment. - Removed the old %q โ€” ingesting the new data. - Removed the tracebloc CLI from this machine. - Removing in-cluster artifactsโ€ฆ - Review - Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record. - Set one up: %s - Sign in to tracebloc - Signed in - Signed in as - Signed in as %s - Signed in as %s. - Signed in to %q, but this run targets %q โ€” run `tracebloc login`. - Signed in. - Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine. - Signed out. - Signed-in token was rejected by the backend โ€” run `tracebloc login`. - Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`. - Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`. - Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own. - Stopped watching โ€” the ingestion keeps running on your secure environment. - Submitted โ€” tracebloc is validating your data and loading it into the table. - Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below. - Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below. - Table %q already exists โ€” replacing it (table + files). - Target - Target cluster - The column holding the duration / time-to-event. e.g. time, tenure_days - The column in your CSV with the answer the model learns to produce. - The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed. - The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn - The ingestion hasn't started yet (usually a slow image pull or a busy cluster). - The name you provided was only control characters โ€” auto-naming this client instead. - The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224 - The tracebloc CLI (your local data & config are kept โ€” --keep-data) - This CLI is out of date โ€” update it: %s - This cluster is already registered as client %q (namespace %s) โ€” adopted it. - This drops the table and removes the files listed above โ€” there's no undo. Pass --yes next time to skip this prompt. - This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later). - This is irreversible. Type the client name to confirm, or leave blank to cancel. - This machine's credential โ€” so tracebloc can no longer reach it - This matches a previous run (same idempotency key) โ€” attaching to the run already in progress. - This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account. - This will remove - To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases - Tore down %q on this machine, but some cleanup above didn't complete and the server-side revoke - Tore down %q on this machine. The server-side revoke didn't complete, so the credential may - Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`. - Uninstalled tracebloc. - We couldn't tell the data type from what's there โ€” which is it? - What's next - Whether this split trains the model or evaluates it. - Will delete - Wrote a support bundle to ./%s - Wrote client id + namespace to %s (no new credential โ€” the existing one stands). - You don't have permission to %s in this account. - Your CPU and memory budget is unchanged โ€” but this machine has no GPU while the cluster still requests one, so I'll clear that stale GPU setting so runs can schedule. - Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata - Your dataset records (marked unavailable, not deleted) - Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you. - Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone - Your secure environment %q and everything it runs on this machine - Your secure environment is equipped with: - Your session expired โ€” run `%s login`. - Your use cases and the models trained here - \"active\" is this machine's selected client; state is its last reported status to tracebloc. - `tracebloc ingest` doesn't stage datasets โ€” did you mean:\n - a dataset path is required - a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually - a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read - account - active client - active client %q runs on another machine โ€” namespace %q isn't on the cluster your kubeconfig points at; - active client %s isn't in your account โ€” run `tracebloc client create` - annotations - app version - authorized โ€” confirming the token with the backend โ€ฆ - backend - backend %s โ€” requesting a device code โ€ฆ - backfilling the cluster anchor onto the existing client: %w - building SPDY transport: %w - building rest config from kubeconfig: %w - building submit request: %w - building tar archive: %w - can't confirm %q exists on this client โ€” refusing to delete without - can't read %q: %w - cancelled by user - chart version - checking %s for a byte-order mark: %w - client - client id - closing tar writer: %w - cluster - column %q isn't all %s: %d value(s) don't match its declared type (e.g. %s). - columns - command - connected: %s โ€” %s - constructing kubernetes clientset: %w - context - couldn't check whether a tracebloc client is already running on this cluster (%w) โ€” - couldn't determine the installed client chart version (the release is missing - couldn't reach the backend to choose a unique client name (%v) โ€” retry, - couldn't read capacity: %v - couldn't read the account's client list to tell whether this cluster is new - couldn't read this machine's capacity: %w - creating SPDY executor for %s/%s: %w - creating credential-file directory: %w - creating port-forwarder: %w - creating stage Pod in namespace %q: %w - creating staging-cleanup pod: %w - creating teardown pod: %w - dashboard id - data CSV - dataset exceeded v0.1 total cap of %s after streaming %s - dataset exceeded v0.1 total cap of %s after streaming %s (reached %s) - dataset exceeded v0.1 total cap of %s during stream - dataset is %s, exceeds v0.1 cap of %s. - dataset is %s, exceeds v0.1 cap of %s. For larger datasets, the - dataset name is required (set --name) - dataset name is required โ€” pass it as an argument: tracebloc data delete - decoding image header %q: %w - decoding submit response (got body %q): %w - deleting stage Pod %s/%s: %w - destination - dropping %s.%s: %w%s - enter a whole number between %d and %d - exec stream against %s/%s: %w - expected %s %s-separated fields (%s), found %d. - expected a single %s record but the file spans multiple lines. - expires - expires in - field %d is empty โ€” every field (%s) must be non-empty - file %q is %s, exceeds v0.1 single-file cap of %s. - file failures - found %d .csv files in %q (%s); the tabular layout expects exactly one. - found %d tracebloc clients in namespace %q (%s); - full log: %s - generating Pod-name random suffix: %w - generating idempotency key: %w - generating staging-dir suffix: %w - images - images/ and annotations/ don't pair up: %s. Every image needs a same-named .xml - images/ and masks/ don't pair up: %s. Every image needs a same-named - inferring schema from CSV: %w - ingestion Job completed but the summary reports failures โ€” see panel above - ingestion Job exited non-zero โ€” see logs above - ingestion Job's final status couldn't be determined within the watch window โ€” - ingestor ID - ingestor SA - ingestor img - inserted - intent - interactive setup: %w - internal: re-parsing synthesized spec: %w\n%s - invalid table name %q: %w - jobs-manager - jobs-manager: %s - keypoint_detection requires --number-of-keypoints (e.g. - keypoints - kube-system namespace has no UID - label column - label column %q isn't in %s's header (columns: %s). Pass --label-column with one of - label policy - labels.csv - labels.csv column %q must be lowercase \"filename\": the cluster reads - labels.csv has no \"filename\" column (columns: %s) โ€” image tasks match each - labels.csv has no \"filename\" column (columns: %s) โ€” the ingestor matches each - listing Pods for service %s/%s: %w - listing chart-managed deployments in namespace %s: %w - listing client deployments to check for an existing client: %w - listing service-account-token secrets in %s: %w - listing stage Pods in %s: %w - loading embedded schema: %w - loading kubeconfig: %w - local dataset path is required โ€” pass it as an argument, or run - locating mysql pod: %w - location - login timed out โ€” re-run `tracebloc login` - love from tracebloc ๐Ÿ’š - marshaling submit request: %w - marshaling synthesized spec: %w - masks - min size - minting token for ServiceAccount %s/%s via TokenRequest: %w - missing %s/ subdirectory in %q - missing images/ subdirectory in %q. The CLI expects - missing labels.csv in %q. Text categories expect - missing labels.csv in %q. The CLI expects - must be a positive integer - must be between %d and %d - mysql table - name - namespace - no .csv file found in %q. Tabular / time-series categories expect a - no .png mask files found in %q. semantic_segmentation expects - no .txt files found in %q. Text categories expect - no .xml annotation files found in %q. object_detection expects - no CLI-supported tasks for %s data yet - no PersistentVolumeClaim named %q found in namespace %q. - no Ready node on this machine to size a training run against - no Running Pod backing service %s/%s (found %d Pod(s); - no Running pod with name containing %q in namespace %q - no Secret of type kubernetes.io/service-account-token bound to - no active client on this machine โ€” nothing to offboard - no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first - no dataset named %q on this client%s - no image files found in %q. Expected .jpg, .jpeg, or .png; - no image files to detect a type from - no such file or directory: %q โ€” check the path to your dataset - no tracebloc client found - no usable image files in %q โ€” found %s, but the ingestor - no usable ingestor token. TokenRequest failed: %v. - not signed in โ€” run `tracebloc login` first - outcome: early exit before the cluster was probed - outcome: early exit โ€” no roll-up verdict (granular checks below) - overwrite prompt: %w - packaging %s: %w - packaging labels.csv: %w - password - path - port-forward allocated zero ports - port-forward to %s/%s failed during startup: %w - pvc path - querying datasets: %w%s - reading %q: %w - reading %s header: %w - reading %s/: %w - reading %s: %w - reading CSV header from %s: %w - reading CSV row from %s: %w - reading PVC %s/%s: %w - reading allocated port: %w - reading dataset directory %q: %w - reading dataset path %q: %w - reading final Job status for %s/%s: %w - reading images/: %w - reading kube-system namespace UID: %w - reading labels.csv: %w - reading raw kubeconfig: %w - reading service %s/%s: %w - reading submit response body: %w - reading the existing client's identity in namespace %q: %w - ready: %s โ€” %s - refusing to change the ceiling without confirmation: pass --yes, or run on a terminal - refusing to delete without confirmation: pass --yes or run on a terminal - refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name - refusing to provision non-interactively without confirmation โ€” pass --yes to - refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover) - release - release: %s (chart %s) - removing PVC paths: %w%s - removing staged copy %s: %w%s - replacing table %q failed partway โ€” its removal may be incomplete, and a plain re-run - resolution - resolving %q: %w - resolving Service %s/%s to a Pod: %w - resolving namespace from kubeconfig: %w - resource env - root - scanning the cluster for tracebloc clients: %w - schema - schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT) - schema type %q for column %q isn't a supported SQL type โ€” the ingestor would - semantic_segmentation needs a %q column in %s (columns: %s) linking each image to its - semantic_segmentation needs a %q column in %s, but found %q (wrong case). - sent to API - server - service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding - session: %s - setting up jobs-manager port-forward: %w - sha256[:8] - shared PVC - shared PVC: %s (%s) - sign-in was denied in the browser - skipped - source - stage Pod %s/%s did not become Ready within %s%s - stage Pod %s/%s did not reach Ready state: %w%s - stage Pod %s/%s terminated in phase %q before becoming Ready%s - stat %q: %w - stat %s/: %w - stat %s: %w - stat images/: %w - stat labels.csv: %w - state - status - stored active client id %q is not numeric: %w - streaming files to %s/%s: %w%s - streaming logs from Pod %s/%s: %w - submit response missing job_name (got body %q) - submit response missing namespace (got body %q) - success rate - synthesized spec failed schema validation; check the flag values above - table %q already exists in this secure environment. Re-ingesting the same table doesn't merge or replace โ€” - table name %q is invalid: must start with a letter or - table name is %d characters; the max is %d - task - task %q isn't a recognized task. Supported tasks: %s. - task %q isn't supported by the CLI yet%s. Supported tasks: %s. - teardown failed: %w - teardown incomplete โ€” the table %s.%s was dropped, but removing its files failed: %w; - the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster - the label column %q has %d distinct value(s) โ€” a classification dataset needs at - the sequence column %q has %d empty/null value(s) (first at data row %d). Every - the sign-in code expired โ€” re-run `tracebloc login` - the time column %q has %d missing/invalid value(s) (first at data row(s) %v%s). - this backend (%s) doesn't support browser login yet โ€” the device-grant - this machine is too small to choose an amount โ€” after tracebloc's ~1 core and 3 GiB - this task's data is sequence-grouped: the schema must declare %q (groups the timestep - time column - timed out after %s before tracebloc could confirm this client โ€” retry, - timed out after %s waiting for tracebloc to report this client online (last state: %s). - timed out after %s waiting for tracebloc to report this client online; - token saved to ~/.tracebloc (0600) - total records - total size - tracebloc auth - tracebloc can see this client. - tracebloc didn't confirm your session (server error). - tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine. - tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry - tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete` - tracebloc's downloaded images - unknown backend environment %q โ€” valid values are dev, stg, prod (default). - values: - waiting for ingestor Pod: %w - waiting for staging-cleanup pod: %w - waiting for teardown pod: %w - watching ingestor Job: %w - which task is this data for? pass --task โ€” one of: %s. - would set each run to - writing credential file %s: %w - your images mix file types (%s) โ€” the ingestor requires one type - your secure environment %q has %d training run(s) active โ€” offboarding would stop them. + "\"active\" is this machine's selected client; state is its last reported status to tracebloc." + "%d system table(s) hidden โ€” show with --all." + "%q exists but is not a directory" + "%q is a directory, not a file" + "%q is not a directory; pass the directory containing labels.csv + images/" + "%q is not a directory; pass the directory containing labels.csv + the text files" + "%s %q must be WxH (e.g. 512x512)" + "%s %q: height is not an integer: %w" + "%s %q: width and height must both be positive" + "%s %q: width is not an integer: %w" + "%s %s โ€” %s" + "%s has a header but no data rows (0 ingestable records). Add at least one data row and re-run." + "%s is empty โ€” add a header and at least one data row, then re-run" + "%s is empty โ€” no header row" + "%s is image tasks only; it doesn't apply to task %q" + "%s requires CLIENT_WRITE permission" + "%s ยท Online%s" + "%s ยท can't reach it from here โ€” run %s" + "%s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s" + "%s ยท running, but tracebloc hasn't heard from it โ€” run %s" + "%s ยท starting up, not ready yet โ€” run %s" + "%s โ€” %s ยท %s" + "%s โ€” %s ยท %s (%s)" + "%s: %w" + "--label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column" + "--number-of-keypoints is keypoint_detection only; it doesn't apply to task %q" + "--overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default)." + "--schema is empty; expected col:TYPE,col:TYPE,..." + "--schema is tabular/time-series tasks only; it doesn't apply to task %q" + "--time-column is time_to_event_prediction only; it doesn't apply to task %q" + "--timeout has no effect without --wait" + "A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train" + "A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load)." + "A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it." + "A training run is allocated up to:" + "Add --help to any command for the flags." + "Already signed out." + "Applies to your next training run; a run already going keeps its size." + "Ask one of these admins (or ask them to grant you access)" + "Available now:" + "CSV %s has no columns" + "Can't reach tracebloc from here." + "Cancelled โ€” %q was left as-is; nothing was ingested." + "Cancelled โ€” nothing was changed." + "Cancelled โ€” nothing was deleted." + "Cancelled โ€” nothing was ingested." + "Cancelled โ€” the name didn't match. Nothing was removed." + "Cancelled." + "Chart uninstall reported: %v" + "Check on it later with: kubectl logs -f -n %s job/%s" + "Check your data" + "Check your network / HTTP(S)_PROXY, then run `%s doctor` again." + "Client install" + "Client status" + "Clients in your account" + "Cluster teardown reported: %v" + "Connecting to your secure environmentโ€ฆ" + "Copy into your secure environment" + "Correlation id: %s" + "Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you." + "Couldn't connect to your secure environment โ€” check your kubeconfig/context." + "Couldn't locate the CLI binary to remove it (%v) โ€” delete it by hand." + "Couldn't read the target cluster's identity โ€” provisioning without a cluster anchor, so re-running won't be idempotent. Point --kubeconfig/--context at the reachable cluster to enable that." + "Couldn't read your tracebloc config โ€” run `%s login` to recreate it." + "Couldn't reclaim the temporary copy (%v). It's harmless โ€” the next re-ingest of %q or a `tracebloc data delete %s` will clear it." + "Couldn't remove the CLI (%v) โ€” remove it by hand: rm -f %s" + "Couldn't remove the CLI (%v). It looks Homebrew-managed โ€” finish with: brew uninstall tracebloc" + "Couldn't remove the `tb` alias (%v) โ€” remove it by hand: rm -f %s" + "Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client) to set it." + "Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client)." + "Couldn't verify your session with the backend (%v)." + "Couldn't write the support bundle: %v" + "Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d." + "DB failures" + "Deleted %s.%s and %d PVC path(s)." + "Destructive and cannot be undone." + "Detached โ€” the ingestion runs in the background on your secure environment." + "Details" + "Details (for support)" + "Diagnose auth / cluster problems with: tracebloc doctor" + "Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal)." + "Docker and related tools โ€” remove them yourself if you no longer need them" + "Dry run โ€” nothing was changed" + "Dry-run complete โ€” your data and secure environment check out; nothing was created." + "Dry-run โ€” nothing was deleted." + "Each training run already uses up to %s โ€” nothing to change." + "Each training run may now use up to %s." + "Email it to support@tracebloc.io." + "Enter" + "Everything looks good โ€” you're ready to run training." + "Follow it later with: kubectl logs -f -n %s job/%s" + "Full log: %s" + "GPU access removed โ€” training runs will use CPU only." + "How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" + "How much of this machine a training run may use" + "Ingest settings" + "Ingestion complete โ€” %s" + "Ingestion complete โ€” showing its logs:" + "Ingestion completed partially โ€” %s" + "Ingestion completed with failures โ€” %s" + "Ingestion completed with skips โ€” %s" + "Ingestion started โ€” live progress:" + "Ingestion started โ€” streaming logs:" + "Ingestion summary" + "Ingestor SA token" + "Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." + "Kept on tracebloc" + "Kubeconfig" + "Learn more: https://docs.tracebloc.io" + "Left %s in place โ€” it isn't tracebloc's `tb` alias." + "Left alone" + "Let's set up your data ingest" + "Local dataset" + "Machine credential โ€” needed by the installer to connect this client" + "Memory" + "No client in namespace %q โ€” using the one in %q (override with --namespace)." + "No clients yet. Run `tracebloc client create`." + "No new credential issued; the existing one stands. This machine is set to enroll as client %d." + "No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail." + "No secure environment on this machine yet โ€” run the installer to set one up." + "No secure environment on this machine yet." + "Not signed in yet." + "Not signed in โ€” run `%s login`." + "Not signed in. Run `tracebloc login`." + "Not yet in the CLI:" + "Offboarded %q. This machine is no longer connected to tracebloc." + "Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." + "Open" + "Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR" + "POST %s%s: %w" + "PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter" + "Pick this dataset when you set it up." + "Press Enter to accept a default; Ctrl-C to cancel." + "Provisioned client %q (namespace %s)." + "Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent):" + "Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away." + "Ready for `tracebloc data ingest`." + "Reclaimed tracebloc's downloaded images." + "Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values." + "Removed local tracebloc data and config." + "Removed stray control characters from the name." + "Removed the local environment." + "Removed the old %q โ€” ingesting the new data." + "Removed the tracebloc CLI from this machine." + "Removing in-cluster artifactsโ€ฆ" + "Review" + "Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record." + "Set one up: %s" + "Sign in to tracebloc" + "Signed in" + "Signed in as %s" + "Signed in as %s." + "Signed in to %q, but this run targets %q โ€” run `tracebloc login`." + "Signed in." + "Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine." + "Signed out." + "Signed-in token was rejected by the backend โ€” run `tracebloc login`." + "Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`." + "Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`." + "Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own." + "Stopped watching โ€” the ingestion keeps running on your secure environment." + "Submitted โ€” tracebloc is validating your data and loading it into the table." + "Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below." + "Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below." + "Table %q already exists โ€” replacing it (table + files)." + "Target" + "Target cluster" + "The column holding the duration / time-to-event. e.g. time, tenure_days" + "The column in your CSV with the answer the model learns to produce." + "The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed." + "The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn" + "The name you provided was only control characters โ€” auto-naming this client instead." + "The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224" + "The tracebloc CLI (your local data & config are kept โ€” --keep-data)" + "This CLI is out of date โ€” update it: %s" + "This cluster is already registered as client %q (namespace %s) โ€” adopted it." + "This drops the table and removes the files listed above โ€” there's no undo. Pass --yes next time to skip this prompt." + "This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later)." + "This is irreversible. Type the client name to confirm, or leave blank to cancel." + "This machine's credential โ€” so tracebloc can no longer reach it" + "This matches a previous run (same idempotency key) โ€” attaching to the run already in progress." + "This permanently removes a dataset you ingested earlier: it drops the table from\nthe cluster and deletes the dataset's files on the shared storage. It can't be\nundone โ€” re-ingesting the data is the only way back." + "This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account." + "This will remove" + "To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases" + "Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." + "Uninstalled tracebloc." + "Validate and load" + "We couldn't tell the data type from what's there โ€” which is it?" + "What's next" + "Whether this split trains the model or evaluates it." + "Will delete" + "Wrote a support bundle to ./%s" + "Wrote client id + namespace to %s (no new credential โ€” the existing one stands)." + "You don't have permission to %s in this account." + "Your CPU and memory budget is unchanged โ€” but this machine has no GPU while the cluster still requests one, so I'll clear that stale GPU setting so runs can schedule." + "Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata" + "Your dataset records (marked unavailable, not deleted)" + "Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you." + "Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone" + "Your secure environment %q and everything it runs on this machine" + "Your secure environment is equipped with:" + "Your session expired โ€” run `%s login`." + "Your use cases and the models trained here" + "a dataset path is required" + "a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually" + "a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read" + "account" + "active client" + "annotations" + "app version" + "authorized โ€” confirming the token with the backend โ€ฆ" + "auto-detect" + "backend" + "backend %s โ€” requesting a device code โ€ฆ" + "backfilling the cluster anchor onto the existing client: %w" + "building SPDY transport: %w" + "building rest config from kubeconfig: %w" + "building submit request: %w" + "building tar archive: %w" + "can't read %q: %w" + "cancelled by user" + "chart version" + "checking %s for a byte-order mark: %w" + "client" + "client id" + "closing tar writer: %w" + "cluster" + "columns" + "command" + "connected: %s โ€” %s" + "constructing kubernetes clientset: %w" + "context" + "couldn't read capacity: %v" + "couldn't read this machine's capacity: %w" + "creating SPDY executor for %s/%s: %w" + "creating credential-file directory: %w" + "creating port-forwarder: %w" + "creating stage Pod in namespace %q: %w" + "creating staging-cleanup pod: %w" + "creating teardown pod: %w" + "dashboard id" + "data CSV" + "dataset exceeded v0.1 total cap of %s after streaming %s (reached %s)" + "dataset name is required (set --name)" + "dataset name is required โ€” pass it as an argument: tracebloc data delete " + "decoding image header %q: %w" + "decoding submit response (got body %q): %w" + "deleting stage Pod %s/%s: %w" + "destination" + "dropping %s.%s: %w%s" + "enter a whole number between %d and %d" + "exec stream against %s/%s: %w" + "expires" + "expires in" + "field %d is empty โ€” every field (%s) must be non-empty" + "file failures" + "full log: %s" + "generating Pod-name random suffix: %w" + "generating idempotency key: %w" + "generating staging-dir suffix: %w" + "images" + "infer from CSV" + "inferring schema from CSV: %w" + "ingestion Job completed but the summary reports failures โ€” see panel above" + "ingestion Job exited non-zero โ€” see logs above" + "ingestor ID" + "ingestor SA" + "ingestor img" + "inserted" + "intent" + "interactive setup: %w" + "internal: re-parsing synthesized spec: %w\n%s" + "invalid table name %q: %w" + "jobs-manager" + "jobs-manager: %s" + "keypoints" + "kube-system namespace has no UID" + "label column" + "label policy" + "labels.csv" + "listing Pods for service %s/%s: %w" + "listing chart-managed deployments in namespace %s: %w" + "listing client deployments to check for an existing client: %w" + "listing service-account-token secrets in %s: %w" + "listing stage Pods in %s: %w" + "loading embedded schema: %w" + "loading kubeconfig: %w" + "locating mysql pod: %w" + "location" + "login timed out โ€” re-run `tracebloc login`" + "love from tracebloc ๐Ÿ’š" + "marshaling submit request: %w" + "marshaling synthesized spec: %w" + "masks" + "min size" + "minting token for ServiceAccount %s/%s via TokenRequest: %w" + "missing %s/ subdirectory in %q" + "must be a positive integer" + "must be between %d and %d" + "mysql table" + "name" + "namespace" + "never (static-secret fallback)" + "no CLI-supported tasks for %s data yet" + "no Ready node on this machine to size a training run against" + "no Running pod with name containing %q in namespace %q" + "no active client on this machine โ€” nothing to offboard" + "no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first" + "no dataset named %q on this client%s" + "no image files to detect a type from" + "no such file or directory: %q โ€” check the path to your dataset" + "no tracebloc client found" + "none detected" + "not signed in โ€” run `tracebloc login` first" + "outcome: early exit before the cluster was probed" + "outcome: early exit โ€” no roll-up verdict (granular checks below)" + "overwrite prompt: %w" + "packaging %s: %w" + "packaging labels.csv: %w" + "password" + "path" + "port-forward allocated zero ports" + "port-forward to %s/%s failed during startup: %w" + "pvc path" + "querying datasets: %w%s" + "reading %q: %w" + "reading %s header: %w" + "reading %s/: %w" + "reading %s: %w" + "reading CSV header from %s: %w" + "reading CSV row from %s: %w" + "reading PVC %s/%s: %w" + "reading allocated port: %w" + "reading dataset directory %q: %w" + "reading dataset path %q: %w" + "reading final Job status for %s/%s: %w" + "reading images/: %w" + "reading kube-system namespace UID: %w" + "reading labels.csv: %w" + "reading raw kubeconfig: %w" + "reading service %s/%s: %w" + "reading submit response body: %w" + "reading the existing client's identity in namespace %q: %w" + "ready: %s โ€” %s" + "refusing to change the ceiling without confirmation: pass --yes, or run on a terminal" + "refusing to delete without confirmation: pass --yes or run on a terminal" + "refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name" + "refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover)" + "release" + "release: %s (chart %s)" + "removed โ€” runs will use CPU only" + "removing PVC paths: %w%s" + "removing staged copy %s: %w%s" + "resolution" + "resolving %q: %w" + "resolving Service %s/%s to a Pod: %w" + "resolving namespace from kubeconfig: %w" + "resource env" + "root" + "scanning the cluster for tracebloc clients: %w" + "schema" + "schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT)" + "sent to API" + "server" + "service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding" + "session: %s" + "setting up jobs-manager port-forward: %w" + "sha256[:8]" + "shared PVC" + "shared PVC: %s (%s)" + "sign-in was denied in the browser" + "signed in" + "skipped" + "source" + "stage Pod %s/%s did not become Ready within %s%s" + "stage Pod %s/%s did not reach Ready state: %w%s" + "stage Pod %s/%s terminated in phase %q before becoming Ready%s" + "stat %q: %w" + "stat %s/: %w" + "stat %s: %w" + "stat images/: %w" + "stat labels.csv: %w" + "state" + "status" + "stored active client id %q is not numeric: %w" + "streaming files to %s/%s: %w%s" + "streaming logs from Pod %s/%s: %w" + "submit response missing job_name (got body %q)" + "submit response missing namespace (got body %q)" + "success rate" + "synthesized spec failed schema validation; check the flag values above" + "task" + "task %q isn't a recognized task. Supported tasks: %s." + "task %q isn't supported by the CLI yet%s. Supported tasks: %s." + "teardown failed: %w" + "the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster" + "the sign-in code expired โ€” re-run `tracebloc login`" + "time column" + "token saved to ~/.tracebloc (0600)" + "total records" + "total size" + "tracebloc auth" + "tracebloc can see this client." + "tracebloc didn't confirm your session (server error)." + "tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine." + "tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry" + "tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete`" + "tracebloc's downloaded images" + "unavailable" + "values:" + "waiting for ingestor Pod: %w" + "waiting for staging-cleanup pod: %w" + "waiting for teardown pod: %w" + "watching ingestor Job: %w" + "would set each run to" + "writing credential file %s: %w" From c24ff981cf667f430b46e1866754620e746e4b58 Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Tue, 21 Jul 2026 21:53:15 +0200 Subject: [PATCH 05/14] Catalog: split into a per-command golden/ folder (structure + first files) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per Lukas's approved structure: replace the monolith screens.golden with testdata/golden/, one file per command, ordered by how often a user hits it. Each file leads with what you see when you RUN the command (byte-exact, every state/path), and ends with its --help. This commit lands the generator + the flagship files: - 00-home.golden โ€” tb / tracebloc, all 7 states + --help - 02-data-list.golden โ€” empty / populated / --all + --help - zz-all-strings.golden โ€” every user-facing string (AST index), completeness backstop Adding a command file is now one entry in the generator. Coming next: 01-ingest (driven run), 03-data-delete, 04-resources, 05-doctor, 06-delete, 07-login, the rest, and the installer folder in the client repo. (The catalog already caught drift: the root --help still says "your client" / "the cluster" instead of "secure environment" โ€” a copy fix to do separately.) Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 232 +++ internal/cli/testdata/golden/00-home.golden | 257 +++ .../cli/testdata/golden/02-data-list.golden | 64 + .../cli/testdata/golden/zz-all-strings.golden | 422 +++++ internal/cli/testdata/screens.golden | 1380 ----------------- 5 files changed, 975 insertions(+), 1380 deletions(-) create mode 100644 internal/cli/copy_catalog_test.go create mode 100644 internal/cli/testdata/golden/00-home.golden create mode 100644 internal/cli/testdata/golden/02-data-list.golden create mode 100644 internal/cli/testdata/golden/zz-all-strings.golden delete mode 100644 internal/cli/testdata/screens.golden diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go new file mode 100644 index 00000000..c737dc43 --- /dev/null +++ b/internal/cli/copy_catalog_test.go @@ -0,0 +1,232 @@ +package cli + +import ( + "bytes" + "go/ast" + "go/parser" + "go/token" + "os" + "path/filepath" + "sort" + "strconv" + "strings" + "testing" + + "github.com/tracebloc/cli/internal/push" + "github.com/tracebloc/cli/internal/ui" +) + +// TestCopyCatalog generates the copy catalog under testdata/golden/ โ€” ONE file +// per command, so every user-facing string can be reviewed a screen at a time +// without deploying. Each file leads with what you see when you RUN the command +// (byte-exact, colour off), covers every state/path, and ends with its `--help`. +// zz-all-strings.golden is a completeness backstop: every user-facing string in +// the source, so nothing is missed even on a rare path. +// +// The test fails on drift; regenerate after an intentional copy change: +// +// TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestCopyCatalog +// +// Files are ordered by how often a user hits them (00 = most). More command +// files (ingest run, resources, doctor, delete, login) land as their driven +// transcripts are wired; this generator makes adding one a single entry below. +func TestCopyCatalog(t *testing.T) { + bi := BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"} + + // help captures a command's `--help` through the REAL flag path (SetArgs+ + // Execute โ€” exactly what the binary runs), byte-exact. + help := func(path ...string) string { + var b bytes.Buffer + r := NewRootCmd(bi) + r.SetOut(&b) + r.SetErr(&b) + r.SetArgs(append(append([]string{}, path...), "--help")) + _ = r.Execute() + return b.String() + } + // rndr renders one screen via the real renderer, colour off. + rndr := func(f func(*ui.Printer)) string { + var b bytes.Buffer + f(ui.New(&b, ui.WithColor(false))) + return b.String() + } + + // doc assembles one command file: a title, then labelled `$ cmd` blocks + // (verbatim output), then the command's --help at the bottom. + type run struct { + cmd, out string // "$ " then verbatim + } + doc := func(title, whenSeen string, runs []run, helpCmd string, helpOut string) string { + var s strings.Builder + s.WriteString(title + "\n" + strings.Repeat("=", len([]rune(title))) + "\n") + s.WriteString(whenSeen + "\n") + for _, r := range runs { + s.WriteString("\n$ " + r.cmd + "\n") + s.WriteString(r.out) + } + if helpOut != "" { + s.WriteString("\n\n" + strings.Repeat("-", 60) + "\n--help\n" + strings.Repeat("-", 60) + "\n") + s.WriteString("$ " + helpCmd + "\n") + s.WriteString(helpOut) + } + return s.String() + } + + // โ”€โ”€ home models (every state resolveHomeModel can produce) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + online := homeModel{ + state: homeOnline, email: "lukas@tracebloc.io", name: "Lukas", envName: "hello-world", + compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, inv: binTB, fullMenu: true, hasResources: true, + } + noComp := online + noComp.state, noComp.hasCompute, noComp.compute = homeRunning, false, computeInfo{} + notOnline := noComp + notOnline.confirmedNotOnline = true + starting := noComp + starting.state = homeStarting + offline := noComp + offline.state = homeOffline + noEnv := noComp + noEnv.state, noEnv.fullMenu, noEnv.envName = homeNoEnv, false, "" + signedOut := homeModel{state: homeNotSignedIn, inv: binTB} + + homeFile := doc( + "tb / tracebloc โ€” home", + "What you see when you run `tb` (or `tracebloc`) with no arguments. Covers every\nstate the home view resolves to.", + []run{ + {"tb # signed in ยท secure environment Online", rndr(func(p *ui.Printer) { renderHome(p, online) })}, + {"tb # signed in ยท running, couldn't confirm connection", rndr(func(p *ui.Printer) { renderHome(p, noComp) })}, + {"tb # signed in ยท running, backend reports not online", rndr(func(p *ui.Printer) { renderHome(p, notOnline) })}, + {"tb # signed in ยท starting up", rndr(func(p *ui.Printer) { renderHome(p, starting) })}, + {"tb # signed in ยท offline (can't reach it)", rndr(func(p *ui.Printer) { renderHome(p, offline) })}, + {"tb # signed in ยท no secure environment on this machine", rndr(func(p *ui.Printer) { renderHome(p, noEnv) })}, + {"tb # not signed in", rndr(func(p *ui.Printer) { renderHome(p, signedOut) })}, + }, + "tracebloc --help", help(), + ) + + // โ”€โ”€ data list โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + sample := []push.DatasetInfo{ + {Name: "xray_train", Intent: "train", Task: "image_classification", Records: 12000, Classes: 2, Extension: "jpg", SizeBytes: 1 << 30}, + {Name: "xray_test", Intent: "test", Task: "image_classification", Records: 3000, Classes: 2, Extension: "jpg", SizeBytes: 256 << 20}, + {Name: "ingest_run_journal", System: true}, + } + dataListFile := doc( + "tb data list โ€” list your datasets", + "What you see when you run `tb data list`. Covers empty, populated, and --all.", + []run{ + {"tb data list # no datasets yet", rndr(func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) })}, + {"tb data list # with datasets (system tables hidden)", rndr(func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) })}, + {"tb data list --all # including system tables", rndr(func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) })}, + }, + "tracebloc data list --help", help("data", "list"), + ) + + files := map[string]string{ + "00-home.golden": homeFile, + "02-data-list.golden": dataListFile, + "zz-all-strings.golden": "every user-facing string in the source (AST-harvested โ€” all arguments, both\n" + + `"โ€ฆ" and ` + "`โ€ฆ`" + " raw strings). The completeness backstop: catches error paths and\nthe multi-step flows (ingest steps, login, progress) not shown as a screen.\n" + + "%s/%d are runtime placeholders.\n\n" + strings.Join(quoteAll(harvestMessages(t)), "\n") + "\n", + } + + update := os.Getenv("TB_UPDATE_GOLDEN") != "" + for name, content := range files { + path := filepath.Join("testdata/golden", name) + if update { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatal(err) + } + continue + } + want, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s (regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestCopyCatalog): %v", path, err) + } + if content != string(want) { + t.Errorf("%s drifted. Regenerate + review the diff:\n TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestCopyCatalog", path) + } + } + if update { + t.Logf("wrote %d catalog files under testdata/golden/", len(files)) + } +} + +func quoteAll(in []string) []string { + out := make([]string, len(in)) + for i, s := range in { + out[i] = strconv.Quote(s) + } + return out +} + +// harvestMessages parses the user-facing packages and returns every string +// literal passed to a Printer method or an error constructor โ€” ALL arguments +// (Step labels, MenuRow descriptions, Field values included), both "โ€ฆ" and `โ€ฆ` +// raw strings. Deduped + sorted. +func harvestMessages(t *testing.T) []string { + t.Helper() + methods := map[string]bool{ + "Successf": true, "Warnf": true, "Errorf": true, "Infof": true, "Hintf": true, + "Detailf": true, "Para": true, "Section": true, "PromptHint": true, "PromptHeader": true, + "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, + "Stat": true, "Field": true, "MenuRow": true, "Banner": true, "Command": true, + } + isCopyCall := func(call *ast.CallExpr) bool { + sel, ok := call.Fun.(*ast.SelectorExpr) + if !ok { + return false + } + if methods[sel.Sel.Name] { + return true + } + if x, ok := sel.X.(*ast.Ident); ok { + return (x.Name == "errors" && sel.Sel.Name == "New") || (x.Name == "fmt" && sel.Sel.Name == "Errorf") + } + return false + } + seen := map[string]struct{}{} + fset := token.NewFileSet() + for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { + _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { + if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { + return nil + } + f, perr := parser.ParseFile(fset, path, nil, 0) + if perr != nil { + return nil + } + ast.Inspect(f, func(n ast.Node) bool { + call, ok := n.(*ast.CallExpr) + if !ok || !isCopyCall(call) { + return true + } + for _, arg := range call.Args { + lit, ok := arg.(*ast.BasicLit) + if !ok || lit.Kind != token.STRING { + continue + } + s, uerr := strconv.Unquote(lit.Value) + if uerr != nil { + continue + } + s = strings.TrimSpace(s) + if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { + continue + } + seen[s] = struct{}{} + } + return true + }) + return nil + }) + } + out := make([]string, 0, len(seen)) + for s := range seen { + out = append(out, s) + } + sort.Strings(out) + return out +} diff --git a/internal/cli/testdata/golden/00-home.golden b/internal/cli/testdata/golden/00-home.golden new file mode 100644 index 00000000..e2166ef9 --- /dev/null +++ b/internal/cli/testdata/golden/00-home.golden @@ -0,0 +1,257 @@ +tb / tracebloc โ€” home +===================== +What you see when you run `tb` (or `tracebloc`) with no arguments. Covers every +state the home view resolves to. + +$ tb # signed in ยท secure environment Online + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โœ“ Secure environment "hello-world" ยท Online (12 CPU ยท 23 GiB) + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # signed in ยท running, couldn't confirm connection + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # signed in ยท running, backend reports not online + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท running, but tracebloc hasn't heard from it โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # signed in ยท starting up + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  Secure environment "hello-world" ยท starting up, not ready yet โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # signed in ยท offline (can't reach it) + + + Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โœ— Secure environment "hello-world" ยท can't reach it from here โ€” run tb doctor + + + Your data + + ยท tb data ingest load a dataset into your secure environment + ยท tb data list list your datasets + ยท tb data delete remove a dataset + + + Your secure environment + + ยท tb resources manage compute & memory + ยท tb doctor check the connection & diagnose issues + ยท tb delete remove tracebloc from this machine + + + Add --help to any command for the flags. + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # signed in ยท no secure environment on this machine + + + Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ“ Signed in as lukas@tracebloc.io + โš  No secure environment on this machine yet โ€” run the installer to set one up. + + + Your secure environment + + ยท tb doctor check the connection & diagnose issues + + + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + +$ tb # not signed in + + + Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + โœ— Not signed in yet. + + + Start here + + ยท tb login sign in to tracebloc + + + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + love from tracebloc ๐Ÿ’š + + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc --help +The tracebloc CLI connects machines to tracebloc as clients and +manages the datasets that models train on. Your data stays on your +infrastructure โ€” models from other collaborators come to it, once you +approve them. + +Two kinds of commands: + + Your account (sign in first): login, logout, auth, client + This machine's client: data, cluster + +A typical first session: + + tracebloc login # sign in or create your account (browser) + tracebloc data ingest ./my-data # stage a dataset into your client + tracebloc data list # see what's in the cluster + +The CLI finds your cluster through your kubeconfig, stages data onto +the cluster's shared storage, and reports progress as it goes. No +Helm, no YAML, no kubectl needed. + +Usage: + tracebloc [flags] + tracebloc [command] + +Available Commands: + auth Inspect tracebloc authentication state + client Provision this machine's tracebloc client + cluster Inspect the cluster the CLI is currently targeting + completion Generate the autocompletion script for the specified shell + data Manage the datasets in your secure environment + delete Offboard this machine from tracebloc (revoke, uninstall, reclaim disk) + doctor Check your secure environment is connected and ready to run training + help Help about any command + login Sign in to tracebloc in your browser (device flow) + logout Sign out (revoke the token server-side and clear it locally) + resources Show how much of this machine tracebloc may use + version Print the tracebloc CLI version, git SHA, and build date + +Flags: + -h, --help help for tracebloc + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc [command] --help" for more information about a command. diff --git a/internal/cli/testdata/golden/02-data-list.golden b/internal/cli/testdata/golden/02-data-list.golden new file mode 100644 index 00000000..17931f48 --- /dev/null +++ b/internal/cli/testdata/golden/02-data-list.golden @@ -0,0 +1,64 @@ +tb data list โ€” list your datasets +================================= +What you see when you run `tb data list`. Covers empty, populated, and --all. + +$ tb data list # no datasets yet + + Datasets in hello-world (0) + + No datasets yet โ€” ingest one with `tracebloc data ingest`. + +$ tb data list # with datasets (system tables hidden) + + Datasets in hello-world โ€” 2 ยท 1.25 GiB + 1 system table(s) hidden โ€” show with --all. + + Image classification ยท 2 + โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” + โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” + +$ tb data list --all # including system tables + + Datasets in hello-world โ€” 2 ยท 1.25 GiB + + Image classification ยท 2 + โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” + โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” + + System ยท 1 + ยท ingest_run_journal โ€” + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc data list --help +Lists the datasets ingested into your client โ€” the tables in training_test_datasets +on the cluster โ€” grouped by modality, with each dataset's split (train/test), +record count, size, format, and when it was ingested. + +With no flags it uses your current kubeconfig context and its namespace; +the flags below override that, same as `cluster info` and `data ingest`. +Framework tables (the ingest-run journal) are hidden unless you pass --all. +For the full catalog, see the dashboard at https://ai.tracebloc.io/metadata. + +Exit codes: + 0 listed successfully (including an empty list) + 3 kubeconfig error + 4 cluster reachable but no tracebloc client in the namespace + 7 couldn't query the cluster for datasets + +Usage: + tracebloc data list [flags] + +Flags: + --all include framework/system tables (e.g. the ingest-run journal), normally hidden + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for list + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed + --output-json emit the dataset list as JSON on stdout (human output โ†’ stderr) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden new file mode 100644 index 00000000..59ea49ce --- /dev/null +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -0,0 +1,422 @@ +every user-facing string in the source (AST-harvested โ€” all arguments, both +"โ€ฆ" and `โ€ฆ` raw strings). The completeness backstop: catches error paths and +the multi-step flows (ingest steps, login, progress) not shown as a screen. +%s/%d are runtime placeholders. + +"\"active\" is this machine's selected client; state is its last reported status to tracebloc." +"%d system table(s) hidden โ€” show with --all." +"%q exists but is not a directory" +"%q is a directory, not a file" +"%q is not a directory; pass the directory containing labels.csv + images/" +"%q is not a directory; pass the directory containing labels.csv + the text files" +"%s %q must be WxH (e.g. 512x512)" +"%s %q: height is not an integer: %w" +"%s %q: width and height must both be positive" +"%s %q: width is not an integer: %w" +"%s %s โ€” %s" +"%s has a header but no data rows (0 ingestable records). Add at least one data row and re-run." +"%s is empty โ€” add a header and at least one data row, then re-run" +"%s is empty โ€” no header row" +"%s is image tasks only; it doesn't apply to task %q" +"%s requires CLIENT_WRITE permission" +"%s ยท Online%s" +"%s ยท can't reach it from here โ€” run %s" +"%s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s" +"%s ยท running, but tracebloc hasn't heard from it โ€” run %s" +"%s ยท starting up, not ready yet โ€” run %s" +"%s โ€” %s ยท %s" +"%s โ€” %s ยท %s (%s)" +"%s: %w" +"--label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column" +"--number-of-keypoints is keypoint_detection only; it doesn't apply to task %q" +"--overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default)." +"--schema is empty; expected col:TYPE,col:TYPE,..." +"--schema is tabular/time-series tasks only; it doesn't apply to task %q" +"--time-column is time_to_event_prediction only; it doesn't apply to task %q" +"--timeout has no effect without --wait" +"A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train" +"A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load)." +"A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it." +"A training run is allocated up to:" +"Add --help to any command for the flags." +"Already signed out." +"Applies to your next training run; a run already going keeps its size." +"Ask one of these admins (or ask them to grant you access)" +"Available now:" +"CSV %s has no columns" +"Can't reach tracebloc from here." +"Cancelled โ€” %q was left as-is; nothing was ingested." +"Cancelled โ€” nothing was changed." +"Cancelled โ€” nothing was deleted." +"Cancelled โ€” nothing was ingested." +"Cancelled โ€” the name didn't match. Nothing was removed." +"Cancelled." +"Chart uninstall reported: %v" +"Check on it later with: kubectl logs -f -n %s job/%s" +"Check your data" +"Check your network / HTTP(S)_PROXY, then run `%s doctor` again." +"Client install" +"Client status" +"Clients in your account" +"Cluster teardown reported: %v" +"Connecting to your secure environmentโ€ฆ" +"Copy into your secure environment" +"Correlation id: %s" +"Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you." +"Couldn't connect to your secure environment โ€” check your kubeconfig/context." +"Couldn't locate the CLI binary to remove it (%v) โ€” delete it by hand." +"Couldn't read the target cluster's identity โ€” provisioning without a cluster anchor, so re-running won't be idempotent. Point --kubeconfig/--context at the reachable cluster to enable that." +"Couldn't read your tracebloc config โ€” run `%s login` to recreate it." +"Couldn't reclaim the temporary copy (%v). It's harmless โ€” the next re-ingest of %q or a `tracebloc data delete %s` will clear it." +"Couldn't remove the CLI (%v) โ€” remove it by hand: rm -f %s" +"Couldn't remove the CLI (%v). It looks Homebrew-managed โ€” finish with: brew uninstall tracebloc" +"Couldn't remove the `tb` alias (%v) โ€” remove it by hand: rm -f %s" +"Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client) to set it." +"Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client)." +"Couldn't verify your session with the backend (%v)." +"Couldn't write the support bundle: %v" +"Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d." +"DB failures" +"Deleted %s.%s and %d PVC path(s)." +"Destructive and cannot be undone." +"Detached โ€” the ingestion runs in the background on your secure environment." +"Details" +"Details (for support)" +"Diagnose auth / cluster problems with: tracebloc doctor" +"Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal)." +"Docker and related tools โ€” remove them yourself if you no longer need them" +"Dry run โ€” nothing was changed" +"Dry-run complete โ€” your data and secure environment check out; nothing was created." +"Dry-run โ€” nothing was deleted." +"Each training run already uses up to %s โ€” nothing to change." +"Each training run may now use up to %s." +"Email it to support@tracebloc.io." +"Enter" +"Everything looks good โ€” you're ready to run training." +"Follow it later with: kubectl logs -f -n %s job/%s" +"Full log: %s" +"GPU access removed โ€” training runs will use CPU only." +"How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" +"How much of this machine a training run may use" +"Ingest settings" +"Ingestion complete โ€” %s" +"Ingestion complete โ€” showing its logs:" +"Ingestion completed partially โ€” %s" +"Ingestion completed with failures โ€” %s" +"Ingestion completed with skips โ€” %s" +"Ingestion started โ€” live progress:" +"Ingestion started โ€” streaming logs:" +"Ingestion summary" +"Ingestor SA token" +"Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." +"Kept on tracebloc" +"Kubeconfig" +"Learn more: https://docs.tracebloc.io" +"Left %s in place โ€” it isn't tracebloc's `tb` alias." +"Left alone" +"Let's set up your data ingest" +"Local dataset" +"Machine credential โ€” needed by the installer to connect this client" +"Memory" +"No client in namespace %q โ€” using the one in %q (override with --namespace)." +"No clients yet. Run `tracebloc client create`." +"No new credential issued; the existing one stands. This machine is set to enroll as client %d." +"No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail." +"No secure environment on this machine yet โ€” run the installer to set one up." +"No secure environment on this machine yet." +"Not signed in yet." +"Not signed in โ€” run `%s login`." +"Not signed in. Run `tracebloc login`." +"Not yet in the CLI:" +"Offboarded %q. This machine is no longer connected to tracebloc." +"Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." +"Open" +"Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR" +"POST %s%s: %w" +"PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter" +"Pick this dataset when you set it up." +"Press Enter to accept a default; Ctrl-C to cancel." +"Provisioned client %q (namespace %s)." +"Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent):" +"Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away." +"Ready for `tracebloc data ingest`." +"Reclaimed tracebloc's downloaded images." +"Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values." +"Removed local tracebloc data and config." +"Removed stray control characters from the name." +"Removed the local environment." +"Removed the old %q โ€” ingesting the new data." +"Removed the tracebloc CLI from this machine." +"Removing in-cluster artifactsโ€ฆ" +"Review" +"Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record." +"Set one up: %s" +"Sign in to tracebloc" +"Signed in" +"Signed in as %s" +"Signed in as %s." +"Signed in to %q, but this run targets %q โ€” run `tracebloc login`." +"Signed in." +"Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine." +"Signed out." +"Signed-in token was rejected by the backend โ€” run `tracebloc login`." +"Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`." +"Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`." +"Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own." +"Stopped watching โ€” the ingestion keeps running on your secure environment." +"Submitted โ€” tracebloc is validating your data and loading it into the table." +"Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below." +"Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below." +"Table %q already exists โ€” replacing it (table + files)." +"Target" +"Target cluster" +"The column holding the duration / time-to-event. e.g. time, tenure_days" +"The column in your CSV with the answer the model learns to produce." +"The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed." +"The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn" +"The name you provided was only control characters โ€” auto-naming this client instead." +"The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224" +"The tracebloc CLI (your local data & config are kept โ€” --keep-data)" +"This CLI is out of date โ€” update it: %s" +"This cluster is already registered as client %q (namespace %s) โ€” adopted it." +"This drops the table and removes the files listed above โ€” there's no undo. Pass --yes next time to skip this prompt." +"This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later)." +"This is irreversible. Type the client name to confirm, or leave blank to cancel." +"This machine's credential โ€” so tracebloc can no longer reach it" +"This matches a previous run (same idempotency key) โ€” attaching to the run already in progress." +"This permanently removes a dataset you ingested earlier: it drops the table from\nthe cluster and deletes the dataset's files on the shared storage. It can't be\nundone โ€” re-ingesting the data is the only way back." +"This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account." +"This will remove" +"To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases" +"Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." +"Uninstalled tracebloc." +"Validate and load" +"We couldn't tell the data type from what's there โ€” which is it?" +"What's next" +"Whether this split trains the model or evaluates it." +"Will delete" +"Wrote a support bundle to ./%s" +"Wrote client id + namespace to %s (no new credential โ€” the existing one stands)." +"You don't have permission to %s in this account." +"Your CPU and memory budget is unchanged โ€” but this machine has no GPU while the cluster still requests one, so I'll clear that stale GPU setting so runs can schedule." +"Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata" +"Your dataset records (marked unavailable, not deleted)" +"Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you." +"Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone" +"Your secure environment %q and everything it runs on this machine" +"Your secure environment is equipped with:" +"Your session expired โ€” run `%s login`." +"Your use cases and the models trained here" +"a dataset path is required" +"a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually" +"a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read" +"account" +"active client" +"annotations" +"app version" +"authorized โ€” confirming the token with the backend โ€ฆ" +"auto-detect" +"backend" +"backend %s โ€” requesting a device code โ€ฆ" +"backfilling the cluster anchor onto the existing client: %w" +"building SPDY transport: %w" +"building rest config from kubeconfig: %w" +"building submit request: %w" +"building tar archive: %w" +"can't read %q: %w" +"cancelled by user" +"chart version" +"checking %s for a byte-order mark: %w" +"client" +"client id" +"closing tar writer: %w" +"cluster" +"columns" +"command" +"connected: %s โ€” %s" +"constructing kubernetes clientset: %w" +"context" +"couldn't read capacity: %v" +"couldn't read this machine's capacity: %w" +"creating SPDY executor for %s/%s: %w" +"creating credential-file directory: %w" +"creating port-forwarder: %w" +"creating stage Pod in namespace %q: %w" +"creating staging-cleanup pod: %w" +"creating teardown pod: %w" +"dashboard id" +"data CSV" +"dataset exceeded v0.1 total cap of %s after streaming %s (reached %s)" +"dataset name is required (set --name)" +"dataset name is required โ€” pass it as an argument: tracebloc data delete " +"decoding image header %q: %w" +"decoding submit response (got body %q): %w" +"deleting stage Pod %s/%s: %w" +"destination" +"dropping %s.%s: %w%s" +"enter a whole number between %d and %d" +"exec stream against %s/%s: %w" +"expires" +"expires in" +"field %d is empty โ€” every field (%s) must be non-empty" +"file failures" +"full log: %s" +"generating Pod-name random suffix: %w" +"generating idempotency key: %w" +"generating staging-dir suffix: %w" +"images" +"infer from CSV" +"inferring schema from CSV: %w" +"ingestion Job completed but the summary reports failures โ€” see panel above" +"ingestion Job exited non-zero โ€” see logs above" +"ingestor ID" +"ingestor SA" +"ingestor img" +"inserted" +"intent" +"interactive setup: %w" +"internal: re-parsing synthesized spec: %w\n%s" +"invalid table name %q: %w" +"jobs-manager" +"jobs-manager: %s" +"keypoints" +"kube-system namespace has no UID" +"label column" +"label policy" +"labels.csv" +"listing Pods for service %s/%s: %w" +"listing chart-managed deployments in namespace %s: %w" +"listing client deployments to check for an existing client: %w" +"listing service-account-token secrets in %s: %w" +"listing stage Pods in %s: %w" +"loading embedded schema: %w" +"loading kubeconfig: %w" +"locating mysql pod: %w" +"location" +"login timed out โ€” re-run `tracebloc login`" +"love from tracebloc ๐Ÿ’š" +"marshaling submit request: %w" +"marshaling synthesized spec: %w" +"masks" +"min size" +"minting token for ServiceAccount %s/%s via TokenRequest: %w" +"missing %s/ subdirectory in %q" +"must be a positive integer" +"must be between %d and %d" +"mysql table" +"name" +"namespace" +"never (static-secret fallback)" +"no CLI-supported tasks for %s data yet" +"no Ready node on this machine to size a training run against" +"no Running pod with name containing %q in namespace %q" +"no active client on this machine โ€” nothing to offboard" +"no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first" +"no dataset named %q on this client%s" +"no image files to detect a type from" +"no such file or directory: %q โ€” check the path to your dataset" +"no tracebloc client found" +"none detected" +"not signed in โ€” run `tracebloc login` first" +"outcome: early exit before the cluster was probed" +"outcome: early exit โ€” no roll-up verdict (granular checks below)" +"overwrite prompt: %w" +"packaging %s: %w" +"packaging labels.csv: %w" +"password" +"path" +"port-forward allocated zero ports" +"port-forward to %s/%s failed during startup: %w" +"pvc path" +"querying datasets: %w%s" +"reading %q: %w" +"reading %s header: %w" +"reading %s/: %w" +"reading %s: %w" +"reading CSV header from %s: %w" +"reading CSV row from %s: %w" +"reading PVC %s/%s: %w" +"reading allocated port: %w" +"reading dataset directory %q: %w" +"reading dataset path %q: %w" +"reading final Job status for %s/%s: %w" +"reading images/: %w" +"reading kube-system namespace UID: %w" +"reading labels.csv: %w" +"reading raw kubeconfig: %w" +"reading service %s/%s: %w" +"reading submit response body: %w" +"reading the existing client's identity in namespace %q: %w" +"ready: %s โ€” %s" +"refusing to change the ceiling without confirmation: pass --yes, or run on a terminal" +"refusing to delete without confirmation: pass --yes or run on a terminal" +"refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name" +"refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover)" +"release" +"release: %s (chart %s)" +"removed โ€” runs will use CPU only" +"removing PVC paths: %w%s" +"removing staged copy %s: %w%s" +"resolution" +"resolving %q: %w" +"resolving Service %s/%s to a Pod: %w" +"resolving namespace from kubeconfig: %w" +"resource env" +"root" +"scanning the cluster for tracebloc clients: %w" +"schema" +"schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT)" +"sent to API" +"server" +"service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding" +"session: %s" +"setting up jobs-manager port-forward: %w" +"sha256[:8]" +"shared PVC" +"shared PVC: %s (%s)" +"sign-in was denied in the browser" +"signed in" +"skipped" +"source" +"stage Pod %s/%s did not become Ready within %s%s" +"stage Pod %s/%s did not reach Ready state: %w%s" +"stage Pod %s/%s terminated in phase %q before becoming Ready%s" +"stat %q: %w" +"stat %s/: %w" +"stat %s: %w" +"stat images/: %w" +"stat labels.csv: %w" +"state" +"status" +"stored active client id %q is not numeric: %w" +"streaming files to %s/%s: %w%s" +"streaming logs from Pod %s/%s: %w" +"submit response missing job_name (got body %q)" +"submit response missing namespace (got body %q)" +"success rate" +"synthesized spec failed schema validation; check the flag values above" +"task" +"task %q isn't a recognized task. Supported tasks: %s." +"task %q isn't supported by the CLI yet%s. Supported tasks: %s." +"teardown failed: %w" +"the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster" +"the sign-in code expired โ€” re-run `tracebloc login`" +"time column" +"token saved to ~/.tracebloc (0600)" +"total records" +"total size" +"tracebloc auth" +"tracebloc can see this client." +"tracebloc didn't confirm your session (server error)." +"tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine." +"tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry" +"tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete`" +"tracebloc's downloaded images" +"unavailable" +"values:" +"waiting for ingestor Pod: %w" +"waiting for staging-cleanup pod: %w" +"waiting for teardown pod: %w" +"watching ingestor Job: %w" +"would set each run to" +"writing credential file %s: %w" diff --git a/internal/cli/testdata/screens.golden b/internal/cli/testdata/screens.golden deleted file mode 100644 index 9e19fe08..00000000 --- a/internal/cli/testdata/screens.golden +++ /dev/null @@ -1,1380 +0,0 @@ -tracebloc CLI โ€” complete copy catalog -A verbatim transcript: each `$ command` is followed by its byte-exact output -(line breaks, tabs, blank lines โ€” all as the terminal prints them). The final -section indexes every user-facing string, incl. multi-step flows not shown as -a single screen. Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden - - -============================================================================== -= COMMANDS โ€” every `--help`, byte-exact -============================================================================== -$ tracebloc --help -The tracebloc CLI connects machines to tracebloc as clients and -manages the datasets that models train on. Your data stays on your -infrastructure โ€” models from other collaborators come to it, once you -approve them. - -Two kinds of commands: - - Your account (sign in first): login, logout, auth, client - This machine's client: data, cluster - -A typical first session: - - tracebloc login # sign in or create your account (browser) - tracebloc data ingest ./my-data # stage a dataset into your client - tracebloc data list # see what's in the cluster - -The CLI finds your cluster through your kubeconfig, stages data onto -the cluster's shared storage, and reports progress as it goes. No -Helm, no YAML, no kubectl needed. - -Usage: - tracebloc [flags] - tracebloc [command] - -Available Commands: - auth Inspect tracebloc authentication state - client Provision this machine's tracebloc client - cluster Inspect the cluster the CLI is currently targeting - completion Generate the autocompletion script for the specified shell - data Manage the datasets in your secure environment - delete Offboard this machine from tracebloc (revoke, uninstall, reclaim disk) - doctor Check your secure environment is connected and ready to run training - help Help about any command - login Sign in to tracebloc in your browser (device flow) - logout Sign out (revoke the token server-side and clear it locally) - resources Show how much of this machine tracebloc may use - version Print the tracebloc CLI version, git SHA, and build date - -Flags: - -h, --help help for tracebloc - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc [command] --help" for more information about a command. -$ tracebloc auth --help -Inspect tracebloc authentication state - -Usage: - tracebloc auth [flags] - tracebloc auth [command] - -Available Commands: - status Show whether you're signed in, and to which backend - -Flags: - -h, --help help for auth - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc auth [command] --help" for more information about a command. -$ tracebloc auth status --help -Show whether you're signed in, and to which backend - -Usage: - tracebloc auth status [flags] - -Flags: - --check exit 0 only if signed in with a backend-valid token, else 1; silent unless --verbose - --env string backend environment the check targets: dev|stg|prod (default: $CLIENT_ENV, then prod) - -h, --help help for status - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc client --help -Provision a tracebloc client for this machine. Requires sign-in first -(`tracebloc login`). To remove tracebloc from this machine, use -`tracebloc delete`. - -Usage: - tracebloc client [flags] - tracebloc client [command] - -Available Commands: - status Show whether tracebloc can see this machine's client (online) - -Flags: - -h, --help help for client - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc client [command] --help" for more information about a command. -$ tracebloc client create --help -Provision a tracebloc client for this machine (auto-named; no flags required) - -Usage: - tracebloc client create [flags] - -Flags: - --context string kubeconfig context for the target cluster (default: current-context) - --credential-file string write the machine credential to this path (mode 0600, sourceable env) instead of printing it โ€” for the installer to feed the chart (never shown on the terminal) - -h, --help help for create - --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) โ€” read to anchor the client to this cluster - --location string optional location zone for carbon reporting, e.g. DE (default: $TRACEBLOC_CLIENT_LOCATION; omitted if unset) - --name string client name (default: $TRACEBLOC_CLIENT_NAME, else auto-generated -NN; shown on your dashboard + carbon reports) - --yes skip the confirmation prompt - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc client list --help -List the clients in your account - -Usage: - tracebloc client list [flags] - -Aliases: - list, ls - -Flags: - -h, --help help for list - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc client status --help -Report tracebloc's view of this machine's active client โ€” online, offline, -or pending. With --wait, poll until tracebloc reports it online (exit 0) or the -timeout elapses (non-zero), to confirm the client connected after setup. - -Usage: - tracebloc client status [flags] - -Flags: - -h, --help help for status - --timeout duration with --wait, give up after this long (default 2m0s) - --wait poll until tracebloc reports this client online - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc cluster --help -Commands for inspecting the Kubernetes cluster the CLI is -configured to talk to. - -Use `cluster info` to verify which cluster, namespace, and -client the next `data ingest` will target. Useful as a -pre-flight before doing anything destructive (e.g. ingesting into -the wrong cluster). - -Usage: - tracebloc cluster [flags] - tracebloc cluster [command] - -Available Commands: - info Show the cluster, namespace, client install, and ingestor token state - -Flags: - -h, --help help for cluster - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc cluster [command] --help" for more information about a command. -$ tracebloc cluster doctor --help -Checks, in plain terms, whether your secure environment is connected to -tracebloc and ready to run training โ€” and if something's wrong, exactly what to -do about it. - - --verbose the full technical breakdown (for support) - --diagnose write a redacted support bundle to email to tracebloc - -Exit codes: - 0 healthy - 2 a problem was found - 3 couldn't read your local config - -Usage: - tracebloc cluster doctor [flags] - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - --diagnose write a redacted support bundle for tracebloc support and exit - -h, --help help for doctor - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your secure environment is installed (default: your active client's) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc cluster info --help -Discovers the tracebloc client installed in the configured -cluster + namespace and prints: - - โ€ข Which kubeconfig context the CLI used - โ€ข The namespace it resolved to - โ€ข The client's release name + chart version + appVersion - โ€ข The jobs-manager Service the next data ingest would POST to - โ€ข The ingestor ServiceAccount the post-install hook would auth as - โ€ข The cluster's configured INGESTOR_IMAGE_DIGEST default - โ€ข Whether the user's kubeconfig can mint short-lived SA tokens - via TokenRequest, or has to fall back to a static - service-account-token Secret - -The actual token bytes are never printed; the diagnostic shows -SHA256(token)[:8] so the customer can verify "yes, that's the -token I expect" without leaking it to terminal scrollback. - -Exit codes: - 0 cluster discovered + token mintable; CLI is ready - 4 cluster reachable but no tracebloc client found - 5 cluster reachable + release found but no usable SA token - -Usage: - tracebloc cluster info [flags] - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - -h, --help help for info - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') - --token-expiry-seconds int requested SA token expiration in seconds (default 600 = 10 min; ignored for static-secret fallback) (default 600) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc data --help -Commands for ingesting and managing the datasets your secure environment holds โ€” -the data models train on. It stays on your infrastructure. - -`data ingest` ingests a local dataset into your secure environment's storage, -submits the ingestion run, and watches it to completion (streaming -logs + the final summary). `data validate` checks an ingest.yaml -locally first. - -What a dataset looks like depends on the task: - tabular / time-series โ€” a .csv file, or a folder with one .csv - image โ€” a folder with labels.csv + images/ - text โ€” a folder with labels.csv + texts/ - -`tracebloc cluster info` is the pre-flight you'd typically run -before the first ingest. - -Usage: - tracebloc data [flags] - tracebloc data [command] - -Aliases: - data, dataset - -Available Commands: - delete Delete an ingested dataset's in-cluster artifacts (table + PVC files) - ingest Ingest a local dataset into your secure environment - list List datasets ingested in the cluster, with size / records / format - validate Validate an ingest.yaml against the embedded v1 schema, locally - -Flags: - -h, --help help for data - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc data [command] --help" for more information about a command. -$ tracebloc data delete --help -Removes the in-cluster artifacts a previous `data ingest` created -for a table: the MySQL table in training_test_datasets and the dataset's -directories on the shared PVC. Destructive and not undoable. - -The dataset's catalog metadata on the tracebloc backend is never removed โ€” it -is kept as a record on tracebloc, marked unavailable, so a collaborator's run -that referenced it still has its history. - -Exit codes: - 0 artifacts removed (or --dry-run, or the user declined) - 2 invalid table name - 3 kubeconfig error, or refused (no confirmation off a terminal) - 4 cluster reachable but no tracebloc client / shared storage missing, - or the client's dataset list couldn't be read (can't confirm the target) - 5 no dataset by that name on this client (nothing to delete) - 7 teardown failed mid-flight (table drop or PVC rm errored) - -With --output-json, stdout carries exactly one JSON result object per run -(human output goes to stderr) and the exit codes above are unchanged; see -docs/json-output.md for the shape and the stability promise. - -Usage: - tracebloc data delete
[flags] - -Aliases: - delete, rm - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - --dry-run show what would be deleted without deleting anything - -h, --help help for delete - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your tracebloc client is installed - --output-json emit the delete result as JSON on stdout (human output โ†’ stderr; never prompts โ€” pass --yes to delete, or --dry-run) - -y, --yes skip the confirmation prompt (required when not on a terminal) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc data ingest --help -Ingests a local dataset into your secure environment's storage, -submits the ingestion run, and follows it to completion (streaming -progress + the final summary). Your data never leaves your own -infrastructure. Supports 16 tasks across the image, text, and -tabular / time-series families; pick one with --task. - - is the data itself. What it looks like depends on the task: - - tabular / time-series โ€” the dataset is a single CSV. Pass the .csv - file directly, or a folder holding exactly one .csv: - - churn.csv (the .csv file itself) - or - churn/ - data.csv (the one .csv in the folder) - - image (classification, object/keypoint detection) โ€” a folder with - labels.csv + an images/ subfolder: - - cats_dogs/ - labels.csv (required) - images/ (required) - 001.jpg - ... - - text (classification, masked language modeling) โ€” a folder with - labels.csv + a texts/ subfolder (masked language modeling uses sequences/): - - reviews/ - labels.csv (required) - texts/ (required โ€” sequences/ for masked language modeling) - 001.txt - ... - -A bare .csv file is accepted only for the tabular / time-series family; -image and text datasets must be a folder. - -Accepted image extensions: .jpg, .jpeg, or .png (case-insensitive). -All images in one dataset must share a single type โ€” the cluster -validates the type it was told to expect. - -v0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger -datasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€” -see tracebloc/client#147 non-goals. - -Exit codes: - 0 files staged + ingested successfully (or --detach: just staged + submitted) - 2 schema validation failed (synthesized spec rejected) or - v0.1-unsupported task passed - 3 local-layout or kubeconfig error - 4 cluster reachable but no tracebloc client / shared storage missing - 5 ingestor SA token couldn't be obtained, or jobs-manager - rejected the token (401/403) - 6 destination table already exists (re-run with --overwrite to - replace it, or pick a different --name) - 7 pre-flight succeeded but staging the files failed - (Pod creation, image pull, exec stream, or remote tar error) โ€” - or, with --overwrite, removing the old table failed - 8 jobs-manager rejected the submit (4xx/5xx other than auth) - 9 ingestion Job exited non-zero, or completed with row-level - failures the summary panel reports - -Usage: - tracebloc data ingest [flags] - -Aliases: - ingest, push - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - --detach kubectl logs -f -n job/ exit immediately after jobs-manager accepts the run (no log streaming, no summary panel). Use for CI scenarios; reconnect later with kubectl logs -f -n job/. - --dry-run validate + discover + walk, but don't create any cluster resources - -h, --help help for ingest - --idempotency-key string reuse this idempotency key across retry attempts (default: fresh per invocation). jobs-manager treats a duplicate key as a replay and attaches to the existing Job rather than spawning a new one โ€” useful for at-most-once-across-attempts semantics. - --image-digest images.ingestor.digest pin the ingestor container image to a specific digest (default: jobs-manager picks the cluster-configured images.ingestor.digest). Format: sha256:. - --intent string is this training or test data? train|test (default train) - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - --label-column string name of the label/target column (in labels.csv for image tasks, in the data CSV for tabular) - --label-policy string regression-class only (tabular_regression, time_series_forecasting, time_to_event_prediction): passthrough|bucket (default bucket โ€” bins the target so the raw value never leaves the cluster) - --min-size string image tasks only: reject images smaller than WxH before the ingest (e.g. 64x64). Set it to the smallest size your model can train on โ€” raise or lower it freely. Default: unset (no local size check). - --name string a name for this dataset โ€” start with a letter or underscore, then letters/digits/underscores โ€” you'll reference it by this name when you start a training run - -n, --namespace string namespace where your tracebloc client is installed - --no-input disable interactive prompts; fail on missing required values (for CI/scripts) - --number-of-keypoints int keypoint_detection only: number of keypoints per sample (required; e.g. 17 for COCO pose) - --output-json emit a machine-readable JSON result on stdout (human output โ†’ stderr; implies --no-input) - --overwrite tracebloc data delete replace the destination table if it already exists: its current table + files are removed first (same as tracebloc data delete), then the new data is ingested. Not combinable with --idempotency-key - --schema string tabular/time-series only: column types as col:TYPE,col:TYPE (e.g. age:INT,price:FLOAT). Default: inferred from the CSV (INT/BIGINT/FLOAT/BOOLEAN/DATE/DATETIME/VARCHAR(n)). - --stage-pod-image string override the ephemeral stage Pod's image (default: digest-pinned alpine 3.20 baked into the CLI). Pin by digest in your override too โ€” tag-only refs drift silently. - --target-size string image tasks only: the resolution your images already are, as WxH (e.g. 512x512). tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Default: read from your first image. - --task string the task this data is for, one of: image_classification, object_detection, keypoint_detection, text_classification, masked_language_modeling, tabular_classification, tabular_regression, time_series_forecasting, time_series_classification, time_to_event_prediction, causal_language_modeling, seq2seq, token_classification, sentence_pair_classification, embeddings, semantic_segmentation. Omit it on a terminal to pick interactively. - --time-column string time_to_event_prediction only: name of the time/duration column (default: a column named "time") - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc data list --help -Lists the datasets ingested into your client โ€” the tables in training_test_datasets -on the cluster โ€” grouped by modality, with each dataset's split (train/test), -record count, size, format, and when it was ingested. - -With no flags it uses your current kubeconfig context and its namespace; -the flags below override that, same as `cluster info` and `data ingest`. -Framework tables (the ingest-run journal) are hidden unless you pass --all. -For the full catalog, see the dashboard at https://ai.tracebloc.io/metadata. - -Exit codes: - 0 listed successfully (including an empty list) - 3 kubeconfig error - 4 cluster reachable but no tracebloc client in the namespace - 7 couldn't query the cluster for datasets - -Usage: - tracebloc data list [flags] - -Flags: - --all include framework/system tables (e.g. the ingest-run journal), normally hidden - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - -h, --help help for list - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your tracebloc client is installed - --output-json emit the dataset list as JSON on stdout (human output โ†’ stderr) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc data validate --help -Reads , parses it as YAML, and validates it against the bundled -ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints -violations in the same JSON-pointer-prefixed format the cluster's -jobs-manager uses, and exits non-zero if any are found. - -Useful as a pre-flight before running `tracebloc data ingest` โ€” -millisecond local feedback instead of a multi-second cluster round -trip. - -Exit codes: - 0 YAML parses and validates cleanly - 2 YAML parses but has schema violations (printed to stderr) - 3 YAML doesn't parse or file isn't readable - -Usage: - tracebloc data validate [flags] - -Flags: - -h, --help help for validate - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc delete --help -Removes tracebloc from this machine: revokes the machine credential, -uninstalls the Helm release, deletes the local cluster, reclaims the tracebloc -container images, and clears local state โ€” then removes the CLI itself. - -Your use cases, datasets' catalog entries, and the models trained here are KEPT -on tracebloc as a record (a colleague's model must not vanish because you -reclaimed this box). System software the installer laid down โ€” Docker, kubectl, -k3d, helm, NVIDIA drivers โ€” is left in place; remove it yourself if unused. - -Destructive: on a single-host install the on-prem datasets live on this machine -and are erased. Not undoable. - -Usage: - tracebloc delete [flags] - -Flags: - --context string kubeconfig context for the target cluster (default: current-context) - --force offboard even if tracebloc still reports this client online - -h, --help help for delete - --keep-data uninstall the software but keep ~/.tracebloc (local config + on-host datasets) - --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace of this machine's tracebloc release (default: the active client's namespace) - --yes skip the typed-name confirmation (for automation) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc doctor --help -Checks, in plain terms, whether your secure environment is connected to -tracebloc and ready to run training โ€” and if something's wrong, exactly what to -do about it. - - --verbose the full technical breakdown (for support) - --diagnose write a redacted support bundle to email to tracebloc - -Exit codes: - 0 healthy - 2 a problem was found - 3 couldn't read your local config - -Usage: - tracebloc doctor [flags] - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - --diagnose write a redacted support bundle for tracebloc support and exit - -h, --help help for doctor - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your secure environment is installed (default: your active client's) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc ingest --help -Deprecated alias for `tracebloc data validate` - -Usage: - tracebloc ingest [flags] - tracebloc ingest [command] - -Available Commands: - validate Validate an ingest.yaml against the embedded v1 schema, locally - -Flags: - -h, --help help for ingest - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc ingest [command] --help" for more information about a command. -$ tracebloc ingest validate --help -Reads , parses it as YAML, and validates it against the bundled -ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints -violations in the same JSON-pointer-prefixed format the cluster's -jobs-manager uses, and exits non-zero if any are found. - -Useful as a pre-flight before running `tracebloc data ingest` โ€” -millisecond local feedback instead of a multi-second cluster round -trip. - -Exit codes: - 0 YAML parses and validates cleanly - 2 YAML parses but has schema violations (printed to stderr) - 3 YAML doesn't parse or file isn't readable - -Usage: - tracebloc ingest validate [flags] - -Flags: - -h, --help help for validate - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc login --help -Sign in to tracebloc. The CLI prints a URL + short code; open the URL -on any device (your laptop or phone), sign in the way you already do -(password, Google, or GitHub), and approve the code. The CLI stores a -user token in ~/.tracebloc (mode 0600). - -Works on a headless / SSH box โ€” the browser and the CLI need not share a -machine. Honors HTTP(S)_PROXY / NO_PROXY for corporate-proxy networks. - -Usage: - tracebloc login [flags] - -Flags: - --env string backend environment: dev|stg|prod (default: $CLIENT_ENV, then prod) - -h, --help help for login - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc logout --help -Sign out (revoke the token server-side and clear it locally) - -Usage: - tracebloc logout [flags] - -Flags: - -h, --help help for logout - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc resources --help -Shows, in plain terms, how much of this machine tracebloc may use: - - โ€ข Your secure environment โ€” the CPU and memory it can schedule - โ€ข Each training run โ€” the per-run ceiling every run may use (cluster-wide) - -No Kubernetes concepts, no YAML โ€” one number for your environment and one for -each training run's share of it. - -Raise the share with `tracebloc resources set`. Run with --verbose for the -per-node breakdown and the raw values. - -Exit codes: - 0 shown - 3 kubeconfig could not be loaded / cluster unreachable - 4 cluster reachable but no tracebloc client found here - -Usage: - tracebloc resources [flags] - tracebloc resources [command] - -Available Commands: - set Raise how much of this machine tracebloc may use - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - -h, --help help for resources - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - -Use "tracebloc resources [command] --help" for more information about a command. -$ tracebloc resources set --help -Raise the per-training-run ceiling โ€” how much of this machine a single -training run may use. - -Run it on a terminal with no flags for a guided walkthrough: - - tracebloc resources set - -Or set it directly (for scripts / non-interactive shells): - - tracebloc resources set --cores 4 --memory 16Gi an explicit per-run ceiling - tracebloc resources set --cores 4 change CPU only, keep the rest - tracebloc resources set max let a run use the whole machine - -The number you set is what ONE training run may use. tracebloc keeps a small fixed -amount (about 1 core and 3 GiB) for itself on top โ€” you never have to subtract it. -The new ceiling applies to your NEXT training run; a run already going keeps its -size. - -Exit codes: - 0 applied (or nothing to change) - 2 the requested size doesn't fit this machine / bad input - 3 kubeconfig could not be loaded / cluster unreachable - 4 cluster reachable but no tracebloc client found here - -Usage: - tracebloc resources set [max] [flags] - -Flags: - --context string name of the kubeconfig context to use (default: kubeconfig's current-context) - --cores string CPU cores one training run may use (e.g. 4) - --dry-run show exactly what would change and apply nothing - --gpus int whole GPUs one training run may use (only on a GPU machine) - -h, --help help for set - --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) - --memory string memory one training run may use (e.g. 16 or 16Gi โ€” the number is GiB) - -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') - --yes skip the confirmation prompt (for automation) - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) -$ tracebloc version --help -Print the tracebloc CLI version, git SHA, and build date - -Usage: - tracebloc version [flags] - -Flags: - -h, --help help for version - --output-json emit the version payload as indented JSON instead of a single human-readable line - -Global Flags: - --plain disable color and decorative output (also honors $NO_COLOR) - --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) - - -============================================================================== -= SCREENS โ€” byte-exact renderer output -============================================================================== -$ tb # home ยท Online - - - Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โœ“ Secure environment "hello-world" ยท Online (12 CPU ยท 23 GiB) - - - Your data - - ยท tb data ingest load a dataset into your secure environment - ยท tb data list list your datasets - ยท tb data delete remove a dataset - - - Your secure environment - - ยท tb resources manage compute & memory - ยท tb doctor check the connection & diagnose issues - ยท tb delete remove tracebloc from this machine - - - Add --help to any command for the flags. - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท running (couldn't confirm) - - - Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โš  Secure environment "hello-world" ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run tb doctor - - - Your data - - ยท tb data ingest load a dataset into your secure environment - ยท tb data list list your datasets - ยท tb data delete remove a dataset - - - Your secure environment - - ยท tb resources manage compute & memory - ยท tb doctor check the connection & diagnose issues - ยท tb delete remove tracebloc from this machine - - - Add --help to any command for the flags. - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท running (backend not online) - - - Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โš  Secure environment "hello-world" ยท running, but tracebloc hasn't heard from it โ€” run tb doctor - - - Your data - - ยท tb data ingest load a dataset into your secure environment - ยท tb data list list your datasets - ยท tb data delete remove a dataset - - - Your secure environment - - ยท tb resources manage compute & memory - ยท tb doctor check the connection & diagnose issues - ยท tb delete remove tracebloc from this machine - - - Add --help to any command for the flags. - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท starting up - - - Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โš  Secure environment "hello-world" ยท starting up, not ready yet โ€” run tb doctor - - - Your data - - ยท tb data ingest load a dataset into your secure environment - ยท tb data list list your datasets - ยท tb data delete remove a dataset - - - Your secure environment - - ยท tb resources manage compute & memory - ยท tb doctor check the connection & diagnose issues - ยท tb delete remove tracebloc from this machine - - - Add --help to any command for the flags. - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท offline - - - Welcome to your secure environment for AI, Lukas ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โœ— Secure environment "hello-world" ยท can't reach it from here โ€” run tb doctor - - - Your data - - ยท tb data ingest load a dataset into your secure environment - ยท tb data list list your datasets - ยท tb data delete remove a dataset - - - Your secure environment - - ยท tb resources manage compute & memory - ยท tb doctor check the connection & diagnose issues - ยท tb delete remove tracebloc from this machine - - - Add --help to any command for the flags. - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท no secure environment - - - Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ“ Signed in as lukas@tracebloc.io - โš  No secure environment on this machine yet โ€” run the installer to set one up. - - - Your secure environment - - ยท tb doctor check the connection & diagnose issues - - - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb # home ยท not signed in - - - Welcome to tracebloc โ€” your secure environment for AI ๐Ÿ‘‹ - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - โœ— Not signed in yet. - - - Start here - - ยท tb login sign in to tracebloc - - - - โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - - love from tracebloc ๐Ÿ’š - -$ tb data list # empty - - Datasets in hello-world (0) - - No datasets yet โ€” ingest one with `tracebloc data ingest`. -$ tb data list # populated - - Datasets in hello-world โ€” 2 ยท 1.25 GiB - 1 system table(s) hidden โ€” show with --all. - - Image classification ยท 2 - โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” - โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” -$ tb data list --all - - Datasets in hello-world โ€” 2 ยท 1.25 GiB - - Image classification ยท 2 - โœ” xray_test test 3000 images 256.00 MiB jpg ยท 2 classes โ€” - โœ” xray_train train 12000 images 1.00 GiB jpg ยท 2 classes โ€” - - System ยท 1 - ยท ingest_run_journal โ€” -$ tb data ingest ./data # pre-flight review - - Review - name: xray_train - task: image_classification - intent: train - path: ./data - resolution: auto-detect -$ tb client create # review - - Review - name: lukas-macbook - namespace: lukas-macbook - location: DE - cluster: a1b2c3d4 (anchors this client โ€” re-runs adopt it) -$ tb delete # keep data - - This will remove - ยท This machine's credential โ€” so tracebloc can no longer reach it - ยท Your secure environment "lukas-macbook" and everything it runs on this machine - ยท tracebloc's downloaded images - ยท The tracebloc CLI (your local data & config are kept โ€” --keep-data) - - Kept on tracebloc - ยท Your use cases and the models trained here - ยท Your dataset records (marked unavailable, not deleted) - - Left alone - ยท Docker and related tools โ€” remove them yourself if you no longer need them -$ tb delete # remove data - - This will remove - ยท This machine's credential โ€” so tracebloc can no longer reach it - ยท Your secure environment "lukas-macbook" and everything it runs on this machine - ยท tracebloc's downloaded images - ยท Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone - - Kept on tracebloc - ยท Your use cases and the models trained here - ยท Your dataset records (marked unavailable, not deleted) - - Left alone - ยท Docker and related tools โ€” remove them yourself if you no longer need them - - -============================================================================== -= MESSAGE INDEX โ€” every user-facing string in the source (templates, not -= rendered; %s/%d are runtime placeholders). Catches the multi-step flows the -= transcript above can't show whole: ingest steps + progress, login device flow, -= delete confirmation, and every error/hint. -============================================================================== - - "\"active\" is this machine's selected client; state is its last reported status to tracebloc." - "%d system table(s) hidden โ€” show with --all." - "%q exists but is not a directory" - "%q is a directory, not a file" - "%q is not a directory; pass the directory containing labels.csv + images/" - "%q is not a directory; pass the directory containing labels.csv + the text files" - "%s %q must be WxH (e.g. 512x512)" - "%s %q: height is not an integer: %w" - "%s %q: width and height must both be positive" - "%s %q: width is not an integer: %w" - "%s %s โ€” %s" - "%s has a header but no data rows (0 ingestable records). Add at least one data row and re-run." - "%s is empty โ€” add a header and at least one data row, then re-run" - "%s is empty โ€” no header row" - "%s is image tasks only; it doesn't apply to task %q" - "%s requires CLIENT_WRITE permission" - "%s ยท Online%s" - "%s ยท can't reach it from here โ€” run %s" - "%s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s" - "%s ยท running, but tracebloc hasn't heard from it โ€” run %s" - "%s ยท starting up, not ready yet โ€” run %s" - "%s โ€” %s ยท %s" - "%s โ€” %s ยท %s (%s)" - "%s: %w" - "--label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column" - "--number-of-keypoints is keypoint_detection only; it doesn't apply to task %q" - "--overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default)." - "--schema is empty; expected col:TYPE,col:TYPE,..." - "--schema is tabular/time-series tasks only; it doesn't apply to task %q" - "--time-column is time_to_event_prediction only; it doesn't apply to task %q" - "--timeout has no effect without --wait" - "A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train" - "A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load)." - "A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it." - "A training run is allocated up to:" - "Add --help to any command for the flags." - "Already signed out." - "Applies to your next training run; a run already going keeps its size." - "Ask one of these admins (or ask them to grant you access)" - "Available now:" - "CSV %s has no columns" - "Can't reach tracebloc from here." - "Cancelled โ€” %q was left as-is; nothing was ingested." - "Cancelled โ€” nothing was changed." - "Cancelled โ€” nothing was deleted." - "Cancelled โ€” nothing was ingested." - "Cancelled โ€” the name didn't match. Nothing was removed." - "Cancelled." - "Chart uninstall reported: %v" - "Check on it later with: kubectl logs -f -n %s job/%s" - "Check your data" - "Check your network / HTTP(S)_PROXY, then run `%s doctor` again." - "Client install" - "Client status" - "Clients in your account" - "Cluster teardown reported: %v" - "Connecting to your secure environmentโ€ฆ" - "Copy into your secure environment" - "Correlation id: %s" - "Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you." - "Couldn't connect to your secure environment โ€” check your kubeconfig/context." - "Couldn't locate the CLI binary to remove it (%v) โ€” delete it by hand." - "Couldn't read the target cluster's identity โ€” provisioning without a cluster anchor, so re-running won't be idempotent. Point --kubeconfig/--context at the reachable cluster to enable that." - "Couldn't read your tracebloc config โ€” run `%s login` to recreate it." - "Couldn't reclaim the temporary copy (%v). It's harmless โ€” the next re-ingest of %q or a `tracebloc data delete %s` will clear it." - "Couldn't remove the CLI (%v) โ€” remove it by hand: rm -f %s" - "Couldn't remove the CLI (%v). It looks Homebrew-managed โ€” finish with: brew uninstall tracebloc" - "Couldn't remove the `tb` alias (%v) โ€” remove it by hand: rm -f %s" - "Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client) to set it." - "Couldn't save the active-client pointer (%v) โ€” re-run `tracebloc client create` (it adopts this cluster's client)." - "Couldn't verify your session with the backend (%v)." - "Couldn't write the support bundle: %v" - "Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d." - "DB failures" - "Deleted %s.%s and %d PVC path(s)." - "Destructive and cannot be undone." - "Detached โ€” the ingestion runs in the background on your secure environment." - "Details" - "Details (for support)" - "Diagnose auth / cluster problems with: tracebloc doctor" - "Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal)." - "Docker and related tools โ€” remove them yourself if you no longer need them" - "Dry run โ€” nothing was changed" - "Dry-run complete โ€” your data and secure environment check out; nothing was created." - "Dry-run โ€” nothing was deleted." - "Each training run already uses up to %s โ€” nothing to change." - "Each training run may now use up to %s." - "Email it to support@tracebloc.io." - "Enter" - "Everything looks good โ€” you're ready to run training." - "Follow it later with: kubectl logs -f -n %s job/%s" - "Full log: %s" - "GPU access removed โ€” training runs will use CPU only." - "How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" - "How much of this machine a training run may use" - "Ingest settings" - "Ingestion complete โ€” %s" - "Ingestion complete โ€” showing its logs:" - "Ingestion completed partially โ€” %s" - "Ingestion completed with failures โ€” %s" - "Ingestion completed with skips โ€” %s" - "Ingestion started โ€” live progress:" - "Ingestion started โ€” streaming logs:" - "Ingestion summary" - "Ingestor SA token" - "Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." - "Kept on tracebloc" - "Kubeconfig" - "Learn more: https://docs.tracebloc.io" - "Left %s in place โ€” it isn't tracebloc's `tb` alias." - "Left alone" - "Let's set up your data ingest" - "Local dataset" - "Machine credential โ€” needed by the installer to connect this client" - "Memory" - "No client in namespace %q โ€” using the one in %q (override with --namespace)." - "No clients yet. Run `tracebloc client create`." - "No new credential issued; the existing one stands. This machine is set to enroll as client %d." - "No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail." - "No secure environment on this machine yet โ€” run the installer to set one up." - "No secure environment on this machine yet." - "Not signed in yet." - "Not signed in โ€” run `%s login`." - "Not signed in. Run `tracebloc login`." - "Not yet in the CLI:" - "Offboarded %q. This machine is no longer connected to tracebloc." - "Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." - "Open" - "Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR" - "POST %s%s: %w" - "PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter" - "Pick this dataset when you set it up." - "Press Enter to accept a default; Ctrl-C to cancel." - "Provisioned client %q (namespace %s)." - "Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent):" - "Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away." - "Ready for `tracebloc data ingest`." - "Reclaimed tracebloc's downloaded images." - "Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values." - "Removed local tracebloc data and config." - "Removed stray control characters from the name." - "Removed the local environment." - "Removed the old %q โ€” ingesting the new data." - "Removed the tracebloc CLI from this machine." - "Removing in-cluster artifactsโ€ฆ" - "Review" - "Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record." - "Set one up: %s" - "Sign in to tracebloc" - "Signed in" - "Signed in as %s" - "Signed in as %s." - "Signed in to %q, but this run targets %q โ€” run `tracebloc login`." - "Signed in." - "Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine." - "Signed out." - "Signed-in token was rejected by the backend โ€” run `tracebloc login`." - "Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`." - "Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`." - "Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own." - "Stopped watching โ€” the ingestion keeps running on your secure environment." - "Submitted โ€” tracebloc is validating your data and loading it into the table." - "Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below." - "Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below." - "Table %q already exists โ€” replacing it (table + files)." - "Target" - "Target cluster" - "The column holding the duration / time-to-event. e.g. time, tenure_days" - "The column in your CSV with the answer the model learns to produce." - "The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed." - "The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn" - "The name you provided was only control characters โ€” auto-naming this client instead." - "The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224" - "The tracebloc CLI (your local data & config are kept โ€” --keep-data)" - "This CLI is out of date โ€” update it: %s" - "This cluster is already registered as client %q (namespace %s) โ€” adopted it." - "This drops the table and removes the files listed above โ€” there's no undo. Pass --yes next time to skip this prompt." - "This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later)." - "This is irreversible. Type the client name to confirm, or leave blank to cancel." - "This machine's credential โ€” so tracebloc can no longer reach it" - "This matches a previous run (same idempotency key) โ€” attaching to the run already in progress." - "This permanently removes a dataset you ingested earlier: it drops the table from\nthe cluster and deletes the dataset's files on the shared storage. It can't be\nundone โ€” re-ingesting the data is the only way back." - "This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account." - "This will remove" - "To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases" - "Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." - "Uninstalled tracebloc." - "Validate and load" - "We couldn't tell the data type from what's there โ€” which is it?" - "What's next" - "Whether this split trains the model or evaluates it." - "Will delete" - "Wrote a support bundle to ./%s" - "Wrote client id + namespace to %s (no new credential โ€” the existing one stands)." - "You don't have permission to %s in this account." - "Your CPU and memory budget is unchanged โ€” but this machine has no GPU while the cluster still requests one, so I'll clear that stale GPU setting so runs can schedule." - "Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata" - "Your dataset records (marked unavailable, not deleted)" - "Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you." - "Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone" - "Your secure environment %q and everything it runs on this machine" - "Your secure environment is equipped with:" - "Your session expired โ€” run `%s login`." - "Your use cases and the models trained here" - "a dataset path is required" - "a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually" - "a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read" - "account" - "active client" - "annotations" - "app version" - "authorized โ€” confirming the token with the backend โ€ฆ" - "auto-detect" - "backend" - "backend %s โ€” requesting a device code โ€ฆ" - "backfilling the cluster anchor onto the existing client: %w" - "building SPDY transport: %w" - "building rest config from kubeconfig: %w" - "building submit request: %w" - "building tar archive: %w" - "can't read %q: %w" - "cancelled by user" - "chart version" - "checking %s for a byte-order mark: %w" - "client" - "client id" - "closing tar writer: %w" - "cluster" - "columns" - "command" - "connected: %s โ€” %s" - "constructing kubernetes clientset: %w" - "context" - "couldn't read capacity: %v" - "couldn't read this machine's capacity: %w" - "creating SPDY executor for %s/%s: %w" - "creating credential-file directory: %w" - "creating port-forwarder: %w" - "creating stage Pod in namespace %q: %w" - "creating staging-cleanup pod: %w" - "creating teardown pod: %w" - "dashboard id" - "data CSV" - "dataset exceeded v0.1 total cap of %s after streaming %s (reached %s)" - "dataset name is required (set --name)" - "dataset name is required โ€” pass it as an argument: tracebloc data delete " - "decoding image header %q: %w" - "decoding submit response (got body %q): %w" - "deleting stage Pod %s/%s: %w" - "destination" - "dropping %s.%s: %w%s" - "enter a whole number between %d and %d" - "exec stream against %s/%s: %w" - "expires" - "expires in" - "field %d is empty โ€” every field (%s) must be non-empty" - "file failures" - "full log: %s" - "generating Pod-name random suffix: %w" - "generating idempotency key: %w" - "generating staging-dir suffix: %w" - "images" - "infer from CSV" - "inferring schema from CSV: %w" - "ingestion Job completed but the summary reports failures โ€” see panel above" - "ingestion Job exited non-zero โ€” see logs above" - "ingestor ID" - "ingestor SA" - "ingestor img" - "inserted" - "intent" - "interactive setup: %w" - "internal: re-parsing synthesized spec: %w\n%s" - "invalid table name %q: %w" - "jobs-manager" - "jobs-manager: %s" - "keypoints" - "kube-system namespace has no UID" - "label column" - "label policy" - "labels.csv" - "listing Pods for service %s/%s: %w" - "listing chart-managed deployments in namespace %s: %w" - "listing client deployments to check for an existing client: %w" - "listing service-account-token secrets in %s: %w" - "listing stage Pods in %s: %w" - "loading embedded schema: %w" - "loading kubeconfig: %w" - "locating mysql pod: %w" - "location" - "login timed out โ€” re-run `tracebloc login`" - "love from tracebloc ๐Ÿ’š" - "marshaling submit request: %w" - "marshaling synthesized spec: %w" - "masks" - "min size" - "minting token for ServiceAccount %s/%s via TokenRequest: %w" - "missing %s/ subdirectory in %q" - "must be a positive integer" - "must be between %d and %d" - "mysql table" - "name" - "namespace" - "never (static-secret fallback)" - "no CLI-supported tasks for %s data yet" - "no Ready node on this machine to size a training run against" - "no Running pod with name containing %q in namespace %q" - "no active client on this machine โ€” nothing to offboard" - "no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first" - "no dataset named %q on this client%s" - "no image files to detect a type from" - "no such file or directory: %q โ€” check the path to your dataset" - "no tracebloc client found" - "none detected" - "not signed in โ€” run `tracebloc login` first" - "outcome: early exit before the cluster was probed" - "outcome: early exit โ€” no roll-up verdict (granular checks below)" - "overwrite prompt: %w" - "packaging %s: %w" - "packaging labels.csv: %w" - "password" - "path" - "port-forward allocated zero ports" - "port-forward to %s/%s failed during startup: %w" - "pvc path" - "querying datasets: %w%s" - "reading %q: %w" - "reading %s header: %w" - "reading %s/: %w" - "reading %s: %w" - "reading CSV header from %s: %w" - "reading CSV row from %s: %w" - "reading PVC %s/%s: %w" - "reading allocated port: %w" - "reading dataset directory %q: %w" - "reading dataset path %q: %w" - "reading final Job status for %s/%s: %w" - "reading images/: %w" - "reading kube-system namespace UID: %w" - "reading labels.csv: %w" - "reading raw kubeconfig: %w" - "reading service %s/%s: %w" - "reading submit response body: %w" - "reading the existing client's identity in namespace %q: %w" - "ready: %s โ€” %s" - "refusing to change the ceiling without confirmation: pass --yes, or run on a terminal" - "refusing to delete without confirmation: pass --yes or run on a terminal" - "refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name" - "refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover)" - "release" - "release: %s (chart %s)" - "removed โ€” runs will use CPU only" - "removing PVC paths: %w%s" - "removing staged copy %s: %w%s" - "resolution" - "resolving %q: %w" - "resolving Service %s/%s to a Pod: %w" - "resolving namespace from kubeconfig: %w" - "resource env" - "root" - "scanning the cluster for tracebloc clients: %w" - "schema" - "schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT)" - "sent to API" - "server" - "service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding" - "session: %s" - "setting up jobs-manager port-forward: %w" - "sha256[:8]" - "shared PVC" - "shared PVC: %s (%s)" - "sign-in was denied in the browser" - "signed in" - "skipped" - "source" - "stage Pod %s/%s did not become Ready within %s%s" - "stage Pod %s/%s did not reach Ready state: %w%s" - "stage Pod %s/%s terminated in phase %q before becoming Ready%s" - "stat %q: %w" - "stat %s/: %w" - "stat %s: %w" - "stat images/: %w" - "stat labels.csv: %w" - "state" - "status" - "stored active client id %q is not numeric: %w" - "streaming files to %s/%s: %w%s" - "streaming logs from Pod %s/%s: %w" - "submit response missing job_name (got body %q)" - "submit response missing namespace (got body %q)" - "success rate" - "synthesized spec failed schema validation; check the flag values above" - "task" - "task %q isn't a recognized task. Supported tasks: %s." - "task %q isn't supported by the CLI yet%s. Supported tasks: %s." - "teardown failed: %w" - "the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster" - "the sign-in code expired โ€” re-run `tracebloc login`" - "time column" - "token saved to ~/.tracebloc (0600)" - "total records" - "total size" - "tracebloc auth" - "tracebloc can see this client." - "tracebloc didn't confirm your session (server error)." - "tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine." - "tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry" - "tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete`" - "tracebloc's downloaded images" - "unavailable" - "values:" - "waiting for ingestor Pod: %w" - "waiting for staging-cleanup pod: %w" - "waiting for teardown pod: %w" - "watching ingestor Job: %w" - "would set each run to" - "writing credential file %s: %w" From 5656193e36d6a2ff3a3a94576ad56cbbcf15ea8b Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 08:38:55 +0200 Subject: [PATCH 06/14] Catalog: complete the per-command folder + fix the build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a golden file for every remaining command, in the order a user hits them: 01-data-ingest, 03-data-delete, 04-resources, 05-doctor, 06-delete, 07-login, 08-client, 09-cluster, 10-version. Each leads with the byte-exact screens we can render deterministically (ingest review, doctor rollup via the real summarizeDoctor/renderHealth/doctorVerdict, delete + client review) and ends with the command's --help. Flow-only copy (ingest steps, login device flow, confirmations, and the launcher-dependent failure remedies) can't be a stable screen, so it stays in the zz-all-strings backstop, which each file points to. Strengthens the backstop harvest to also catch fmt.Sprintf templates and the text/remedy fields of healthLine{} / doctor.Result{} literals โ€” user-facing copy that never passes through a Printer call and so was missing before (417 -> 556 strings; e.g. every "Not connected โ€” โ€ฆ" / "Not ready โ€” โ€ฆ" line). Fixes the build: the prior commit left internal/cli/screens_golden_test.go in the tree alongside copy_catalog_test.go, so harvestMessages was defined twice (the deletion never got staged). This commit stages the deletion. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 271 +++++++++++++++--- internal/cli/screens_golden_test.go | 223 -------------- .../cli/testdata/golden/01-data-ingest.golden | 143 +++++++++ .../cli/testdata/golden/03-data-delete.golden | 50 ++++ .../cli/testdata/golden/04-resources.golden | 90 ++++++ internal/cli/testdata/golden/05-doctor.golden | 85 ++++++ internal/cli/testdata/golden/06-delete.golden | 69 +++++ internal/cli/testdata/golden/07-login.golden | 58 ++++ internal/cli/testdata/golden/08-client.golden | 91 ++++++ .../cli/testdata/golden/09-cluster.golden | 71 +++++ .../cli/testdata/golden/10-version.golden | 27 ++ .../cli/testdata/golden/zz-all-strings.golden | 149 +++++++++- 12 files changed, 1059 insertions(+), 268 deletions(-) delete mode 100644 internal/cli/screens_golden_test.go create mode 100644 internal/cli/testdata/golden/01-data-ingest.golden create mode 100644 internal/cli/testdata/golden/03-data-delete.golden create mode 100644 internal/cli/testdata/golden/04-resources.golden create mode 100644 internal/cli/testdata/golden/05-doctor.golden create mode 100644 internal/cli/testdata/golden/06-delete.golden create mode 100644 internal/cli/testdata/golden/07-login.golden create mode 100644 internal/cli/testdata/golden/08-client.golden create mode 100644 internal/cli/testdata/golden/09-cluster.golden create mode 100644 internal/cli/testdata/golden/10-version.golden diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index c737dc43..4d1bf0f4 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -2,6 +2,7 @@ package cli import ( "bytes" + "fmt" "go/ast" "go/parser" "go/token" @@ -12,6 +13,7 @@ import ( "strings" "testing" + "github.com/tracebloc/cli/internal/doctor" "github.com/tracebloc/cli/internal/push" "github.com/tracebloc/cli/internal/ui" ) @@ -19,17 +21,19 @@ import ( // TestCopyCatalog generates the copy catalog under testdata/golden/ โ€” ONE file // per command, so every user-facing string can be reviewed a screen at a time // without deploying. Each file leads with what you see when you RUN the command -// (byte-exact, colour off), covers every state/path, and ends with its `--help`. -// zz-all-strings.golden is a completeness backstop: every user-facing string in -// the source, so nothing is missed even on a rare path. +// (byte-exact, colour off), covers every state/path we can render deterministically, +// and ends with the command's `--help`. Copy that only appears mid-flow (ingest +// steps + progress, the login device flow, delete confirmation, and every +// failure remedy โ€” many of which embed the launcher name, which varies by +// install) can't be pinned as a stable screen; zz-all-strings.golden is the +// completeness backstop for those โ€” every user-facing string in the source. // // The test fails on drift; regenerate after an intentional copy change: // // TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestCopyCatalog // -// Files are ordered by how often a user hits them (00 = most). More command -// files (ingest run, resources, doctor, delete, login) land as their driven -// transcripts are wired; this generator makes adding one a single entry below. +// Files are ordered by how often a user hits them (00 = most). Adding a command +// is a single entry in the files map below. func TestCopyCatalog(t *testing.T) { bi := BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"} @@ -51,12 +55,14 @@ func TestCopyCatalog(t *testing.T) { return b.String() } - // doc assembles one command file: a title, then labelled `$ cmd` blocks - // (verbatim output), then the command's --help at the bottom. + // doc assembles one command file: a title, a "when you see this" note, then + // labelled `$ cmd` blocks (verbatim output), then the command's --help + // block(s) at the bottom (one command can have several โ€” e.g. client has + // create/list/status). type run struct { cmd, out string // "$ " then verbatim } - doc := func(title, whenSeen string, runs []run, helpCmd string, helpOut string) string { + doc := func(title, whenSeen string, runs []run, helps []run) string { var s strings.Builder s.WriteString(title + "\n" + strings.Repeat("=", len([]rune(title))) + "\n") s.WriteString(whenSeen + "\n") @@ -64,15 +70,20 @@ func TestCopyCatalog(t *testing.T) { s.WriteString("\n$ " + r.cmd + "\n") s.WriteString(r.out) } - if helpOut != "" { + if len(helps) > 0 { s.WriteString("\n\n" + strings.Repeat("-", 60) + "\n--help\n" + strings.Repeat("-", 60) + "\n") - s.WriteString("$ " + helpCmd + "\n") - s.WriteString(helpOut) + for i, h := range helps { + if i > 0 { + s.WriteString("\n") + } + s.WriteString("$ " + h.cmd + "\n") + s.WriteString(h.out) + } } return s.String() } - // โ”€โ”€ home models (every state resolveHomeModel can produce) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + // โ”€โ”€ 00 home โ€” every state resolveHomeModel can produce โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ online := homeModel{ state: homeOnline, email: "lukas@tracebloc.io", name: "Lukas", envName: "hello-world", compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, inv: binTB, fullMenu: true, hasResources: true, @@ -101,10 +112,27 @@ func TestCopyCatalog(t *testing.T) { {"tb # signed in ยท no secure environment on this machine", rndr(func(p *ui.Printer) { renderHome(p, noEnv) })}, {"tb # not signed in", rndr(func(p *ui.Printer) { renderHome(p, signedOut) })}, }, - "tracebloc --help", help(), + []run{{"tracebloc --help", help()}}, + ) + + // โ”€โ”€ 01 data ingest โ€” stage a dataset โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + ingestReview := &runDataIngestArgs{ + LocalPath: "./data", + Spec: push.SpecArgs{Table: "xray_train", Category: "image_classification", Intent: "train"}, + } + dataIngestFile := doc( + "tb data ingest โ€” stage a dataset into your secure environment", + "What you see when you run `tb data ingest `. The pre-flight review (below)\nis shown before you confirm. The live run then streams step lines (Checking โ†’\nCopying โ†’ Registering), a progress bar, and any validation error โ€” those aren't a\nstable screen, so every one of their strings is in zz-all-strings.golden.\n(`tb ingest` is a hidden deprecated alias of `tb data ingest`; `push` is a\ndeprecated alias of the verb.)", + []run{ + {"tb data ingest ./data --as train:xray_train --task image_classification # pre-flight review, before you confirm", rndr(func(p *ui.Printer) { renderReview(p, ingestReview) })}, + }, + []run{ + {"tracebloc data ingest --help", help("data", "ingest")}, + {"tracebloc data validate --help", help("data", "validate")}, + }, ) - // โ”€โ”€ data list โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + // โ”€โ”€ 02 data list โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ sample := []push.DatasetInfo{ {Name: "xray_train", Intent: "train", Task: "image_classification", Records: 12000, Classes: 2, Extension: "jpg", SizeBytes: 1 << 30}, {Name: "xray_test", Intent: "test", Task: "image_classification", Records: 3000, Classes: 2, Extension: "jpg", SizeBytes: 256 << 20}, @@ -118,15 +146,144 @@ func TestCopyCatalog(t *testing.T) { {"tb data list # with datasets (system tables hidden)", rndr(func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) })}, {"tb data list --all # including system tables", rndr(func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) })}, }, - "tracebloc data list --help", help("data", "list"), + []run{{"tracebloc data list --help", help("data", "list")}}, + ) + + // โ”€โ”€ 03 data delete โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + dataDeleteFile := doc( + "tb data delete โ€” delete a dataset", + "What you see when you run `tb data delete `. The command confirms before\nit deletes; the confirmation prompt, the progress, and the success/failure lines\nstream during the flow (not a stable screen) โ€” they're all in zz-all-strings.golden.", + nil, + []run{{"tracebloc data delete --help", help("data", "delete")}}, + ) + + // โ”€โ”€ 04 resources โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + resourcesFile := doc( + "tb resources โ€” see / change what a training run may use", + "What you see when you run `tb resources`. The view reads live cluster capacity,\nso it isn't a stable screen: it prints \"Your secure environment is equipped\nwith: โ€ฆ\", \"A training run is allocated up to: โ€ฆ\", and a hint to run\n`tb resources set` โ€” all indexed in zz-all-strings.golden. `tb resources set` is\na guided walkthrough (prompts stream during the flow; also in the backstop).", + nil, + []run{ + {"tracebloc resources --help", help("resources")}, + {"tracebloc resources set --help", help("resources", "set")}, + }, + ) + + // โ”€โ”€ 05 doctor โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + // doctorRollup mirrors runDoctor's rollup tail (doctor.go ~197-224) using the + // REAL summarizeDoctor + renderHealth + doctorVerdict, so this copy is + // drift-caught. Only launcher-free rollups are rendered here โ€” the failure + // lines ("Not connected โ€” โ€ฆ", "Not ready โ€” โ€ฆ") and their remedies embed the + // launcher name (tb vs tracebloc, install-dependent) and so are catalogued in + // zz-all-strings.golden instead. + doctorRollup := func(p *ui.Printer, email, envName string, results []doctor.Result, tok tokenState) { + p.Para("Signed in as " + email) + p.Para(fmt.Sprintf("Secure environment %q", envName)) + connected, ready := summarizeDoctor(results, tok) + p.Newline() + renderHealth(p, connected) + renderHealth(p, ready) + p.Newline() + fail, allGood := doctorVerdict(connected.status, ready.status) + switch { + case fail: + p.Hintf("Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`.", launcher()) + case allGood: + p.Successf("Everything looks good โ€” you're ready to run training.") + default: + p.Infof("No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail.") + } + } + // Connected + readiness unknown: Pod health warns with a list failure, which + // summarizeDoctor maps to an honest "couldn't check your workloads". + cantCheck := []doctor.Result{{Name: "Pod health", Status: doctor.StatusWarn, Detail: "could not list pods: forbidden"}} + doctorFile := doc( + "tb doctor โ€” is my secure environment healthy?", + "What you see when you run `tb doctor`. The two rollup lines (Connected, Ready)\nplus a verdict are shown below for the healthy and the can't-fully-check cases.\nThe failure variants (Not connected โ€” โ€ฆ, Not ready โ€” โ€ฆ) and their remedies vary\nwith the reachability classification and embed the launcher name, so the full set\nis indexed in zz-all-strings.golden. --verbose adds a Kubernetes breakdown\n(context/server/namespace + each granular check); those strings are in the\nbackstop too.", + []run{ + {"tb doctor # healthy", rndr(func(p *ui.Printer) { doctorRollup(p, "lukas@tracebloc.io", "hello-world", nil, tokenOK) })}, + {"tb doctor # connected, but a check couldn't complete (e.g. RBAC)", rndr(func(p *ui.Printer) { doctorRollup(p, "lukas@tracebloc.io", "hello-world", cantCheck, tokenOK) })}, + }, + []run{ + {"tracebloc doctor --help", help("doctor")}, + {"tracebloc cluster doctor --help", help("cluster", "doctor")}, + }, + ) + + // โ”€โ”€ 06 delete (offboard this machine) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + deleteFile := doc( + "tb delete โ€” remove tracebloc from this machine", + "What you see when you run `tb delete`. The pre-flight summary (below) is shown\nbefore you confirm, for both keep-data and remove-data. The confirmation prompt\nand the teardown progress stream during the flow โ€” those strings are in\nzz-all-strings.golden.", + []run{ + {"tb delete # summary ยท keep my data", rndr(func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) })}, + {"tb delete --remove-data # summary ยท remove my data too", rndr(func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) })}, + }, + []run{{"tracebloc delete --help", help("delete")}}, + ) + + // โ”€โ”€ 07 login / logout / auth โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + loginFile := doc( + "tb login / logout โ€” sign in and out", + "What you see when you run `tb login`. Sign-in is a device flow: the CLI prints an\n\"Open \" line and an \"Enter \" line, waits, then confirms โ€” that copy\nstreams during the flow (not a stable screen), so it's in zz-all-strings.golden.\n`tb auth status` reports who you're signed in as.", + nil, + []run{ + {"tracebloc login --help", help("login")}, + {"tracebloc logout --help", help("logout")}, + {"tracebloc auth status --help", help("auth", "status")}, + }, + ) + + // โ”€โ”€ 08 client โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + clientFile := doc( + "tb client โ€” register / list / inspect environments", + "What you see under `tb client`. `tb client create` shows a review (below) before\nit registers a new secure environment. `tb client list` / `tb client status` read\nlive backend state, so they aren't stable screens โ€” their strings are in\nzz-all-strings.golden.", + []run{ + {"tb client create # review, before you confirm", rndr(func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") })}, + }, + []run{ + {"tracebloc client --help", help("client")}, + {"tracebloc client create --help", help("client", "create")}, + {"tracebloc client list --help", help("client", "list")}, + {"tracebloc client status --help", help("client", "status")}, + }, + ) + + // โ”€โ”€ 09 cluster โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + clusterFile := doc( + "tb cluster โ€” low-level cluster info", + "What you see under `tb cluster`. `tb cluster info` reads live cluster state, so\nit isn't a stable screen; its strings are in zz-all-strings.golden. (`tb cluster\ndoctor` is the same health check as `tb doctor` โ€” see 05-doctor.)", + nil, + []run{ + {"tracebloc cluster --help", help("cluster")}, + {"tracebloc cluster info --help", help("cluster", "info")}, + }, + ) + + // โ”€โ”€ 10 version โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + versionFile := doc( + "tb version โ€” print the CLI version", + "What you see when you run `tb version`. It prints one line:\n\n tracebloc (, built , on /)\n\nThe go-version and os/arch are filled in at runtime, so the exact line varies by\nmachine (that's why it isn't pinned byte-exact here). `--output-json` emits the\nsame fields as indented JSON. Only the --help is byte-exact below.", + nil, + []run{{"tracebloc version --help", help("version")}}, ) files := map[string]string{ - "00-home.golden": homeFile, - "02-data-list.golden": dataListFile, - "zz-all-strings.golden": "every user-facing string in the source (AST-harvested โ€” all arguments, both\n" + - `"โ€ฆ" and ` + "`โ€ฆ`" + " raw strings). The completeness backstop: catches error paths and\nthe multi-step flows (ingest steps, login, progress) not shown as a screen.\n" + - "%s/%d are runtime placeholders.\n\n" + strings.Join(quoteAll(harvestMessages(t)), "\n") + "\n", + "00-home.golden": homeFile, + "01-data-ingest.golden": dataIngestFile, + "02-data-list.golden": dataListFile, + "03-data-delete.golden": dataDeleteFile, + "04-resources.golden": resourcesFile, + "05-doctor.golden": doctorFile, + "06-delete.golden": deleteFile, + "07-login.golden": loginFile, + "08-client.golden": clientFile, + "09-cluster.golden": clusterFile, + "10-version.golden": versionFile, + "zz-all-strings.golden": "every user-facing string in the source (AST-harvested โ€” all arguments to the\n" + + "Printer methods + errors.New/fmt.Errorf/fmt.Sprintf, plus the text/remedy\n" + + "fields of healthLine{} and doctor.Result{} literals, both \"โ€ฆ\" and `โ€ฆ` raw\n" + + "strings). The completeness backstop: catches the failure remedies and the\n" + + "multi-step flows (ingest steps, login, progress, confirmations) not shown as a\n" + + "screen. %s/%d are runtime placeholders.\n\n" + strings.Join(quoteAll(harvestMessages(t)), "\n") + "\n", } update := os.Getenv("TB_UPDATE_GOLDEN") != "" @@ -163,9 +320,12 @@ func quoteAll(in []string) []string { } // harvestMessages parses the user-facing packages and returns every string -// literal passed to a Printer method or an error constructor โ€” ALL arguments -// (Step labels, MenuRow descriptions, Field values included), both "โ€ฆ" and `โ€ฆ` -// raw strings. Deduped + sorted. +// literal that reaches a user: ALL arguments to a Printer method or an error / +// format constructor (errors.New, fmt.Errorf, fmt.Sprintf), PLUS the string +// fields of healthLine{} and doctor.Result{} composite literals โ€” those carry +// user-facing text (the doctor rollup lines, check details + remedies) that is +// never passed to a Printer call, so an arguments-only harvest would miss it. +// Both "โ€ฆ" and `โ€ฆ` raw strings; deduped + sorted. func harvestMessages(t *testing.T) []string { t.Helper() methods := map[string]bool{ @@ -183,11 +343,43 @@ func harvestMessages(t *testing.T) []string { return true } if x, ok := sel.X.(*ast.Ident); ok { - return (x.Name == "errors" && sel.Sel.Name == "New") || (x.Name == "fmt" && sel.Sel.Name == "Errorf") + return (x.Name == "errors" && sel.Sel.Name == "New") || + (x.Name == "fmt" && (sel.Sel.Name == "Errorf" || sel.Sel.Name == "Sprintf")) } return false } + // healthLine{} (this package) and doctor.Result{} (this package as + // doctor.Result, the doctor package as a bare Result) hold user-facing text + // in struct fields, not call arguments. + isCopyStruct := func(t ast.Expr) bool { + switch tt := t.(type) { + case *ast.Ident: + return tt.Name == "healthLine" || tt.Name == "Result" + case *ast.SelectorExpr: + return tt.Sel.Name == "Result" + } + return false + } + seen := map[string]struct{}{} + collect := func(exprs []ast.Expr) { + for _, arg := range exprs { + lit, ok := arg.(*ast.BasicLit) + if !ok || lit.Kind != token.STRING { + continue + } + s, uerr := strconv.Unquote(lit.Value) + if uerr != nil { + continue + } + s = strings.TrimSpace(s) + // Skip empties and format-only fragments (e.g. "%s", " ") โ€” no words. + if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { + continue + } + seen[s] = struct{}{} + } + } fset := token.NewFileSet() for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { @@ -199,24 +391,21 @@ func harvestMessages(t *testing.T) []string { return nil } ast.Inspect(f, func(n ast.Node) bool { - call, ok := n.(*ast.CallExpr) - if !ok || !isCopyCall(call) { - return true - } - for _, arg := range call.Args { - lit, ok := arg.(*ast.BasicLit) - if !ok || lit.Kind != token.STRING { - continue - } - s, uerr := strconv.Unquote(lit.Value) - if uerr != nil { - continue + switch node := n.(type) { + case *ast.CallExpr: + if isCopyCall(node) { + collect(node.Args) } - s = strings.TrimSpace(s) - if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { - continue + case *ast.CompositeLit: + if isCopyStruct(node.Type) { + for _, el := range node.Elts { + if kv, ok := el.(*ast.KeyValueExpr); ok { + collect([]ast.Expr{kv.Value}) + } else { + collect([]ast.Expr{el}) + } + } } - seen[s] = struct{}{} } return true }) diff --git a/internal/cli/screens_golden_test.go b/internal/cli/screens_golden_test.go deleted file mode 100644 index 9bfd2db2..00000000 --- a/internal/cli/screens_golden_test.go +++ /dev/null @@ -1,223 +0,0 @@ -package cli - -import ( - "bytes" - "go/ast" - "go/parser" - "go/token" - "os" - "path/filepath" - "sort" - "strconv" - "strings" - "testing" - - "github.com/spf13/cobra" - - "github.com/tracebloc/cli/internal/push" - "github.com/tracebloc/cli/internal/ui" -) - -// TestScreensGolden pins EVERY piece of user-facing copy in one committed file, -// testdata/screens.golden, so wording AND exact layout (line breaks, tabs, -// blank lines) can be reviewed without deploying โ€” read the file, or the diff on -// any PR that changes copy. A verbatim terminal transcript in three parts: -// -// A. Commands โ€” the `--help` of every command, byte-exact (real --help path) -// B. Screens โ€” the stateful views rendered verbatim (home, data list, review, โ€ฆ) -// C. Messages โ€” every user-facing STRING in the source, harvested via AST (so -// error paths + live flows โ€” ingest steps, login, progress โ€” -// that aren't a single rendered screen are still all here). -// -// The test fails on drift; regenerate after an intentional copy change: -// -// TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden -func TestScreensGolden(t *testing.T) { - const goldenPath = "testdata/screens.golden" - bi := BuildInfo{Version: "1.4.4", GitSHA: "0000000", BuildDate: "2026-01-01"} - var cat strings.Builder - - cat.WriteString("tracebloc CLI โ€” complete copy catalog\n") - cat.WriteString("A verbatim transcript: each `$ command` is followed by its byte-exact output\n") - cat.WriteString("(line breaks, tabs, blank lines โ€” all as the terminal prints them). The final\n") - cat.WriteString("section indexes every user-facing string, incl. multi-step flows not shown as\n") - cat.WriteString("a single screen. Regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden\n") - - block := func(cmd, output string) { - cat.WriteString("$ " + cmd + "\n") - cat.WriteString(output) - } - - // โ”€โ”€ PART A: every command's --help, through the real flag path โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= COMMANDS โ€” every `--help`, byte-exact\n" + strings.Repeat("=", 78) + "\n") - var paths [][]string - var walk func(c *cobra.Command, prefix []string) - walk = func(c *cobra.Command, prefix []string) { - paths = append(paths, prefix) - subs := append([]*cobra.Command(nil), c.Commands()...) - sort.Slice(subs, func(i, j int) bool { return subs[i].Name() < subs[j].Name() }) - for _, s := range subs { - if s.Name() == "help" || s.Name() == "completion" { - continue - } - walk(s, append(append([]string{}, prefix...), s.Name())) - } - } - walk(NewRootCmd(bi), nil) - for _, p := range paths { - var b bytes.Buffer - r := NewRootCmd(bi) - r.SetOut(&b) - r.SetErr(&b) - r.SetArgs(append(append([]string{}, p...), "--help")) - _ = r.Execute() - block(strings.TrimSpace("tracebloc "+strings.Join(p, " "))+" --help", b.String()) - } - - // โ”€โ”€ PART B: screens, rendered verbatim (same code the binary runs) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= SCREENS โ€” byte-exact renderer output\n" + strings.Repeat("=", 78) + "\n") - render := func(cmd string, f func(*ui.Printer)) { - var b bytes.Buffer - f(ui.New(&b, ui.WithColor(false))) - block(cmd, b.String()) - } - - online := homeModel{ - state: homeOnline, email: "lukas@tracebloc.io", name: "Lukas", envName: "hello-world", - compute: computeInfo{CPU: 12, MemGiB: 23}, hasCompute: true, inv: binTB, fullMenu: true, hasResources: true, - } - noComp := online - noComp.state, noComp.hasCompute, noComp.compute = homeRunning, false, computeInfo{} - notOnline := noComp - notOnline.confirmedNotOnline = true - starting := noComp - starting.state = homeStarting - offline := noComp - offline.state = homeOffline - noEnv := noComp - noEnv.state, noEnv.fullMenu, noEnv.envName = homeNoEnv, false, "" - signedOut := homeModel{state: homeNotSignedIn, inv: binTB} - - render("tb # home ยท Online", func(p *ui.Printer) { renderHome(p, online) }) - render("tb # home ยท running (couldn't confirm)", func(p *ui.Printer) { renderHome(p, noComp) }) - render("tb # home ยท running (backend not online)", func(p *ui.Printer) { renderHome(p, notOnline) }) - render("tb # home ยท starting up", func(p *ui.Printer) { renderHome(p, starting) }) - render("tb # home ยท offline", func(p *ui.Printer) { renderHome(p, offline) }) - render("tb # home ยท no secure environment", func(p *ui.Printer) { renderHome(p, noEnv) }) - render("tb # home ยท not signed in", func(p *ui.Printer) { renderHome(p, signedOut) }) - - sample := []push.DatasetInfo{ - {Name: "xray_train", Intent: "train", Task: "image_classification", Records: 12000, Classes: 2, Extension: "jpg", SizeBytes: 1 << 30}, - {Name: "xray_test", Intent: "test", Task: "image_classification", Records: 3000, Classes: 2, Extension: "jpg", SizeBytes: 256 << 20}, - {Name: "ingest_run_journal", System: true}, - } - render("tb data list # empty", func(p *ui.Printer) { renderDataList(p, "hello-world", nil, false) }) - render("tb data list # populated", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, false) }) - render("tb data list --all", func(p *ui.Printer) { renderDataList(p, "hello-world", sample, true) }) - - ingestReview := &runDataIngestArgs{ - LocalPath: "./data", - Spec: push.SpecArgs{Table: "xray_train", Category: "image_classification", Intent: "train"}, - } - render("tb data ingest ./data # pre-flight review", func(p *ui.Printer) { renderReview(p, ingestReview) }) - render("tb client create # review", func(p *ui.Printer) { renderClientReview(p, "lukas-macbook", "lukas-macbook", "DE", "a1b2c3d4") }) - render("tb delete # keep data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", true) }) - render("tb delete # remove data", func(p *ui.Printer) { renderOffboardSummary(p, "lukas-macbook", false) }) - - // โ”€โ”€ PART C: every user-facing string (AST harvest โ€” catches all args) โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - cat.WriteString("\n\n" + strings.Repeat("=", 78) + "\n= MESSAGE INDEX โ€” every user-facing string in the source (templates, not\n= rendered; %s/%d are runtime placeholders). Catches the multi-step flows the\n= transcript above can't show whole: ingest steps + progress, login device flow,\n= delete confirmation, and every error/hint.\n" + strings.Repeat("=", 78) + "\n\n") - for _, m := range harvestMessages(t) { - cat.WriteString(" " + strconv.Quote(m) + "\n") - } - - got := cat.String() - if os.Getenv("TB_UPDATE_GOLDEN") != "" { - if err := os.MkdirAll("testdata", 0o755); err != nil { - t.Fatal(err) - } - if err := os.WriteFile(goldenPath, []byte(got), 0o644); err != nil { - t.Fatal(err) - } - t.Logf("wrote %s (%d bytes)", goldenPath, len(got)) - return - } - want, err := os.ReadFile(goldenPath) - if err != nil { - t.Fatalf("read %s (regenerate: TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden): %v", goldenPath, err) - } - if got != string(want) { - t.Errorf("copy catalog drifted from %s.\nRegenerate + review the diff:\n TB_UPDATE_GOLDEN=1 go test ./internal/cli/ -run TestScreensGolden", goldenPath) - } -} - -// harvestMessages parses the user-facing packages and returns every string -// literal passed to a Printer method or an error constructor โ€” ALL arguments -// (so Step labels, MenuRow descriptions, Field values are included), both "โ€ฆ" and -// `โ€ฆ` raw strings. Deduped + sorted. A complete index of user-facing copy, -// independent of whether a screen renders it. -func harvestMessages(t *testing.T) []string { - t.Helper() - methods := map[string]bool{ - "Successf": true, "Warnf": true, "Errorf": true, "Infof": true, "Hintf": true, - "Detailf": true, "Para": true, "Section": true, "PromptHint": true, "PromptHeader": true, - "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, - "Stat": true, "Field": true, "MenuRow": true, "Banner": true, "Command": true, - } - isCopyCall := func(call *ast.CallExpr) bool { - sel, ok := call.Fun.(*ast.SelectorExpr) - if !ok { - return false - } - if methods[sel.Sel.Name] { - return true - } - if x, ok := sel.X.(*ast.Ident); ok { - return (x.Name == "errors" && sel.Sel.Name == "New") || (x.Name == "fmt" && sel.Sel.Name == "Errorf") - } - return false - } - - seen := map[string]struct{}{} - fset := token.NewFileSet() - for _, dir := range []string{".", "../submit", "../push", "../doctor", "../cluster"} { - _ = filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error { - if err != nil || d.IsDir() || !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") { - return nil - } - f, perr := parser.ParseFile(fset, path, nil, 0) - if perr != nil { - return nil - } - ast.Inspect(f, func(n ast.Node) bool { - call, ok := n.(*ast.CallExpr) - if !ok || !isCopyCall(call) { - return true - } - for _, arg := range call.Args { - lit, ok := arg.(*ast.BasicLit) - if !ok || lit.Kind != token.STRING { - continue - } - s, uerr := strconv.Unquote(lit.Value) - if uerr != nil { - continue - } - s = strings.TrimSpace(s) - // Skip empties and format-only fragments (e.g. "%s", " ") โ€” no words. - if len([]rune(s)) < 4 || !strings.ContainsAny(s, "abcdefghijklmnopqrstuvwxyz") { - continue - } - seen[s] = struct{}{} - } - return true - }) - return nil - }) - } - out := make([]string, 0, len(seen)) - for s := range seen { - out = append(out, s) - } - sort.Strings(out) - return out -} diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden new file mode 100644 index 00000000..e9ba3c1c --- /dev/null +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -0,0 +1,143 @@ +tb data ingest โ€” stage a dataset into your secure environment +============================================================= +What you see when you run `tb data ingest `. The pre-flight review (below) +is shown before you confirm. The live run then streams step lines (Checking โ†’ +Copying โ†’ Registering), a progress bar, and any validation error โ€” those aren't a +stable screen, so every one of their strings is in zz-all-strings.golden. +(`tb ingest` is a hidden deprecated alias of `tb data ingest`; `push` is a +deprecated alias of the verb.) + +$ tb data ingest ./data --as train:xray_train --task image_classification # pre-flight review, before you confirm + + Review + name: xray_train + task: image_classification + intent: train + path: ./data + resolution: auto-detect + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc data ingest --help +Ingests a local dataset into your secure environment's storage, +submits the ingestion run, and follows it to completion (streaming +progress + the final summary). Your data never leaves your own +infrastructure. Supports 16 tasks across the image, text, and +tabular / time-series families; pick one with --task. + + is the data itself. What it looks like depends on the task: + + tabular / time-series โ€” the dataset is a single CSV. Pass the .csv + file directly, or a folder holding exactly one .csv: + + churn.csv (the .csv file itself) + or + churn/ + data.csv (the one .csv in the folder) + + image (classification, object/keypoint detection) โ€” a folder with + labels.csv + an images/ subfolder: + + cats_dogs/ + labels.csv (required) + images/ (required) + 001.jpg + ... + + text (classification, masked language modeling) โ€” a folder with + labels.csv + a texts/ subfolder (masked language modeling uses sequences/): + + reviews/ + labels.csv (required) + texts/ (required โ€” sequences/ for masked language modeling) + 001.txt + ... + +A bare .csv file is accepted only for the tabular / time-series family; +image and text datasets must be a folder. + +Accepted image extensions: .jpg, .jpeg, or .png (case-insensitive). +All images in one dataset must share a single type โ€” the cluster +validates the type it was told to expect. + +v0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger +datasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€” +see tracebloc/client#147 non-goals. + +Exit codes: + 0 files staged + ingested successfully (or --detach: just staged + submitted) + 2 schema validation failed (synthesized spec rejected) or + v0.1-unsupported task passed + 3 local-layout or kubeconfig error + 4 cluster reachable but no tracebloc client / shared storage missing + 5 ingestor SA token couldn't be obtained, or jobs-manager + rejected the token (401/403) + 6 destination table already exists (re-run with --overwrite to + replace it, or pick a different --name) + 7 pre-flight succeeded but staging the files failed + (Pod creation, image pull, exec stream, or remote tar error) โ€” + or, with --overwrite, removing the old table failed + 8 jobs-manager rejected the submit (4xx/5xx other than auth) + 9 ingestion Job exited non-zero, or completed with row-level + failures the summary panel reports + +Usage: + tracebloc data ingest [flags] + +Aliases: + ingest, push + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --detach kubectl logs -f -n job/ exit immediately after jobs-manager accepts the run (no log streaming, no summary panel). Use for CI scenarios; reconnect later with kubectl logs -f -n job/. + --dry-run validate + discover + walk, but don't create any cluster resources + -h, --help help for ingest + --idempotency-key string reuse this idempotency key across retry attempts (default: fresh per invocation). jobs-manager treats a duplicate key as a replay and attaches to the existing Job rather than spawning a new one โ€” useful for at-most-once-across-attempts semantics. + --image-digest images.ingestor.digest pin the ingestor container image to a specific digest (default: jobs-manager picks the cluster-configured images.ingestor.digest). Format: sha256:. + --intent string is this training or test data? train|test (default train) + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + --label-column string name of the label/target column (in labels.csv for image tasks, in the data CSV for tabular) + --label-policy string regression-class only (tabular_regression, time_series_forecasting, time_to_event_prediction): passthrough|bucket (default bucket โ€” bins the target so the raw value never leaves the cluster) + --min-size string image tasks only: reject images smaller than WxH before the ingest (e.g. 64x64). Set it to the smallest size your model can train on โ€” raise or lower it freely. Default: unset (no local size check). + --name string a name for this dataset โ€” start with a letter or underscore, then letters/digits/underscores โ€” you'll reference it by this name when you start a training run + -n, --namespace string namespace where your tracebloc client is installed + --no-input disable interactive prompts; fail on missing required values (for CI/scripts) + --number-of-keypoints int keypoint_detection only: number of keypoints per sample (required; e.g. 17 for COCO pose) + --output-json emit a machine-readable JSON result on stdout (human output โ†’ stderr; implies --no-input) + --overwrite tracebloc data delete replace the destination table if it already exists: its current table + files are removed first (same as tracebloc data delete), then the new data is ingested. Not combinable with --idempotency-key + --schema string tabular/time-series only: column types as col:TYPE,col:TYPE (e.g. age:INT,price:FLOAT). Default: inferred from the CSV (INT/BIGINT/FLOAT/BOOLEAN/DATE/DATETIME/VARCHAR(n)). + --stage-pod-image string override the ephemeral stage Pod's image (default: digest-pinned alpine 3.20 baked into the CLI). Pin by digest in your override too โ€” tag-only refs drift silently. + --target-size string image tasks only: the resolution your images already are, as WxH (e.g. 512x512). tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Default: read from your first image. + --task string the task this data is for, one of: image_classification, object_detection, keypoint_detection, text_classification, masked_language_modeling, tabular_classification, tabular_regression, time_series_forecasting, time_series_classification, time_to_event_prediction, causal_language_modeling, seq2seq, token_classification, sentence_pair_classification, embeddings, semantic_segmentation. Omit it on a terminal to pick interactively. + --time-column string time_to_event_prediction only: name of the time/duration column (default: a column named "time") + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc data validate --help +Reads , parses it as YAML, and validates it against the bundled +ingest.v1.json schema (synced from tracebloc/data-ingestors). Prints +violations in the same JSON-pointer-prefixed format the cluster's +jobs-manager uses, and exits non-zero if any are found. + +Useful as a pre-flight before running `tracebloc data ingest` โ€” +millisecond local feedback instead of a multi-second cluster round +trip. + +Exit codes: + 0 YAML parses and validates cleanly + 2 YAML parses but has schema violations (printed to stderr) + 3 YAML doesn't parse or file isn't readable + +Usage: + tracebloc data validate [flags] + +Flags: + -h, --help help for validate + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/03-data-delete.golden b/internal/cli/testdata/golden/03-data-delete.golden new file mode 100644 index 00000000..a47f8b1e --- /dev/null +++ b/internal/cli/testdata/golden/03-data-delete.golden @@ -0,0 +1,50 @@ +tb data delete โ€” delete a dataset +================================= +What you see when you run `tb data delete `. The command confirms before +it deletes; the confirmation prompt, the progress, and the success/failure lines +stream during the flow (not a stable screen) โ€” they're all in zz-all-strings.golden. + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc data delete --help +Removes the in-cluster artifacts a previous `data ingest` created +for a table: the MySQL table in training_test_datasets and the dataset's +directories on the shared PVC. Destructive and not undoable. + +The dataset's catalog metadata on the tracebloc backend is never removed โ€” it +is kept as a record on tracebloc, marked unavailable, so a collaborator's run +that referenced it still has its history. + +Exit codes: + 0 artifacts removed (or --dry-run, or the user declined) + 2 invalid table name + 3 kubeconfig error, or refused (no confirmation off a terminal) + 4 cluster reachable but no tracebloc client / shared storage missing, + or the client's dataset list couldn't be read (can't confirm the target) + 5 no dataset by that name on this client (nothing to delete) + 7 teardown failed mid-flight (table drop or PVC rm errored) + +With --output-json, stdout carries exactly one JSON result object per run +(human output goes to stderr) and the exit codes above are unchanged; see +docs/json-output.md for the shape and the stability promise. + +Usage: + tracebloc data delete
[flags] + +Aliases: + delete, rm + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --dry-run show what would be deleted without deleting anything + -h, --help help for delete + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed + --output-json emit the delete result as JSON on stdout (human output โ†’ stderr; never prompts โ€” pass --yes to delete, or --dry-run) + -y, --yes skip the confirmation prompt (required when not on a terminal) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/04-resources.golden b/internal/cli/testdata/golden/04-resources.golden new file mode 100644 index 00000000..7c804d22 --- /dev/null +++ b/internal/cli/testdata/golden/04-resources.golden @@ -0,0 +1,90 @@ +tb resources โ€” see / change what a training run may use +======================================================= +What you see when you run `tb resources`. The view reads live cluster capacity, +so it isn't a stable screen: it prints "Your secure environment is equipped +with: โ€ฆ", "A training run is allocated up to: โ€ฆ", and a hint to run +`tb resources set` โ€” all indexed in zz-all-strings.golden. `tb resources set` is +a guided walkthrough (prompts stream during the flow; also in the backstop). + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc resources --help +Shows, in plain terms, how much of this machine tracebloc may use: + + โ€ข Your secure environment โ€” the CPU and memory it can schedule + โ€ข Each training run โ€” the per-run ceiling every run may use (cluster-wide) + +No Kubernetes concepts, no YAML โ€” one number for your environment and one for +each training run's share of it. + +Raise the share with `tracebloc resources set`. Run with --verbose for the +per-node breakdown and the raw values. + +Exit codes: + 0 shown + 3 kubeconfig could not be loaded / cluster unreachable + 4 cluster reachable but no tracebloc client found here + +Usage: + tracebloc resources [flags] + tracebloc resources [command] + +Available Commands: + set Raise how much of this machine tracebloc may use + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for resources + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc resources [command] --help" for more information about a command. + +$ tracebloc resources set --help +Raise the per-training-run ceiling โ€” how much of this machine a single +training run may use. + +Run it on a terminal with no flags for a guided walkthrough: + + tracebloc resources set + +Or set it directly (for scripts / non-interactive shells): + + tracebloc resources set --cores 4 --memory 16Gi an explicit per-run ceiling + tracebloc resources set --cores 4 change CPU only, keep the rest + tracebloc resources set max let a run use the whole machine + +The number you set is what ONE training run may use. tracebloc keeps a small fixed +amount (about 1 core and 3 GiB) for itself on top โ€” you never have to subtract it. +The new ceiling applies to your NEXT training run; a run already going keeps its +size. + +Exit codes: + 0 applied (or nothing to change) + 2 the requested size doesn't fit this machine / bad input + 3 kubeconfig could not be loaded / cluster unreachable + 4 cluster reachable but no tracebloc client found here + +Usage: + tracebloc resources set [max] [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --cores string CPU cores one training run may use (e.g. 4) + --dry-run show exactly what would change and apply nothing + --gpus int whole GPUs one training run may use (only on a GPU machine) + -h, --help help for set + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + --memory string memory one training run may use (e.g. 16 or 16Gi โ€” the number is GiB) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + --yes skip the confirmation prompt (for automation) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/05-doctor.golden b/internal/cli/testdata/golden/05-doctor.golden new file mode 100644 index 00000000..51d6d648 --- /dev/null +++ b/internal/cli/testdata/golden/05-doctor.golden @@ -0,0 +1,85 @@ +tb doctor โ€” is my secure environment healthy? +============================================= +What you see when you run `tb doctor`. The two rollup lines (Connected, Ready) +plus a verdict are shown below for the healthy and the can't-fully-check cases. +The failure variants (Not connected โ€” โ€ฆ, Not ready โ€” โ€ฆ) and their remedies vary +with the reachability classification and embed the launcher name, so the full set +is indexed in zz-all-strings.golden. --verbose adds a Kubernetes breakdown +(context/server/namespace + each granular check); those strings are in the +backstop too. + +$ tb doctor # healthy + Signed in as lukas@tracebloc.io + Secure environment "hello-world" + + โœ” Connected to tracebloc + โœ” Ready to run training + + โœ” Everything looks good โ€” you're ready to run training. + +$ tb doctor # connected, but a check couldn't complete (e.g. RBAC) + Signed in as lukas@tracebloc.io + Secure environment "hello-world" + + โœ” Connected to tracebloc + ยท Ready to run training โ€” couldn't check your workloads (run with --verbose) + + ยท No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail. + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc doctor --help +Checks, in plain terms, whether your secure environment is connected to +tracebloc and ready to run training โ€” and if something's wrong, exactly what to +do about it. + + --verbose the full technical breakdown (for support) + --diagnose write a redacted support bundle to email to tracebloc + +Exit codes: + 0 healthy + 2 a problem was found + 3 couldn't read your local config + +Usage: + tracebloc doctor [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --diagnose write a redacted support bundle for tracebloc support and exit + -h, --help help for doctor + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your secure environment is installed (default: your active client's) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc cluster doctor --help +Checks, in plain terms, whether your secure environment is connected to +tracebloc and ready to run training โ€” and if something's wrong, exactly what to +do about it. + + --verbose the full technical breakdown (for support) + --diagnose write a redacted support bundle to email to tracebloc + +Exit codes: + 0 healthy + 2 a problem was found + 3 couldn't read your local config + +Usage: + tracebloc cluster doctor [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + --diagnose write a redacted support bundle for tracebloc support and exit + -h, --help help for doctor + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your secure environment is installed (default: your active client's) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/06-delete.golden b/internal/cli/testdata/golden/06-delete.golden new file mode 100644 index 00000000..ed65049b --- /dev/null +++ b/internal/cli/testdata/golden/06-delete.golden @@ -0,0 +1,69 @@ +tb delete โ€” remove tracebloc from this machine +============================================== +What you see when you run `tb delete`. The pre-flight summary (below) is shown +before you confirm, for both keep-data and remove-data. The confirmation prompt +and the teardown progress stream during the flow โ€” those strings are in +zz-all-strings.golden. + +$ tb delete # summary ยท keep my data + + This will remove + ยท This machine's credential โ€” so tracebloc can no longer reach it + ยท Your secure environment "lukas-macbook" and everything it runs on this machine + ยท tracebloc's downloaded images + ยท The tracebloc CLI (your local data & config are kept โ€” --keep-data) + + Kept on tracebloc + ยท Your use cases and the models trained here + ยท Your dataset records (marked unavailable, not deleted) + + Left alone + ยท Docker and related tools โ€” remove them yourself if you no longer need them + +$ tb delete --remove-data # summary ยท remove my data too + + This will remove + ยท This machine's credential โ€” so tracebloc can no longer reach it + ยท Your secure environment "lukas-macbook" and everything it runs on this machine + ยท tracebloc's downloaded images + ยท Your local data & config (~/.tracebloc) and the tracebloc CLI โ€” can't be undone + + Kept on tracebloc + ยท Your use cases and the models trained here + ยท Your dataset records (marked unavailable, not deleted) + + Left alone + ยท Docker and related tools โ€” remove them yourself if you no longer need them + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc delete --help +Removes tracebloc from this machine: revokes the machine credential, +uninstalls the Helm release, deletes the local cluster, reclaims the tracebloc +container images, and clears local state โ€” then removes the CLI itself. + +Your use cases, datasets' catalog entries, and the models trained here are KEPT +on tracebloc as a record (a colleague's model must not vanish because you +reclaimed this box). System software the installer laid down โ€” Docker, kubectl, +k3d, helm, NVIDIA drivers โ€” is left in place; remove it yourself if unused. + +Destructive: on a single-host install the on-prem datasets live on this machine +and are erased. Not undoable. + +Usage: + tracebloc delete [flags] + +Flags: + --context string kubeconfig context for the target cluster (default: current-context) + --force offboard even if tracebloc still reports this client online + -h, --help help for delete + --keep-data uninstall the software but keep ~/.tracebloc (local config + on-host datasets) + --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace of this machine's tracebloc release (default: the active client's namespace) + --yes skip the typed-name confirmation (for automation) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/07-login.golden b/internal/cli/testdata/golden/07-login.golden new file mode 100644 index 00000000..d08e9359 --- /dev/null +++ b/internal/cli/testdata/golden/07-login.golden @@ -0,0 +1,58 @@ +tb login / logout โ€” sign in and out +=================================== +What you see when you run `tb login`. Sign-in is a device flow: the CLI prints an +"Open " line and an "Enter " line, waits, then confirms โ€” that copy +streams during the flow (not a stable screen), so it's in zz-all-strings.golden. +`tb auth status` reports who you're signed in as. + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc login --help +Sign in to tracebloc. The CLI prints a URL + short code; open the URL +on any device (your laptop or phone), sign in the way you already do +(password, Google, or GitHub), and approve the code. The CLI stores a +user token in ~/.tracebloc (mode 0600). + +Works on a headless / SSH box โ€” the browser and the CLI need not share a +machine. Honors HTTP(S)_PROXY / NO_PROXY for corporate-proxy networks. + +Usage: + tracebloc login [flags] + +Flags: + --env string backend environment: dev|stg|prod (default: $CLIENT_ENV, then prod) + -h, --help help for login + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc logout --help +Sign out (revoke the token server-side and clear it locally) + +Usage: + tracebloc logout [flags] + +Flags: + -h, --help help for logout + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc auth status --help +Show whether you're signed in, and to which backend + +Usage: + tracebloc auth status [flags] + +Flags: + --check exit 0 only if signed in with a backend-valid token, else 1; silent unless --verbose + --env string backend environment the check targets: dev|stg|prod (default: $CLIENT_ENV, then prod) + -h, --help help for status + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/08-client.golden b/internal/cli/testdata/golden/08-client.golden new file mode 100644 index 00000000..6ac292bf --- /dev/null +++ b/internal/cli/testdata/golden/08-client.golden @@ -0,0 +1,91 @@ +tb client โ€” register / list / inspect environments +================================================== +What you see under `tb client`. `tb client create` shows a review (below) before +it registers a new secure environment. `tb client list` / `tb client status` read +live backend state, so they aren't stable screens โ€” their strings are in +zz-all-strings.golden. + +$ tb client create # review, before you confirm + + Review + name: lukas-macbook + namespace: lukas-macbook + location: DE + cluster: a1b2c3d4 (anchors this client โ€” re-runs adopt it) + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc client --help +Provision a tracebloc client for this machine. Requires sign-in first +(`tracebloc login`). To remove tracebloc from this machine, use +`tracebloc delete`. + +Usage: + tracebloc client [flags] + tracebloc client [command] + +Available Commands: + status Show whether tracebloc can see this machine's client (online) + +Flags: + -h, --help help for client + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc client [command] --help" for more information about a command. + +$ tracebloc client create --help +Provision a tracebloc client for this machine (auto-named; no flags required) + +Usage: + tracebloc client create [flags] + +Flags: + --context string kubeconfig context for the target cluster (default: current-context) + --credential-file string write the machine credential to this path (mode 0600, sourceable env) instead of printing it โ€” for the installer to feed the chart (never shown on the terminal) + -h, --help help for create + --kubeconfig string path to the kubeconfig for the target cluster (default: $KUBECONFIG, then ~/.kube/config) โ€” read to anchor the client to this cluster + --location string optional location zone for carbon reporting, e.g. DE (default: $TRACEBLOC_CLIENT_LOCATION; omitted if unset) + --name string client name (default: $TRACEBLOC_CLIENT_NAME, else auto-generated -NN; shown on your dashboard + carbon reports) + --yes skip the confirmation prompt + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc client list --help +List the clients in your account + +Usage: + tracebloc client list [flags] + +Aliases: + list, ls + +Flags: + -h, --help help for list + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +$ tracebloc client status --help +Report tracebloc's view of this machine's active client โ€” online, offline, +or pending. With --wait, poll until tracebloc reports it online (exit 0) or the +timeout elapses (non-zero), to confirm the client connected after setup. + +Usage: + tracebloc client status [flags] + +Flags: + -h, --help help for status + --timeout duration with --wait, give up after this long (default 2m0s) + --wait poll until tracebloc reports this client online + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/09-cluster.golden b/internal/cli/testdata/golden/09-cluster.golden new file mode 100644 index 00000000..b42e3068 --- /dev/null +++ b/internal/cli/testdata/golden/09-cluster.golden @@ -0,0 +1,71 @@ +tb cluster โ€” low-level cluster info +=================================== +What you see under `tb cluster`. `tb cluster info` reads live cluster state, so +it isn't a stable screen; its strings are in zz-all-strings.golden. (`tb cluster +doctor` is the same health check as `tb doctor` โ€” see 05-doctor.) + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc cluster --help +Commands for inspecting the Kubernetes cluster the CLI is +configured to talk to. + +Use `cluster info` to verify which cluster, namespace, and +client the next `data ingest` will target. Useful as a +pre-flight before doing anything destructive (e.g. ingesting into +the wrong cluster). + +Usage: + tracebloc cluster [flags] + tracebloc cluster [command] + +Available Commands: + info Show the cluster, namespace, client install, and ingestor token state + +Flags: + -h, --help help for cluster + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) + +Use "tracebloc cluster [command] --help" for more information about a command. + +$ tracebloc cluster info --help +Discovers the tracebloc client installed in the configured +cluster + namespace and prints: + + โ€ข Which kubeconfig context the CLI used + โ€ข The namespace it resolved to + โ€ข The client's release name + chart version + appVersion + โ€ข The jobs-manager Service the next data ingest would POST to + โ€ข The ingestor ServiceAccount the post-install hook would auth as + โ€ข The cluster's configured INGESTOR_IMAGE_DIGEST default + โ€ข Whether the user's kubeconfig can mint short-lived SA tokens + via TokenRequest, or has to fall back to a static + service-account-token Secret + +The actual token bytes are never printed; the diagnostic shows +SHA256(token)[:8] so the customer can verify "yes, that's the +token I expect" without leaking it to terminal scrollback. + +Exit codes: + 0 cluster discovered + token mintable; CLI is ready + 4 cluster reachable but no tracebloc client found + 5 cluster reachable + release found but no usable SA token + +Usage: + tracebloc cluster info [flags] + +Flags: + --context string name of the kubeconfig context to use (default: kubeconfig's current-context) + -h, --help help for info + --kubeconfig string path to the kubeconfig file (default: $KUBECONFIG, then ~/.kube/config) + -n, --namespace string namespace where your tracebloc client is installed (default: the context's namespace, or 'default') + --token-expiry-seconds int requested SA token expiration in seconds (default 600 = 10 min; ignored for static-secret fallback) (default 600) + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/10-version.golden b/internal/cli/testdata/golden/10-version.golden new file mode 100644 index 00000000..a01dc573 --- /dev/null +++ b/internal/cli/testdata/golden/10-version.golden @@ -0,0 +1,27 @@ +tb version โ€” print the CLI version +================================== +What you see when you run `tb version`. It prints one line: + + tracebloc (, built , on /) + +The go-version and os/arch are filled in at runtime, so the exact line varies by +machine (that's why it isn't pinned byte-exact here). `--output-json` emits the +same fields as indented JSON. Only the --help is byte-exact below. + + +------------------------------------------------------------ +--help +------------------------------------------------------------ +$ tracebloc version --help +Print the tracebloc CLI version, git SHA, and build date + +Usage: + tracebloc version [flags] + +Flags: + -h, --help help for version + --output-json emit the version payload as indented JSON instead of a single human-readable line + +Global Flags: + --plain disable color and decorative output (also honors $NO_COLOR) + --verbose stream detailed step-by-step progress (also via $TRACEBLOC_LOG_LEVEL=debug) diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index 59ea49ce..be54c90c 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -1,32 +1,79 @@ -every user-facing string in the source (AST-harvested โ€” all arguments, both -"โ€ฆ" and `โ€ฆ` raw strings). The completeness backstop: catches error paths and -the multi-step flows (ingest steps, login, progress) not shown as a screen. -%s/%d are runtime placeholders. +every user-facing string in the source (AST-harvested โ€” all arguments to the +Printer methods + errors.New/fmt.Errorf/fmt.Sprintf, plus the text/remedy +fields of healthLine{} and doctor.Result{} literals, both "โ€ฆ" and `โ€ฆ` raw +strings). The completeness backstop: catches the failure remedies and the +multi-step flows (ingest steps, login, progress, confirmations) not shown as a +screen. %s/%d are runtime placeholders. "\"active\" is this machine's selected client; state is its last reported status to tracebloc." +"%.1f%%" +"%.2f GiB" +"%.2f KiB" +"%.2f MiB" +"%d %s" +"%d B" +"%d annotation(s) without an image (%s)" +"%d files" +"%d files (%s)" +"%d image pull secret(s) present and well-formed" +"%d image(s) without a mask (%s)" +"%d image(s) without an annotation (%s)" +"%d mask(s) not named _mask.png (%s)" +"%d mask(s) without an image (%s)" +"%d of %d" +"%d pod(s), none crash-looping or stuck Pending" +"%d pod(s), none restarted โ‰ฅ%d times" "%d system table(s) hidden โ€” show with --all." +"%dd ago" +"%dh ago" +"%dm ago" "%q exists but is not a directory" "%q is a directory, not a file" "%q is not a directory; pass the directory containing labels.csv + images/" "%q is not a directory; pass the directory containing labels.csv + the text files" +"%s state=%s namespace=%s location=%s" "%s %q must be WxH (e.g. 512x512)" "%s %q: height is not an integer: %w" "%s %q: width and height must both be positive" "%s %q: width is not an integer: %w" "%s %s โ€” %s" +"%s (%dx%d)" +"%s (unreadable: %v)" +"%s Bound, mounted at %s" "%s has a header but no data rows (0 ingestable records). Add at least one data row and re-run." "%s is empty โ€” add a header and at least one data row, then re-run" "%s is empty โ€” no header row" "%s is image tasks only; it doesn't apply to task %q" +"%s of %s GiB" +"%s of %s cores" "%s requires CLIENT_WRITE permission" +"%s unreachable: %v" +"%s ยท %d" "%s ยท Online%s" "%s ยท can't reach it from here โ€” run %s" "%s ยท running โ€” couldn't confirm it's connected to tracebloc โ€” run %s" "%s ยท running, but tracebloc hasn't heard from it โ€” run %s" "%s ยท starting up, not ready yet โ€” run %s" +"%s ร—%d" "%s โ€” %s ยท %s" "%s โ€” %s ยท %s (%s)" +"%s, โ€ฆ and %d more" +"%s/%s" "%s: %w" +"%s=%s,%s=%s" +"%v (policy: %v)" +"(%d CPU ยท %d GiB" +"(+%d more)" +"(Pod phase: %s)" +"(couldn't check whether %q already exists โ€” continuing; the cluster still refuses duplicates: %v)" +"(datasets on this client: %s)" +"(last container state: %s โ€” %s)" +"(remote tar stderr: %s)" +"(scheduling: %s โ€” %s)" +"(table: %s)" +", %s=%s" +"- %s, age %s%s" +"-%02d" "--label-column doesn't apply to task %q โ€” it trains on the text itself, with no label column" "--number-of-keypoints is keypoint_detection only; it doesn't apply to task %q" "--overwrite can't be combined with --idempotency-key: a reused key makes the cluster replay the previous run instead of ingesting the new data โ€” after --overwrite's removal that would report success while loading nothing. Drop one of the two (a fresh per-run key is the default)." @@ -34,15 +81,20 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "--schema is tabular/time-series tasks only; it doesn't apply to task %q" "--time-column is time_to_event_prediction only; it doesn't apply to task %q" "--timeout has no effect without --wait" +"0:%d" +"3 GiB" +"A dataset named %q already exists โ€” replace it?" "A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train" "A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load)." "A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it." "A training run is allocated up to:" "Add --help to any command for the flags." +"Add/resize a node to meet the job's requests, or lower RESOURCE_REQUESTS on jobs-manager." "Already signed out." "Applies to your next training run; a run already going keeps its size." "Ask one of these admins (or ask them to grant you access)" "Available now:" +"CPU cores for one run (1โ€“%d)" "CSV %s has no columns" "Can't reach tracebloc from here." "Cancelled โ€” %q was left as-is; nothing was ingested." @@ -59,8 +111,10 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Client status" "Clients in your account" "Cluster teardown reported: %v" +"Connected to tracebloc" "Connecting to your secure environmentโ€ฆ" "Copy into your secure environment" +"Copying %s" "Correlation id: %s" "Couldn't check for active training runs (%v) โ€” continuing; the confirmation below still guards you." "Couldn't connect to your secure environment โ€” check your kubeconfig/context." @@ -77,6 +131,10 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Couldn't write the support bundle: %v" "Credential written to %s (mode 0600, not shown). This machine is set to enroll as client %d." "DB failures" +"DROP TABLE IF EXISTS `%s`.`%s`" +"Datasets in %s (0)" +"Datasets in %s โ€” %d" +"Delete %q and its files?" "Deleted %s.%s and %d PVC path(s)." "Destructive and cannot be undone." "Detached โ€” the ingestion runs in the background on your secure environment." @@ -91,13 +149,19 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Each training run already uses up to %s โ€” nothing to change." "Each training run may now use up to %s." "Email it to support@tracebloc.io." +"Email support@tracebloc.io with the output of `%s doctor --diagnose`." +"Ensure your kubeconfig user can list nodes." "Enter" "Everything looks good โ€” you're ready to run training." "Follow it later with: kubectl logs -f -n %s job/%s" +"Found labels.csv and a %s folder โ€” this looks like text data." +"Free some up, or raise the machine's allocation in Docker Desktop โ†’ Resources." "Full log: %s" "GPU access removed โ€” training runs will use CPU only." +"How many GPUs for one run (1โ€“%d)" "How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" "How much of this machine a training run may use" +"If GPU training is expected, ensure one node has both the compute and the GPU capacity, with its device plugin." "Ingest settings" "Ingestion complete โ€” %s" "Ingestion complete โ€” showing its logs:" @@ -108,48 +172,76 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Ingestion started โ€” streaming logs:" "Ingestion summary" "Ingestor SA token" +"Ingests a local dataset into your secure environment's storage,\nsubmits the ingestion run, and follows it to completion (streaming\nprogress + the final summary). Your data never leaves your own\ninfrastructure. Supports %[1]d tasks across the image, text, and\ntabular / time-series families; pick one with --task.\n\n is the data itself. What it looks like depends on the task:\n\n tabular / time-series โ€” the dataset is a single CSV. Pass the .csv\n file directly, or a folder holding exactly one .csv:\n\n churn.csv (the .csv file itself)\n or\n churn/\n data.csv (the one .csv in the folder)\n\n image (classification, object/keypoint detection) โ€” a folder with\n labels.csv + an images/ subfolder:\n\n cats_dogs/\n labels.csv (required)\n images/ (required)\n 001.jpg\n ...\n\n text (classification, masked language modeling) โ€” a folder with\n labels.csv + a %[2]s/ subfolder (masked language modeling uses %[3]s/):\n\n reviews/\n labels.csv (required)\n %[2]s/ (required โ€” %[3]s/ for masked language modeling)\n 001.txt\n ...\n\nA bare .csv file is accepted only for the tabular / time-series family;\nimage and text datasets must be a folder.\n\nAccepted image extensions: .jpg, .jpeg, or .png (case-insensitive).\nAll images in one dataset must share a single type โ€” the cluster\nvalidates the type it was told to expect.\n\nv0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger\ndatasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€”\nsee tracebloc/client#147 non-goals.\n\nExit codes:\n 0 files staged + ingested successfully (or --detach: just staged + submitted)\n 2 schema validation failed (synthesized spec rejected) or\n v0.1-unsupported task passed\n 3 local-layout or kubeconfig error\n 4 cluster reachable but no tracebloc client / shared storage missing\n 5 ingestor SA token couldn't be obtained, or jobs-manager\n rejected the token (401/403)\n 6 destination table already exists (re-run with --overwrite to\n replace it, or pick a different --name)\n 7 pre-flight succeeded but staging the files failed\n (Pod creation, image pull, exec stream, or remote tar error) โ€”\n or, with --overwrite, removing the old table failed\n 8 jobs-manager rejected the submit (4xx/5xx other than auth)\n 9 ingestion Job exited non-zero, or completed with row-level\n failures the summary panel reports" "Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." "Kept on tracebloc" "Kubeconfig" "Learn more: https://docs.tracebloc.io" "Left %s in place โ€” it isn't tracebloc's `tb` alias." "Left alone" +"Let each training run use up to %s?" "Let's set up your data ingest" "Local dataset" "Machine credential โ€” needed by the installer to connect this client" "Memory" +"Memory for one run in GiB (2โ€“%d)" "No client in namespace %q โ€” using the one in %q (override with --namespace)." "No clients yet. Run `tracebloc client create`." +"No datasets yet โ€” ingest one with `%s data ingest`." "No new credential issued; the existing one stands. This machine is set to enroll as client %d." "No problems found, but some checks couldn't finish โ€” re-run with --verbose for detail." "No secure environment on this machine yet โ€” run the installer to set one up." "No secure environment on this machine yet." +"Not connected โ€” can't reach tracebloc from here." +"Not connected โ€” couldn't read your secure environment." +"Not connected โ€” tracebloc didn't confirm your session (server error)." +"Not connected โ€” your secure environment isn't answering." +"Not ready โ€” dataset storage isn't available." +"Not ready โ€” not enough free compute to start a training." +"Not ready โ€” part of your secure environment can't start yet." +"Not ready โ€” part of your secure environment isn't running." +"Not ready โ€” the training images can't be pulled." "Not signed in yet." "Not signed in โ€” run `%s login`." "Not signed in. Run `tracebloc login`." "Not yet in the CLI:" +"Note: %d file(s) in images/ have no labels.csv row and won't be part of the dataset: %s" "Offboarded %q. This machine is no longer connected to tracebloc." "Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." "Open" "Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR" "POST %s%s: %w" "PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter" +"Pending > %s: %v" "Pick this dataset when you set it up." "Press Enter to accept a default; Ctrl-C to cancel." +"Private image pulls will ImagePullBackOff. Reinstall the chart with valid registry credentials." "Provisioned client %q (namespace %s)." "Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent):" "Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away." "Ready for `tracebloc data ingest`." +"Ready to run training" +"Ready to run training โ€” can't check yet" +"Ready to run training โ€” couldn't check free compute (run with --verbose)" +"Ready to run training โ€” couldn't check your workloads (run with --verbose)" "Reclaimed tracebloc's downloaded images." +"Recreate it as a docker-registry secret (kubectl create secret docker-registry)." +"Recreate the registry secret; its .dockerconfigjson isn't valid JSON." "Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values." +"Reinstall with `%s`, or email support@tracebloc.io with `%s doctor --diagnose`." "Removed local tracebloc data and config." "Removed stray control characters from the name." "Removed the local environment." "Removed the old %q โ€” ingesting the new data." "Removed the tracebloc CLI from this machine." "Removing in-cluster artifactsโ€ฆ" +"Removing the existing %q first" "Review" "Revoked this machine's credential โ€” your secure environment %q stays on tracebloc as a record." +"Run '%s --help' for the available commands." +"SELECT '%s',%s,COUNT(*),%s,%s FROM `%s`.`%s`" +"SELECT table_name FROM information_schema.tables WHERE table_schema='%s' ORDER BY table_name" +"Secure environment %q" "Set one up: %s" "Sign in to tracebloc" "Signed in" @@ -160,6 +252,7 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Signed out locally, but couldn't revoke the token server-side (%v). Revoke from the dashboard if this was a shared machine." "Signed out." "Signed-in token was rejected by the backend โ€” run `tracebloc login`." +"Some pods are stuck starting โ€” usually not enough free compute, or a training image that can't be pulled. Free some up in Docker Desktop โ†’ Resources, then re-run `%s doctor`; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." "Some tracebloc images couldn't be reclaimed (harmless) โ€” remove them later with `docker rmi $(docker images --filter=reference='ghcr.io/tracebloc/*' --format '{{.Repository}}:{{.Tag}}')`." "Still stuck? Email support@tracebloc.io with the output of `%s doctor --diagnose`." "Stopped following after 1 hour โ€” the ingestion is still running and will finish on its own." @@ -167,9 +260,11 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Submitted โ€” tracebloc is validating your data and loading it into the table." "Submitting the run โ€” with --detach it keeps running on your secure environment after this command returns; the reconnect command is shown below." "Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below." +"System ยท %d" "Table %q already exists โ€” replacing it (table + files)." "Target" "Target cluster" +"Tasks for %s data" "The column holding the duration / time-to-event. e.g. time, tenure_days" "The column in your CSV with the answer the model learns to produce." "The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed." @@ -188,10 +283,15 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account." "This will remove" "To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases" +"Training results can't reach tracebloc โ€” experiments will stall." "Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." +"Type %q to offboard this machine" "Uninstalled tracebloc." +"VARCHAR(%d)" "Validate and load" +"Verify the requests-proxy is wired: kubectl set env deploy/-jobs-manager --list | grep PROXY" "We couldn't tell the data type from what's there โ€” which is it?" +"Welcome to your secure environment for AI, %s ๐Ÿ‘‹" "What's next" "Whether this split trains the model or evaluates it." "Will delete" @@ -207,9 +307,11 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "Your secure environment is equipped with:" "Your session expired โ€” run `%s login`." "Your use cases and the models trained here" +"a Ready node can schedule a training job (%s)" "a dataset path is required" "a tracebloc client is already running on this cluster, but listing your account to verify ownership failed (%w) โ€” re-run once tracebloc is reachable, or resolve manually" "a tracebloc client is installed in namespace %q but its CLIENT_ID could not be read" +"a training run needs at least %s โ€” %s is too little." "account" "active client" "annotations" @@ -236,16 +338,23 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "connected: %s โ€” %s" "constructing kubernetes clientset: %w" "context" +"could not check โ€” cluster API unreachable (see 'Cluster reachable' above)" +"couldn't read RESOURCE_REQUESTS from jobs-manager โ€” skipping node-fit" "couldn't read capacity: %v" +"couldn't read jobs-manager to resolve image pull secrets โ€” skipping" "couldn't read this machine's capacity: %w" +"cpu=%s, memory=%s" +"crash-looping: %v" "creating SPDY executor for %s/%s: %w" "creating credential-file directory: %w" "creating port-forwarder: %w" "creating stage Pod in namespace %q: %w" "creating staging-cleanup pod: %w" "creating teardown pod: %w" +"csv ยท %d cols" "dashboard id" "data CSV" +"data row %d" "dataset exceeded v0.1 total cap of %s after streaming %s (reached %s)" "dataset name is required (set --name)" "dataset name is required โ€” pass it as an argument: tracebloc data delete " @@ -255,7 +364,9 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "destination" "dropping %s.%s: %w%s" "enter a whole number between %d and %d" +"exactly %d" "exec stream against %s/%s: %w" +"exit %d" "expires" "expires in" "field %d is empty โ€” every field (%s) must be non-empty" @@ -264,9 +375,13 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "generating Pod-name random suffix: %w" "generating idempotency key: %w" "generating staging-dir suffix: %w" +"http://%s.%s.svc.cluster.local:%d" +"http://localhost:%d" +"image pull secret %q not found" "images" "infer from CSV" "inferring schema from CSV: %w" +"ingested %s of %s records (%.1f%%)" "ingestion Job completed but the summary reports failures โ€” see panel above" "ingestion Job exited non-zero โ€” see logs above" "ingestor ID" @@ -278,9 +393,12 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "internal: re-parsing synthesized spec: %w\n%s" "invalid table name %q: %w" "jobs-manager" +"jobs-manager %s returned HTTP %d: %s" +"jobs-manager has no literal REQUESTS_PROXY_URL (chart too old, or it's set via a configMap/secret ref)" "jobs-manager: %s" "keypoints" "kube-system namespace has no UID" +"kubectl set env deploy/-jobs-manager --list | grep RESOURCE_REQUESTS" "label column" "label policy" "labels.csv" @@ -303,17 +421,22 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "missing %s/ subdirectory in %q" "must be a positive integer" "must be between %d and %d" +"mysql -uroot -p\"$MYSQL_ROOT_PASSWORD\" -N -e \"%s\"" +"mysql -uroot -p\"$MYSQL_ROOT_PASSWORD\" -e '%s'" "mysql table" "name" "namespace" "never (static-secret fallback)" "no CLI-supported tasks for %s data yet" +"no Ready node can fit a training job (needs %s)" "no Ready node on this machine to size a training run against" "no Running pod with name containing %q in namespace %q" "no active client on this machine โ€” nothing to offboard" "no active client on this machine โ€” run `tracebloc client create` (or re-run the installer) first" "no dataset named %q on this client%s" "no image files to detect a type from" +"no image pull secret in use (public/digest-pinned images)" +"no single Ready node satisfies cpu+memory AND %s โ€” GPU jobs rely on the CPU fallback (needs %s)" "no such file or directory: %q โ€” check the path to your dataset" "no tracebloc client found" "none detected" @@ -323,8 +446,10 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "overwrite prompt: %w" "packaging %s: %w" "packaging labels.csv: %w" +"parsing embedded layout.v1.json: %v" "password" "path" +"pod %s container %s restarted %d times" "port-forward allocated zero ports" "port-forward to %s/%s failed during startup: %w" "pvc path" @@ -353,19 +478,27 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "refusing to offboard without confirmation: pass --yes, or run on a terminal to type the client name" "refusing to stream symbolic link %q (defense-in-depth; should have been rejected by Discover)" "release" +"release %q, chart %s, appVersion %s (namespace %s)" "release: %s (chart %s)" "removed โ€” runs will use CPU only" "removing PVC paths: %w%s" "removing staged copy %s: %w%s" +"requests-proxy deployment not found" +"requests-proxy is running, but egress to Service Bus is not actively verified โ€” readiness only confirms the relay started, not that it can reach Service Bus" +"requests-proxy not ready (%d/%d replicas)" +"requests-proxy relays training results/metrics to Service Bus; without it, running experiments can't send results back and training stalls mid-run (scheduling is unaffected). Reinstall/upgrade the client chart." "resolution" "resolving %q: %w" "resolving Service %s/%s to a Pod: %w" "resolving namespace from kubeconfig: %w" "resource env" +"restarted โ‰ฅ%d times โ€” check logs: %v" "root" "scanning the cluster for tracebloc clients: %w" "schema" "schema entry %q must be col:TYPE (e.g. age:INT,price:FLOAT)" +"secret %q has an empty or malformed %s" +"secret %q is type %q, not %s" "sent to API" "server" "service %s/%s has no selector โ€” can't resolve to a Pod for port-forwarding" @@ -400,7 +533,9 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "task %q isn't supported by the CLI yet%s. Supported tasks: %s." "teardown failed: %w" "the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster" +"the cluster API server at %s isn't answering โ€” is the cluster running?" "the sign-in code expired โ€” re-run `tracebloc login`" +"this machine has %s, but you asked for %s." "time column" "token saved to ~/.tracebloc (0600)" "total records" @@ -412,7 +547,10 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry" "tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete`" "tracebloc's downloaded images" +"tracebloc-doctor-%s.txt" +"tracebloc-stage-%s-%s" "unavailable" +"unknown command %q for %q" "values:" "waiting for ingestor Pod: %w" "waiting for staging-cleanup pod: %w" @@ -420,3 +558,6 @@ the multi-step flows (ingest steps, login, progress) not shown as a screen. "watching ingestor Job: %w" "would set each run to" "writing credential file %s: %w" +"~%s (requested; server may cap shorter)" +"ยท %d GPU" +"ยท %d classes" From c3d18af5fb12e2e589c845898c7eae9b65f5dcac Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 08:59:41 +0200 Subject: [PATCH 07/14] Catalog: show the guided ingest questionnaire (every prompt, driven) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 01-data-ingest now shows the full guided flow, not just the review. A catalog double for the prompter seam prints each question the way the terminal does ("? ") while the REAL runInteractive drives it, so the transcript is every prompt in order โ€” intro, the core questions, the family sniff/echo, the task picker, the task-specific questions, the review, and the confirm. Driven for two families (tabular + image) so the task-specific questions (schema; image resolution) are visible. The temp data dir is normalised to ~/datasets/hospital. Also harvests the prompter labels + help text (Input/Select/Confirm) into the backstop โ€” prompt copy that never passes through a Printer call and so was missing before (556 -> 589 strings; e.g. "which split this data is", "How many keypoints", "Override the column types", "bucket bins the target"). This covers the prompt copy for flows not driven as a screen (client create, delete). Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 98 ++++++++++++++++-- .../cli/testdata/golden/01-data-ingest.golden | 99 +++++++++++++++++-- .../cli/testdata/golden/zz-all-strings.golden | 33 +++++++ 3 files changed, 216 insertions(+), 14 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 4d1bf0f4..2c201166 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -115,16 +115,52 @@ func TestCopyCatalog(t *testing.T) { []run{{"tracebloc --help", help()}}, ) - // โ”€โ”€ 01 data ingest โ€” stage a dataset โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - ingestReview := &runDataIngestArgs{ - LocalPath: "./data", - Spec: push.SpecArgs{Table: "xray_train", Category: "image_classification", Intent: "train"}, + // โ”€โ”€ 01 data ingest โ€” stage a dataset (the guided questionnaire) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + // driveIngest runs the REAL guided flow (runInteractive) with a prompter that + // prints each question as the terminal shows it, so the transcript is every + // prompt in order: intro, the core questions, the family sniff/echo, the task + // picker, the task-specific questions, the review, and the confirm. The intro + // preamble mirrors data_ingest_local.go (its text is drift-guarded by the + // backstop); the temp data dir is normalised to a stable placeholder. + driveIngest := func(dir string, answers map[string]string) string { + var b bytes.Buffer + p := ui.New(&b, ui.WithColor(false)) + p.Newline() + p.Para("This ingests a dataset so models can train on it. Your files never leave your\n" + + "own infrastructure โ€” tracebloc copies them into your secure environment's storage,\n" + + "checks them, and loads them into a table your training runs read from. Other\n" + + "collaborators can train against that table without ever seeing the raw files.") + p.Hintf("Learn more: https://docs.tracebloc.io") + pr := &catalogPrompter{w: &b, answers: answers} + a := &runDataIngestArgs{} + if err := runInteractive(p, pr, a, false /*taskSet*/); err != nil { + t.Fatalf("driveIngest(%s): %v", dir, err) + } + return strings.ReplaceAll(b.String(), dir, "~/datasets/hospital") } + tabDir := tabularDir(t) + imgDir := imageDirLayout(t) + tabularIngest := driveIngest(tabDir, map[string]string{ + "Is this training or test data?": "train", + "What should we call this dataset?": "hospital_train", + "Where is your data? (file or folder)": tabDir, + "Which task?": "Tabular classification", + "Which column holds the class?": "churned", + }) + imageIngest := driveIngest(imgDir, map[string]string{ + "Is this training or test data?": "train", + "What should we call this dataset?": "xray_train", + "Where is your data? (file or folder)": imgDir, + "Which task?": "Image classification", + "Which column holds the class?": "label", + "Image resolution as WxH (blank = read it from your first image)": "224x224", + }) dataIngestFile := doc( "tb data ingest โ€” stage a dataset into your secure environment", - "What you see when you run `tb data ingest `. The pre-flight review (below)\nis shown before you confirm. The live run then streams step lines (Checking โ†’\nCopying โ†’ Registering), a progress bar, and any validation error โ€” those aren't a\nstable screen, so every one of their strings is in zz-all-strings.golden.\n(`tb ingest` is a hidden deprecated alias of `tb data ingest`; `push` is a\ndeprecated alias of the verb.)", + "What you see when you run `tb data ingest` with no flags: a short intro, then a\nguided questionnaire. Every question is shown below, in order, driven through the\nreal flow for two tasks (tabular + image) so the task-specific questions are\nvisible. Each prompt shows `? `; the line above it is the\nquestion's one-line description. Passing flags (--as, --task, a path, โ€ฆ) skips\nthe matching questions. The remaining tasks' extra questions (keypoints, label\npolicy, time column) and every prompt's `?`-help text are in\nzz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a\ndeprecated alias of the verb.)", []run{ - {"tb data ingest ./data --as train:xray_train --task image_classification # pre-flight review, before you confirm", rndr(func(p *ui.Printer) { renderReview(p, ingestReview) })}, + {"tb data ingest # guided ยท tabular classification", tabularIngest}, + {"tb data ingest # guided ยท image classification", imageIngest}, }, []run{ {"tracebloc data ingest --help", help("data", "ingest")}, @@ -319,6 +355,53 @@ func quoteAll(in []string) []string { return out } +// catalogPrompter is the prompter seam's catalog double: it prints each question +// the way the terminal shows it ("? ") and returns a scripted +// answer, so driving the REAL runInteractive produces a byte-exact transcript of +// the guided flow โ€” every prompt, in order. The description line above each +// question is the real p.PromptHint in runInteractive; only the "? โ€ฆ" line is +// rendered here (survey draws it in production). +type catalogPrompter struct { + w *bytes.Buffer + answers map[string]string +} + +func (c *catalogPrompter) pick(label, def string) string { + if a, ok := c.answers[label]; ok { + return a + } + return def +} + +func (c *catalogPrompter) show(label, ans string) { + if ans == "" { + fmt.Fprintf(c.w, "? %s\n", label) + return + } + fmt.Fprintf(c.w, "? %s %s\n", label, ans) +} + +func (c *catalogPrompter) Input(label, _, def string, _ func(string) error) (string, error) { + ans := c.pick(label, def) + c.show(label, ans) + return ans, nil +} + +func (c *catalogPrompter) Select(label, _ string, _ []string, def string) (string, error) { + ans := c.pick(label, def) + c.show(label, ans) + return ans, nil +} + +func (c *catalogPrompter) Confirm(label string, def bool) (bool, error) { + ans := "No" + if def { + ans = "Yes" + } + c.show(label, ans) + return def, nil +} + // harvestMessages parses the user-facing packages and returns every string // literal that reaches a user: ALL arguments to a Printer method or an error / // format constructor (errors.New, fmt.Errorf, fmt.Sprintf), PLUS the string @@ -333,6 +416,9 @@ func harvestMessages(t *testing.T) []string { "Detailf": true, "Para": true, "Section": true, "PromptHint": true, "PromptHeader": true, "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, "Stat": true, "Field": true, "MenuRow": true, "Banner": true, "Command": true, + // prompter seam (survey) โ€” question labels + help text for every guided + // flow (ingest, client create, delete), incl. flows not driven as a screen. + "Input": true, "Select": true, "Confirm": true, } isCopyCall := func(call *ast.CallExpr) bool { sel, ok := call.Fun.(*ast.SelectorExpr) diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index e9ba3c1c..ae7d50a1 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -1,20 +1,103 @@ tb data ingest โ€” stage a dataset into your secure environment ============================================================= -What you see when you run `tb data ingest `. The pre-flight review (below) -is shown before you confirm. The live run then streams step lines (Checking โ†’ -Copying โ†’ Registering), a progress bar, and any validation error โ€” those aren't a -stable screen, so every one of their strings is in zz-all-strings.golden. -(`tb ingest` is a hidden deprecated alias of `tb data ingest`; `push` is a +What you see when you run `tb data ingest` with no flags: a short intro, then a +guided questionnaire. Every question is shown below, in order, driven through the +real flow for two tasks (tabular + image) so the task-specific questions are +visible. Each prompt shows `? `; the line above it is the +question's one-line description. Passing flags (--as, --task, a path, โ€ฆ) skips +the matching questions. The remaining tasks' extra questions (keypoints, label +policy, time column) and every prompt's `?`-help text are in +zz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a deprecated alias of the verb.) -$ tb data ingest ./data --as train:xray_train --task image_classification # pre-flight review, before you confirm +$ tb data ingest # guided ยท tabular classification + + This ingests a dataset so models can train on it. Your files never leave your + own infrastructure โ€” tracebloc copies them into your secure environment's storage, + checks them, and loads them into a table your training runs read from. Other + collaborators can train against that table without ever seeing the raw files. + Learn more: https://docs.tracebloc.io + + Let's set up your data ingest + Press Enter to accept a default; Ctrl-C to cancel. + + Whether this split trains the model or evaluates it. +? Is this training or test data? train + + A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train +? What should we call this dataset? hospital_train + + The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn +? Where is your data? (file or folder) ~/datasets/hospital + โœ” Found a CSV table โ€” this is tabular data. + + Tasks for tabular data + Available now: + ยท Tabular classification โ€” predict a class from table columns ยท tabular_classification + ยท Tabular regression โ€” predict a number from table columns ยท tabular_regression + ยท Time-series forecasting โ€” predict future values from past ones ยท time_series_forecasting + ยท Time-series classification โ€” predict a class for each whole sequence ยท time_series_classification + ยท Survival analysis โ€” predict how long until an event happens ยท time_to_event_prediction +? Which task? Tabular classification + + The column in your CSV with the answer the model learns to produce. +? Which column holds the class? churned + + Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR +? Column schema as col:TYPE,... (blank = infer from the CSV) + + Review + name: hospital_train + task: tabular_classification + intent: train + path: ~/datasets/hospital + label column: churned + schema: infer from CSV +? Proceed with the ingest? Yes + +$ tb data ingest # guided ยท image classification + + This ingests a dataset so models can train on it. Your files never leave your + own infrastructure โ€” tracebloc copies them into your secure environment's storage, + checks them, and loads them into a table your training runs read from. Other + collaborators can train against that table without ever seeing the raw files. + Learn more: https://docs.tracebloc.io + + Let's set up your data ingest + Press Enter to accept a default; Ctrl-C to cancel. + + Whether this split trains the model or evaluates it. +? Is this training or test data? train + + A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train +? What should we call this dataset? xray_train + + The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn +? Where is your data? (file or folder) ~/datasets/hospital + โœ” Found labels.csv and an images/ folder โ€” this is image data. + + Tasks for image data + Available now: + ยท Image classification โ€” sort images into classes ยท image_classification + ยท Object detection โ€” draw boxes around objects in an image ยท object_detection + ยท Keypoint detection โ€” locate landmark points on an image (e.g. pose) ยท keypoint_detection + ยท Semantic segmentation โ€” label every pixel in an image ยท semantic_segmentation +? Which task? Image classification + + The column in your CSV with the answer the model learns to produce. +? Which column holds the class? label + + The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224 +? Image resolution as WxH (blank = read it from your first image) 224x224 Review name: xray_train task: image_classification intent: train - path: ./data - resolution: auto-detect + path: ~/datasets/hospital + label column: label + resolution: 224x224 +? Proceed with the ingest? Yes ------------------------------------------------------------ diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index be54c90c..1878a0ed 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -111,6 +111,7 @@ screen. %s/%d are runtime placeholders. "Client status" "Clients in your account" "Cluster teardown reported: %v" +"Column schema as col:TYPE,... (blank = infer from the CSV)" "Connected to tracebloc" "Connecting to your secure environmentโ€ฆ" "Copy into your secure environment" @@ -160,8 +161,10 @@ screen. %s/%d are runtime placeholders. "GPU access removed โ€” training runs will use CPU only." "How many GPUs for one run (1โ€“%d)" "How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" +"How much may one training run use?" "How much of this machine a training run may use" "If GPU training is expected, ensure one node has both the compute and the GPU capacity, with its device plugin." +"Image resolution as WxH (blank = read it from your first image)" "Ingest settings" "Ingestion complete โ€” %s" "Ingestion complete โ€” showing its logs:" @@ -173,9 +176,11 @@ screen. %s/%d are runtime placeholders. "Ingestion summary" "Ingestor SA token" "Ingests a local dataset into your secure environment's storage,\nsubmits the ingestion run, and follows it to completion (streaming\nprogress + the final summary). Your data never leaves your own\ninfrastructure. Supports %[1]d tasks across the image, text, and\ntabular / time-series families; pick one with --task.\n\n is the data itself. What it looks like depends on the task:\n\n tabular / time-series โ€” the dataset is a single CSV. Pass the .csv\n file directly, or a folder holding exactly one .csv:\n\n churn.csv (the .csv file itself)\n or\n churn/\n data.csv (the one .csv in the folder)\n\n image (classification, object/keypoint detection) โ€” a folder with\n labels.csv + an images/ subfolder:\n\n cats_dogs/\n labels.csv (required)\n images/ (required)\n 001.jpg\n ...\n\n text (classification, masked language modeling) โ€” a folder with\n labels.csv + a %[2]s/ subfolder (masked language modeling uses %[3]s/):\n\n reviews/\n labels.csv (required)\n %[2]s/ (required โ€” %[3]s/ for masked language modeling)\n 001.txt\n ...\n\nA bare .csv file is accepted only for the tabular / time-series family;\nimage and text datasets must be a folder.\n\nAccepted image extensions: .jpg, .jpeg, or .png (case-insensitive).\nAll images in one dataset must share a single type โ€” the cluster\nvalidates the type it was told to expect.\n\nv0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger\ndatasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€”\nsee tracebloc/client#147 non-goals.\n\nExit codes:\n 0 files staged + ingested successfully (or --detach: just staged + submitted)\n 2 schema validation failed (synthesized spec rejected) or\n v0.1-unsupported task passed\n 3 local-layout or kubeconfig error\n 4 cluster reachable but no tracebloc client / shared storage missing\n 5 ingestor SA token couldn't be obtained, or jobs-manager\n rejected the token (401/403)\n 6 destination table already exists (re-run with --overwrite to\n replace it, or pick a different --name)\n 7 pre-flight succeeded but staging the files failed\n (Pod creation, image pull, exec stream, or remote tar error) โ€”\n or, with --overwrite, removing the old table failed\n 8 jobs-manager rejected the submit (4xx/5xx other than auth)\n 9 ingestion Job exited non-zero, or completed with row-level\n failures the summary panel reports" +"Is this training or test data?" "Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." "Kept on tracebloc" "Kubeconfig" +"Label policy" "Learn more: https://docs.tracebloc.io" "Left %s in place โ€” it isn't tracebloc's `tb` alias." "Left alone" @@ -185,6 +190,7 @@ screen. %s/%d are runtime placeholders. "Machine credential โ€” needed by the installer to connect this client" "Memory" "Memory for one run in GiB (2โ€“%d)" +"MySQL identifier + PVC subdir; start with a letter or underscore, then letters, digits, underscore" "No client in namespace %q โ€” using the one in %q (override with --namespace)." "No clients yet. Run `tracebloc client create`." "No datasets yet โ€” ingest one with `%s data ingest`." @@ -206,6 +212,7 @@ screen. %s/%d are runtime placeholders. "Not signed in. Run `tracebloc login`." "Not yet in the CLI:" "Note: %d file(s) in images/ have no labels.csv row and won't be part of the dataset: %s" +"Number of keypoints per sample" "Offboarded %q. This machine is no longer connected to tracebloc." "Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." "Open" @@ -216,6 +223,8 @@ screen. %s/%d are runtime placeholders. "Pick this dataset when you set it up." "Press Enter to accept a default; Ctrl-C to cancel." "Private image pulls will ImagePullBackOff. Reinstall the chart with valid registry credentials." +"Proceed with the ingest?" +"Provision this client?" "Provisioned client %q (namespace %s)." "Provisioning didn't complete. Re-running is safe โ€” on the same cluster it adopts the existing client instead of minting a duplicate (idempotent):" "Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away." @@ -282,18 +291,24 @@ screen. %s/%d are runtime placeholders. "This permanently removes a dataset you ingested earlier: it drops the table from\nthe cluster and deletes the dataset's files on the shared storage. It can't be\nundone โ€” re-ingesting the data is the only way back." "This secure environment isn't in the signed-in account โ€” continuing; if that's unexpected, check you're logged into the right account." "This will remove" +"Time column" "To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases" "Training results can't reach tracebloc โ€” experiments will stall." "Try again shortly; if it persists, email support@tracebloc.io with `%s doctor --diagnose`." "Type %q to offboard this machine" "Uninstalled tracebloc." +"Use the GPU for training runs?" "VARCHAR(%d)" "Validate and load" "Verify the requests-proxy is wired: kubectl set env deploy/-jobs-manager --list | grep PROXY" "We couldn't tell the data type from what's there โ€” which is it?" "Welcome to your secure environment for AI, %s ๐Ÿ‘‹" +"What kind of data is this?" +"What should we call this dataset?" "What's next" +"Where is your data? (file or folder)" "Whether this split trains the model or evaluates it." +"Which task?" "Will delete" "Wrote a support bundle to ./%s" "Wrote client id + namespace to %s (no new credential โ€” the existing one stands)." @@ -321,6 +336,8 @@ screen. %s/%d are runtime placeholders. "backend" "backend %s โ€” requesting a device code โ€ฆ" "backfilling the cluster anchor onto the existing client: %w" +"bucket" +"bucket bins the target before it leaves the cluster" "building SPDY transport: %w" "building rest config from kubeconfig: %w" "building submit request: %w" @@ -363,6 +380,9 @@ screen. %s/%d are runtime placeholders. "deleting stage Pod %s/%s: %w" "destination" "dropping %s.%s: %w%s" +"e.g. ./my-data" +"e.g. 17 for COCO pose" +"e.g. age:INT,price:FLOAT" "enter a whole number between %d and %d" "exactly %d" "exec stream against %s/%s: %w" @@ -375,6 +395,8 @@ screen. %s/%d are runtime placeholders. "generating Pod-name random suffix: %w" "generating idempotency key: %w" "generating staging-dir suffix: %w" +"how many CPU cores a single training run may use" +"how much memory a single training run may use, in GiB" "http://%s.%s.svc.cluster.local:%d" "http://localhost:%d" "image pull secret %q not found" @@ -449,6 +471,8 @@ screen. %s/%d are runtime placeholders. "parsing embedded layout.v1.json: %v" "password" "path" +"pick the label/target column from your CSV header" +"pick the task this data is for" "pod %s container %s restarted %d times" "port-forward allocated zero ports" "port-forward to %s/%s failed during startup: %w" @@ -528,14 +552,19 @@ screen. %s/%d are runtime placeholders. "submit response missing namespace (got body %q)" "success rate" "synthesized spec failed schema validation; check the flag values above" +"tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/" "task" "task %q isn't a recognized task. Supported tasks: %s." "task %q isn't supported by the CLI yet%s. Supported tasks: %s." "teardown failed: %w" "the client running in this namespace is anchored to a different cluster (%s) than --kubeconfig/--context points at (%s) โ€” check you're targeting the right cluster" "the cluster API server at %s isn't answering โ€” is the cluster running?" +"the duration/time column name" +"the label/target column name" "the sign-in code expired โ€” re-run `tracebloc login`" +"the size your images already are; tracebloc checks it, it never resizes" "this machine has %s, but you asked for %s." +"time" "time column" "token saved to ~/.tracebloc (0600)" "total records" @@ -544,11 +573,13 @@ screen. %s/%d are runtime placeholders. "tracebloc can see this client." "tracebloc didn't confirm your session (server error)." "tracebloc keeps about 1 core and 3 GiB for itself on top of this โ€” it fits on this machine." +"tracebloc keeps about 1 core and 3 GiB for itself on top of your choice" "tracebloc rejected your credentials while waiting โ€” run `tracebloc login`, then retry" "tracebloc rejected your credentials โ€” run `tracebloc login`, then retry `tracebloc delete`" "tracebloc's downloaded images" "tracebloc-doctor-%s.txt" "tracebloc-stage-%s-%s" +"train" "unavailable" "unknown command %q for %q" "values:" @@ -556,6 +587,8 @@ screen. %s/%d are runtime placeholders. "waiting for staging-cleanup pod: %w" "waiting for teardown pod: %w" "watching ingestor Job: %w" +"which split this data is" +"whole GPUs a single run may use" "would set each run to" "writing credential file %s: %w" "~%s (requested; server may cap shorter)" From be10b7abff20ef1a826fe83ae1eb8cda050ce8b3 Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 10:46:25 +0200 Subject: [PATCH 08/14] Ingest: redesign the guided flow as a clear 5-step questionnaire MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reworks `tb data ingest`'s interactive setup per the approved copy pass: - Intro trimmed to one line + the specific data-prep docs link; drops the "Let's set upโ€ฆ / Press Enterโ€ฆ" preamble. - Each question is now a dominant "Step N of 5 ยท " header with any supporting line beneath it and an answer-only prompt (surveyPrompter gains a `bare` mode; the CLI owns the question text via the new PromptStep). - Plainer wording: "Do you want to ingest training or test data?", "Please name the dataset.", "Where is your data?" (+ per-modality file/folder examples), "What kind of machine learning task is this data for?" (airy list, task IDs), "Which column holds the label?" (+ examples), and a schema step reframed to "Column types โ€” infer or override" instead of the cryptic blank prompt. - The task picker lists task IDs + blurbs (the blurb carries what the old gloss said); DisplayName/Gloss metadata is retained (allowlisted, still tested). - The invalid-table-name error drops the MySQL/PVC/path-traversal internals for a one-line "use letters, digits, underscoresโ€ฆ". Tests + the copy catalog (01-data-ingest.golden) updated to the new wording. Execution/summary redesign (clean 3 steps, single summary) follows separately. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 48 +++--- internal/cli/data_ingest_cmd.go | 4 +- internal/cli/data_ingest_local.go | 8 +- internal/cli/interactive.go | 137 ++++++++++++------ internal/cli/interactive_test.go | 120 +++++++-------- .../cli/testdata/golden/01-data-ingest.golden | 115 ++++++++------- .../cli/testdata/golden/zz-all-strings.golden | 43 +++--- internal/push/spec.go | 9 +- internal/ui/ui.go | 8 + scripts/deadcode-allowlist.txt | 9 ++ 10 files changed, 280 insertions(+), 221 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 2c201166..1a370d68 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -126,11 +126,8 @@ func TestCopyCatalog(t *testing.T) { var b bytes.Buffer p := ui.New(&b, ui.WithColor(false)) p.Newline() - p.Para("This ingests a dataset so models can train on it. Your files never leave your\n" + - "own infrastructure โ€” tracebloc copies them into your secure environment's storage,\n" + - "checks them, and loads them into a table your training runs read from. Other\n" + - "collaborators can train against that table without ever seeing the raw files.") - p.Hintf("Learn more: https://docs.tracebloc.io") + p.Para("Ingest a dataset โ€” your files never leave this machine.") + p.Hintf("Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset") pr := &catalogPrompter{w: &b, answers: answers} a := &runDataIngestArgs{} if err := runInteractive(p, pr, a, false /*taskSet*/); err != nil { @@ -141,23 +138,23 @@ func TestCopyCatalog(t *testing.T) { tabDir := tabularDir(t) imgDir := imageDirLayout(t) tabularIngest := driveIngest(tabDir, map[string]string{ - "Is this training or test data?": "train", - "What should we call this dataset?": "hospital_train", - "Where is your data? (file or folder)": tabDir, - "Which task?": "Tabular classification", - "Which column holds the class?": "churned", + "Do you want to ingest training or test data?": "train", + "Please name the dataset.": "hospital_train", + "Where is your data?": tabDir, + "Which task?": "tabular_classification", + "Which column holds the label?": "churned", }) imageIngest := driveIngest(imgDir, map[string]string{ - "Is this training or test data?": "train", - "What should we call this dataset?": "xray_train", - "Where is your data? (file or folder)": imgDir, - "Which task?": "Image classification", - "Which column holds the class?": "label", - "Image resolution as WxH (blank = read it from your first image)": "224x224", + "Do you want to ingest training or test data?": "train", + "Please name the dataset.": "xray_train", + "Where is your data?": imgDir, + "Which task?": "image_classification", + "Which column holds the label?": "label", + "Image resolution": "224x224", }) dataIngestFile := doc( "tb data ingest โ€” stage a dataset into your secure environment", - "What you see when you run `tb data ingest` with no flags: a short intro, then a\nguided questionnaire. Every question is shown below, in order, driven through the\nreal flow for two tasks (tabular + image) so the task-specific questions are\nvisible. Each prompt shows `? `; the line above it is the\nquestion's one-line description. Passing flags (--as, --task, a path, โ€ฆ) skips\nthe matching questions. The remaining tasks' extra questions (keypoints, label\npolicy, time column) and every prompt's `?`-help text are in\nzz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a\ndeprecated alias of the verb.)", + "What you see when you run `tb data ingest` with no flags: a short intro, then a\nfive-step guided setup. Every question is shown below, in order, driven through\nthe real flow for two tasks (tabular + image) so the task-specific questions are\nvisible. Each question prints as a `Step N of 5 ยท โ€ฆ` header (task-specific\nrefinements as their own header); the supporting line sits beneath it, and the\n`?` line shows your answer. Passing flags (--as, --task, a path, โ€ฆ) skips the\nmatching questions. The remaining tasks' extra questions (keypoints, label\npolicy, time column) and every prompt's `?`-help text are in\nzz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a\ndeprecated alias of the verb.)", []run{ {"tb data ingest # guided ยท tabular classification", tabularIngest}, {"tb data ingest # guided ยท image classification", imageIngest}, @@ -373,32 +370,37 @@ func (c *catalogPrompter) pick(label, def string) string { return def } -func (c *catalogPrompter) show(label, ans string) { +// answerLine renders the input line the way the bare surveyPrompter does for the +// guided flow: the question is already printed by the CLI (PromptStep/Section), +// so the prompt shows only "? " (or "?" on a blank/accept-default). +func (c *catalogPrompter) answerLine(ans string) { if ans == "" { - fmt.Fprintf(c.w, "? %s\n", label) + fmt.Fprintf(c.w, "?\n") return } - fmt.Fprintf(c.w, "? %s %s\n", label, ans) + fmt.Fprintf(c.w, "? %s\n", ans) } func (c *catalogPrompter) Input(label, _, def string, _ func(string) error) (string, error) { ans := c.pick(label, def) - c.show(label, ans) + c.answerLine(ans) return ans, nil } func (c *catalogPrompter) Select(label, _ string, _ []string, def string) (string, error) { ans := c.pick(label, def) - c.show(label, ans) + c.answerLine(ans) return ans, nil } +// Confirm keeps its label (a short y/n with no header of its own โ€” matches the +// non-bare surveyPrompter Confirm). func (c *catalogPrompter) Confirm(label string, def bool) (bool, error) { ans := "No" if def { ans = "Yes" } - c.show(label, ans) + fmt.Fprintf(c.w, "? %s %s\n", label, ans) return def, nil } diff --git a/internal/cli/data_ingest_cmd.go b/internal/cli/data_ingest_cmd.go index f4292afd..52e9d73d 100644 --- a/internal/cli/data_ingest_cmd.go +++ b/internal/cli/data_ingest_cmd.go @@ -203,7 +203,9 @@ Exit codes: interactive := !noInput && !outputJSON && isInteractiveTTY() var pr prompter if interactive { - pr = surveyPrompter{} + // bare: the guided flow prints each question as a step header + // (PromptStep), so the prompt line itself stays answer-only. + pr = surveyPrompter{bare: true} } // In --output-json mode, human output goes to stderr so // stdout carries only the JSON result. diff --git a/internal/cli/data_ingest_local.go b/internal/cli/data_ingest_local.go index e29f12cb..1a16ab85 100644 --- a/internal/cli/data_ingest_local.go +++ b/internal/cli/data_ingest_local.go @@ -93,12 +93,8 @@ func resolveLocalInput(out, errOut io.Writer, a *runDataIngestArgs) (layout *pus } a.Printer.Newline() - a.Printer.Para(strings.TrimSpace(` -This ingests a dataset so models can train on it. Your files never leave your -own infrastructure โ€” tracebloc copies them into your secure environment's storage, -checks them, and loads them into a table your training runs read from. Other -collaborators can train against that table without ever seeing the raw files.`)) - a.Printer.Hintf("Learn more: https://docs.tracebloc.io") + a.Printer.Para("Ingest a dataset โ€” your files never leave this machine.") + a.Printer.Hintf("Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset") // 0. Guided mode: prompt for any missing core inputs before // validation. Flags already provided win; non-TTY / --no-input diff --git a/internal/cli/interactive.go b/internal/cli/interactive.go index c52f23af..1c49523f 100644 --- a/internal/cli/interactive.go +++ b/internal/cli/interactive.go @@ -38,11 +38,24 @@ type prompter interface { // surveyPrompter is the production prompter, backed by // AlecAivazis/survey/v2 against the real terminal. -type surveyPrompter struct{} +// +// bare drops the question text from the prompt line (survey's Message), for +// flows where the CLI already prints the question itself as a step header +// (the guided ingest flow, via PromptStep) โ€” so the prompt reads "? " +// with no duplicate question. Confirm always keeps its label (a short y/n with +// no header of its own). +type surveyPrompter struct{ bare bool } + +func (s surveyPrompter) message(label string) string { + if s.bare { + return "" + } + return label +} -func (surveyPrompter) Input(label, help, def string, validate func(string) error) (string, error) { +func (s surveyPrompter) Input(label, help, def string, validate func(string) error) (string, error) { var ans string - q := &survey.Input{Message: label, Help: help, Default: def} + q := &survey.Input{Message: s.message(label), Help: help, Default: def} var opts []survey.AskOpt if validate != nil { // survey hands the validator the raw answer as interface{}; @@ -58,9 +71,9 @@ func (surveyPrompter) Input(label, help, def string, validate func(string) error return ans, nil } -func (surveyPrompter) Select(label, help string, options []string, def string) (string, error) { +func (s surveyPrompter) Select(label, help string, options []string, def string) (string, error) { var ans string - q := &survey.Select{Message: label, Help: help, Options: options, Default: def} + q := &survey.Select{Message: s.message(label), Help: help, Options: options, Default: def} if err := survey.AskOne(q, &ans); err != nil { return "", mapErr(err) } @@ -104,14 +117,19 @@ func isInteractiveTTY() bool { // // Mutates a through the pointer. func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bool) error { - p.PromptHeader("Let's set up your data ingest") - p.Hintf("Press Enter to accept a default; Ctrl-C to cancel.") prompted := false - // (a) intent โ€” the first thing to settle: what this data is for. + // The guided flow is a five-step setup: intent โ†’ name โ†’ path โ†’ task โ†’ + // task-specific details. Each question prints as its own step header + // (PromptStep), with any supporting line beneath it and an answer-only + // prompt (the prompter runs bare โ€” see surveyPrompter). Task-specific + // extras beyond the label (schema, resolution, โ€ฆ) are refinements under + // step 5 and aren't separately numbered. + + // Step 1 โ€” intent: what this data is for. if a.Spec.Intent == "" { - p.PromptHint("Whether this split trains the model or evaluates it.") - ans, err := pr.Select("Is this training or test data?", "which split this data is", + p.PromptStep(1, 5, "Do you want to ingest training or test data?") + ans, err := pr.Select("Do you want to ingest training or test data?", "which split this data is", []string{"train", "test"}, "train") if err != nil { return err @@ -120,12 +138,12 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo prompted = true } - // (b) name โ€” no auto-fill: the example lives in the hint, so the user - // types their own name rather than editing a pre-filled default. + // Step 2 โ€” name. No auto-fill; the character rules surface only if the + // name is rejected (see ValidateTableName), so the prompt stays clean. if a.Spec.Table == "" { - p.PromptHint("A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train") - ans, err := pr.Input("What should we call this dataset?", - "MySQL identifier + PVC subdir; start with a letter or underscore, then letters, digits, underscore", "", + p.PromptStep(2, 5, "Please name the dataset.") + ans, err := pr.Input("Please name the dataset.", + "letters, digits, and underscores; start with a letter or underscore e.g. churn_train", "", push.ValidateTableName) if err != nil { return err @@ -134,10 +152,15 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo prompted = true } - // (c) path โ€” then detect the family from the layout and echo it back. + // Step 3 โ€” path. Show what "file or folder" means per modality, then + // detect the family from the layout and echo it back. if a.LocalPath == "" { - p.PromptHint("The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn") - ans, err := pr.Input("Where is your data? (file or folder)", "e.g. ./my-data", "", validateDatasetPath) + p.PromptStep(3, 5, "Where is your data?") + p.Hintf("Give the path to a file or a folder โ€” whichever holds your data:") + p.Infof("Tabular one CSV file e.g. ~/data/patients.csv") + p.Infof("Images a folder with labels.csv + images/ e.g. ~/data/xray/") + p.Infof("Text a folder with labels.csv + texts/ e.g. ~/data/reviews/") + ans, err := pr.Input("Where is your data?", "e.g. ~/data/patients.csv or ~/data/xray/", "", validateDatasetPath) if err != nil { return err } @@ -219,7 +242,8 @@ func resolveFamily(p *ui.Printer, pr prompter, path string) (push.Family, error) // user what looks off so they can fix the layout. p.Warnf("%s", s.Hint) } - p.PromptHint("We couldn't tell the data type from what's there โ€” which is it?") + p.Section("What kind of data is this?") + p.Hintf("We couldn't tell from the layout โ€” tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/.") opts := push.FamilyNouns() ans, err := pr.Select("What kind of data is this?", "tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/", @@ -250,38 +274,51 @@ func pickTask(p *ui.Printer, pr prompter, fam push.Family) (string, error) { return "", fmt.Errorf("no CLI-supported tasks for %s data yet", push.FamilyNoun(fam)) } - p.Section(fmt.Sprintf("Tasks for %s data", push.FamilyNoun(fam))) - p.Hintf("Available now:") + // Align the task IDs into a column so the blurbs line up, sized to the + // longest ID shown (available + pending). + width := 0 + for _, s := range available { + if len(s.ID) > width { + width = len(s.ID) + } + } + for _, s := range pending { + if len(s.ID) > width { + width = len(s.ID) + } + } + width += 3 + + p.PromptStep(4, 5, "What kind of machine learning task is this data for?") + p.Newline() for _, s := range available { - p.Infof("%s โ€” %s ยท %s", s.DisplayName(), s.Blurb, s.ID) + p.Para(fmt.Sprintf(" %-*s%s", width, s.ID, s.Blurb)) } if len(pending) > 0 { + p.Newline() p.Hintf("Not yet in the CLI:") for _, s := range pending { - p.Hintf(" %s โ€” %s ยท %s (%s)", s.DisplayName(), s.Blurb, s.ID, s.UnsupportedNote) + p.Hintf(" %-*s%s (%s)", width, s.ID, s.Blurb, s.UnsupportedNote) } } + p.Newline() + // The options ARE task IDs (what the list shows and what the user picks), + // so the answer is the category directly. Guard an unexpected answer by + // falling back to the first available โ€” never return an empty category. opts := make([]string, len(available)) for i, s := range available { - opts[i] = s.DisplayName() + opts[i] = s.ID } ans, err := pr.Select("Which task?", "pick the task this data is for", opts, opts[0]) if err != nil { return "", err } - // Map the answer back to a task by POSITION, not through a - // DisplayNameโ†’ID map: two tasks in a family could in principle share a - // display name, and a map would silently keep only the last, returning the - // wrong ID for the first. Matching the offered option by index returns the - // one the user actually saw (the list above is in this same order). - for i, o := range opts { - if o == ans { - return available[i].ID, nil + for _, id := range opts { + if id == ans { + return ans, nil } } - // Defensive: an answer that isn't one of the offered options. Never return - // an empty category โ€” fall back to the first available. return available[0].ID, nil } @@ -302,11 +339,14 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b // free-typing "Label" against a "label" header would cause. Wording is // per-task: a class to sort into vs a numeric value to predict (ยง8). if !push.SelfSupervisedText(cat) && a.Spec.LabelColumn == "" { - question := "Which column holds the class?" + question := "Which column holds the label?" + desc := "The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned" if push.IsRegressionClass(cat) { question = "Which column holds the value to predict?" + desc = "The number the model learns to predict. e.g. price, age, days_to_event" } - p.PromptHint("The column in your CSV with the answer the model learns to produce.") + p.PromptStep(5, 5, question) + p.Hintf("%s", desc) ans, err := promptLabelColumn(pr, cat, a.LocalPath, question) if err != nil { return prompted, err @@ -315,11 +355,15 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b prompted = true } + // Task-specific refinements โ€” shown under step 5, each with its own cyan + // header (Section) rather than a step number, since which ones appear + // depends on the task. switch { case push.IsImage(cat): if cat == "keypoint_detection" && a.Spec.NumberOfKeypoints <= 0 { - p.PromptHint("How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose") - ans, err := pr.Input("Number of keypoints per sample", + p.Section("How many keypoints per sample?") + p.Hintf("The number of landmark points each sample is annotated with โ€” dataset-specific. e.g. 17 for COCO human pose") + ans, err := pr.Input("How many keypoints per sample?", "e.g. 17 for COCO pose", "", validatePositiveInt) if err != nil { return prompted, err @@ -329,8 +373,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b prompted = true } if a.TargetSizeFlag == "" { - p.PromptHint("The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224") - ans, err := pr.Input("Image resolution as WxH (blank = read it from your first image)", + p.Section("Image resolution") + p.Hintf("The size your images already are, as WxH โ€” tracebloc checks every image matches and never resizes. Press Enter to read it from your first image. e.g. 224x224") + ans, err := pr.Input("Image resolution", "the size your images already are; tracebloc checks it, it never resizes", "", validateOptionalTargetSize) if err != nil { @@ -341,9 +386,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b } case push.IsTabular(cat): if a.SchemaFlag == "" { - p.PromptHint("Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR") - ans, err := pr.Input("Column schema as col:TYPE,... (blank = infer from the CSV)", - "e.g. age:INT,price:FLOAT", "", validateOptionalSchema) + p.Section("Column types") + p.Hintf("We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT.") + ans, err := pr.Input("Column types", "e.g. age:INT,price:FLOAT", "", validateOptionalSchema) if err != nil { return prompted, err } @@ -351,7 +396,8 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b prompted = true } if push.IsRegressionClass(cat) && a.Spec.LabelPolicy == "" { - p.PromptHint("Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values.") + p.Section("Label policy") + p.Hintf("Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values.") ans, err := pr.Select("Label policy", "bucket bins the target before it leaves the cluster", []string{"bucket", "passthrough"}, "bucket") @@ -362,7 +408,8 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b prompted = true } if cat == "time_to_event_prediction" && a.Spec.TimeColumn == "" { - p.PromptHint("The column holding the duration / time-to-event. e.g. time, tenure_days") + p.Section("Time column") + p.Hintf("The column holding the duration / time-to-event. e.g. time, tenure_days") ans, err := pr.Input("Time column", "the duration/time column name", "time", nil) if err != nil { return prompted, err diff --git a/internal/cli/interactive_test.go b/internal/cli/interactive_test.go index 2dfd6810..65b26f42 100644 --- a/internal/cli/interactive_test.go +++ b/internal/cli/interactive_test.go @@ -103,11 +103,11 @@ func textDirLayout(t *testing.T) string { func TestRunInteractive_PromptOrder(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "Is this training or test data?": "test", - "What should we call this dataset?": "churn_train", - "Where is your data? (file or folder)": dir, - "Which task?": "Tabular classification", - "Which column holds the class?": "churned", + "Do you want to ingest training or test data?": "test", + "Please name the dataset.": "churn_train", + "Where is your data?": dir, + "Which task?": "tabular_classification", + "Which column holds the label?": "churned", }} a := &runDataIngestArgs{} if err := runInteractive(discardPrinter(), f, a, false /*taskSet*/); err != nil { @@ -117,11 +117,11 @@ func TestRunInteractive_PromptOrder(t *testing.T) { // The four core questions must appear in data-first order, ahead of the // label question. want := []string{ - "Is this training or test data?", - "What should we call this dataset?", - "Where is your data? (file or folder)", + "Do you want to ingest training or test data?", + "Please name the dataset.", + "Where is your data?", "Which task?", - "Which column holds the class?", + "Which column holds the label?", } if !orderedSubsequence(f.asked, want) { t.Errorf("prompt order = %v, want subsequence %v", f.asked, want) @@ -139,11 +139,11 @@ func TestRunInteractive_PromptOrder(t *testing.T) { func TestRunInteractive_PathPromptCopyIsFileOrFolder(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "Is this training or test data?": "train", - "What should we call this dataset?": "churn", - "Where is your data? (file or folder)": dir, - "Which task?": "Tabular classification", - "Which column holds the class?": "churned", + "Do you want to ingest training or test data?": "train", + "Please name the dataset.": "churn", + "Where is your data?": dir, + "Which task?": "tabular_classification", + "Which column holds the label?": "churned", }} a := &runDataIngestArgs{} if err := runInteractive(discardPrinter(), f, a, false /*taskSet*/); err != nil { @@ -151,7 +151,7 @@ func TestRunInteractive_PathPromptCopyIsFileOrFolder(t *testing.T) { } found := false for _, label := range f.asked { - if label == "Where is your data? (file or folder)" { + if label == "Where is your data?" { found = true } if strings.Contains(label, "the folder holding it") { @@ -168,8 +168,8 @@ func TestRunInteractive_PathPromptCopyIsFileOrFolder(t *testing.T) { func TestRunInteractive_SniffEchoesFamily(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "Which column holds the class?": "churned", + "Please name the dataset.": "t", + "Which column holds the label?": "churned", }} a := &runDataIngestArgs{LocalPath: dir, Spec: push.SpecArgs{Intent: "train"}} var buf bytes.Buffer @@ -195,10 +195,10 @@ func TestRunInteractive_SniffEchoesFamily(t *testing.T) { func TestRunInteractive_SniffIsHintNotLock(t *testing.T) { empty := t.TempDir() // no csv, no images/, no texts/ โ†’ ambiguous f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "What kind of data is this?": "image", - "Which task?": "Image classification", - "Which column holds the class?": "label", + "Please name the dataset.": "t", + "What kind of data is this?": "image", + "Which task?": "image_classification", + "Which column holds the label?": "label", }} a := &runDataIngestArgs{LocalPath: empty, Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, false); err != nil { @@ -268,8 +268,8 @@ func TestResolveFamily_SurfacesMiscasedHint(t *testing.T) { func TestRunInteractive_ExplicitTaskSkipsSniff(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "Which column holds the class?": "churned", + "Please name the dataset.": "t", + "Which column holds the label?": "churned", }} a := &runDataIngestArgs{ LocalPath: dir, @@ -298,7 +298,7 @@ func TestPickTask_FamilyScoped(t *testing.T) { // Text family: all tasks are available now โ€” fill-mask (gloss), // classification, the two structured-pair tasks, and the two seq tasks; // image/tabular tasks must not appear. - f := &fakePrompter{answers: map[string]string{"Which task?": "Text classification"}} + f := &fakePrompter{answers: map[string]string{"Which task?": "text_classification"}} var buf bytes.Buffer p := ui.New(&buf, ui.WithColor(false)) id, err := pickTask(p, f, push.FamilyText) @@ -310,14 +310,13 @@ func TestPickTask_FamilyScoped(t *testing.T) { } out := buf.String() for _, want := range []string{ - "Tasks for text data", - "Available now:", - "fill-mask", // MLM gloss (available) - "Text classification", // label - "translation / summarization", // seq2seq gloss (now available) + "What kind of machine learning task is this data for?", + "masked_language_modeling", // MLM gloss (available) + "text_classification", // label + "seq2seq", // seq2seq gloss (now available) "token_classification", // now available "sentence_pair_classification", // now available - "Embeddings", // now available + "embeddings", // now available } { if !strings.Contains(out, want) { t.Errorf("picker output missing %q:\n%s", want, out) @@ -328,7 +327,7 @@ func TestPickTask_FamilyScoped(t *testing.T) { t.Errorf("text picker should have no pending section now:\n%s", out) } // Other families must not leak in. - for _, unwanted := range []string{"Image classification", "Tabular classification", "Survival analysis"} { + for _, unwanted := range []string{"image_classification", "tabular_classification", "time_to_event_prediction"} { if strings.Contains(out, unwanted) { t.Errorf("text picker leaked a non-text task %q:\n%s", unwanted, out) } @@ -339,7 +338,7 @@ func TestPickTask_FamilyScoped(t *testing.T) { // image task is available in the CLI, so the image picker lists them all under // "Available now:" with no greyed "Not yet in the CLI" pending section. func TestPickTask_ImageAllAvailable(t *testing.T) { - f := &fakePrompter{answers: map[string]string{"Which task?": "Image classification"}} + f := &fakePrompter{answers: map[string]string{"Which task?": "image_classification"}} var buf bytes.Buffer p := ui.New(&buf, ui.WithColor(false)) if _, err := pickTask(p, f, push.FamilyImage); err != nil { @@ -347,9 +346,8 @@ func TestPickTask_ImageAllAvailable(t *testing.T) { } out := buf.String() for _, want := range []string{ - "Available now:", - "Image classification", - "Semantic segmentation", // now selectable, no longer pending + "image_classification", + "semantic_segmentation", // now selectable, no longer pending } { if !strings.Contains(out, want) { t.Errorf("image picker missing %q:\n%s", want, out) @@ -366,7 +364,7 @@ func TestPickTask_ImageAllAvailable(t *testing.T) { // TestPickTask_TabularGloss: the tabular picker shows the survival-analysis // gloss for time_to_event_prediction and can select it back to its id. func TestPickTask_TabularGloss(t *testing.T) { - f := &fakePrompter{answers: map[string]string{"Which task?": "Survival analysis"}} + f := &fakePrompter{answers: map[string]string{"Which task?": "time_to_event_prediction"}} var buf bytes.Buffer p := ui.New(&buf, ui.WithColor(false)) id, err := pickTask(p, f, push.FamilyTabular) @@ -376,7 +374,7 @@ func TestPickTask_TabularGloss(t *testing.T) { if id != "time_to_event_prediction" { t.Errorf("id = %q, want time_to_event_prediction", id) } - if !strings.Contains(buf.String(), "Survival analysis") { + if !strings.Contains(buf.String(), "time_to_event_prediction") { t.Errorf("tabular picker missing the survival-analysis gloss:\n%s", buf.String()) } } @@ -388,9 +386,9 @@ func TestRunInteractive_LabelSelectFromHeaders(t *testing.T) { dir := tabularDir(t) // header: age,income,churned // Script an answer that only works if the options were the real headers. f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "Which task?": "Tabular classification", - "Which column holds the class?": "income", + "Please name the dataset.": "t", + "Which task?": "tabular_classification", + "Which column holds the label?": "income", }} a := &runDataIngestArgs{LocalPath: dir, Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, false); err != nil { @@ -418,7 +416,7 @@ func TestRunInteractive_RegressionLabelWording(t *testing.T) { if !contains(f.asked, "Which column holds the value to predict?") { t.Errorf("regression should ask for the value to predict; asked=%v", f.asked) } - if contains(f.asked, "Which column holds the class?") { + if contains(f.asked, "Which column holds the label?") { t.Errorf("regression must not use the class wording") } if a.Spec.LabelColumn != "income" { @@ -432,7 +430,7 @@ func TestRunInteractive_RegressionLabelWording(t *testing.T) { func TestRunInteractive_LabelFreeTextFallback(t *testing.T) { empty := t.TempDir() // no labels.csv โ†’ PreviewLabelHeaders errors f := &fakePrompter{answers: map[string]string{ - "Which column holds the class?": "my_label", + "Which column holds the label?": "my_label", }} a := &runDataIngestArgs{ LocalPath: empty, @@ -451,8 +449,8 @@ func TestRunInteractive_LabelFreeTextFallback(t *testing.T) { func TestRunInteractive_MLMSkipsLabel(t *testing.T) { dir := textDirLayout(t) f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "mlm_train", - "Which task?": "fill-mask", + "Please name the dataset.": "mlm_train", + "Which task?": "masked_language_modeling", }} a := &runDataIngestArgs{LocalPath: dir, Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, false); err != nil { @@ -492,8 +490,8 @@ func TestRunInteractive_SkipsProvidedValues(t *testing.T) { func TestRunInteractive_Keypoint(t *testing.T) { dir := imageDirLayout(t) f := &fakePrompter{answers: map[string]string{ - "Number of keypoints per sample": "17", - "Which column holds the class?": "image_label", + "How many keypoints per sample?": "17", + "Which column holds the label?": "image_label", }} a := &runDataIngestArgs{ LocalPath: dir, @@ -540,8 +538,8 @@ func TestRunInteractive_Cancel(t *testing.T) { no := false f := &fakePrompter{ answers: map[string]string{ - "What should we call this dataset?": "t", - "Which column holds the class?": "churned", + "Please name the dataset.": "t", + "Which column holds the label?": "churned", }, confirm: &no, } @@ -554,7 +552,7 @@ func TestRunInteractive_Cancel(t *testing.T) { // TestRunInteractive_RejectsBadName: the name prompt runs // push.ValidateTableName, so an unsafe name surfaces as an error. func TestRunInteractive_RejectsBadName(t *testing.T) { - f := &fakePrompter{answers: map[string]string{"What should we call this dataset?": "../bad"}} + f := &fakePrompter{answers: map[string]string{"Please name the dataset.": "../bad"}} a := &runDataIngestArgs{Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, true); err == nil { t.Fatal("expected an error for an invalid name, got nil") @@ -566,8 +564,8 @@ func TestRunInteractive_RejectsBadName(t *testing.T) { // directory (empty path โ†’ Abs("") โ†’ cwd). func TestRunInteractive_RejectsEmptyPath(t *testing.T) { f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "Where is your data? (file or folder)": " ", + "Please name the dataset.": "t", + "Where is your data?": " ", }} a := &runDataIngestArgs{Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, false); err == nil { @@ -583,9 +581,9 @@ func TestRunInteractive_RejectsEmptyPath(t *testing.T) { func TestRunInteractive_TrimsPath(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "t", - "Where is your data? (file or folder)": " " + dir + " ", - "Which column holds the class?": "churned", + "Please name the dataset.": "t", + "Where is your data?": " " + dir + " ", + "Which column holds the label?": "churned", }} a := &runDataIngestArgs{Spec: push.SpecArgs{Intent: "train"}} if err := runInteractive(discardPrinter(), f, a, false); err != nil { @@ -601,21 +599,23 @@ func TestRunInteractive_TrimsPath(t *testing.T) { } } -// TestRunInteractive_ShowsExampleHints: the name and path prompts carry a -// visible example, so the guided flow teaches as it goes. +// TestRunInteractive_ShowsExampleHints: the path and schema steps carry a +// visible example, so the guided flow teaches as it goes. (LocalPath is left +// empty so the path step โ€” and its per-modality examples โ€” actually renders.) func TestRunInteractive_ShowsExampleHints(t *testing.T) { dir := tabularDir(t) f := &fakePrompter{answers: map[string]string{ - "What should we call this dataset?": "churn_train", - "Which column holds the class?": "churned", + "Please name the dataset.": "churn_train", + "Where is your data?": dir, + "Which column holds the label?": "churned", }} - a := &runDataIngestArgs{LocalPath: dir, Spec: push.SpecArgs{Intent: "train"}} + a := &runDataIngestArgs{Spec: push.SpecArgs{Intent: "train"}} var buf bytes.Buffer p := ui.New(&buf, ui.WithColor(false)) if err := runInteractive(p, f, a, false); err != nil { t.Fatalf("runInteractive: %v", err) } - for _, want := range []string{"e.g. churn_train", "age:INT"} { + for _, want := range []string{"~/data/patients.csv", "age:INT"} { if !strings.Contains(buf.String(), want) { t.Errorf("interactive output missing hint %q:\n%s", want, buf.String()) } diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index ae7d50a1..c115bf17 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -1,50 +1,52 @@ tb data ingest โ€” stage a dataset into your secure environment ============================================================= What you see when you run `tb data ingest` with no flags: a short intro, then a -guided questionnaire. Every question is shown below, in order, driven through the -real flow for two tasks (tabular + image) so the task-specific questions are -visible. Each prompt shows `? `; the line above it is the -question's one-line description. Passing flags (--as, --task, a path, โ€ฆ) skips -the matching questions. The remaining tasks' extra questions (keypoints, label +five-step guided setup. Every question is shown below, in order, driven through +the real flow for two tasks (tabular + image) so the task-specific questions are +visible. Each question prints as a `Step N of 5 ยท โ€ฆ` header (task-specific +refinements as their own header); the supporting line sits beneath it, and the +`?` line shows your answer. Passing flags (--as, --task, a path, โ€ฆ) skips the +matching questions. The remaining tasks' extra questions (keypoints, label policy, time column) and every prompt's `?`-help text are in zz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a deprecated alias of the verb.) $ tb data ingest # guided ยท tabular classification - This ingests a dataset so models can train on it. Your files never leave your - own infrastructure โ€” tracebloc copies them into your secure environment's storage, - checks them, and loads them into a table your training runs read from. Other - collaborators can train against that table without ever seeing the raw files. - Learn more: https://docs.tracebloc.io + Ingest a dataset โ€” your files never leave this machine. + Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset - Let's set up your data ingest - Press Enter to accept a default; Ctrl-C to cancel. + Step 1 of 5 ยท Do you want to ingest training or test data? +? train - Whether this split trains the model or evaluates it. -? Is this training or test data? train + Step 2 of 5 ยท Please name the dataset. +? hospital_train - A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train -? What should we call this dataset? hospital_train - - The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn -? Where is your data? (file or folder) ~/datasets/hospital + Step 3 of 5 ยท Where is your data? + Give the path to a file or a folder โ€” whichever holds your data: + ยท Tabular one CSV file e.g. ~/data/patients.csv + ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ + ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ +? ~/datasets/hospital โœ” Found a CSV table โ€” this is tabular data. - Tasks for tabular data - Available now: - ยท Tabular classification โ€” predict a class from table columns ยท tabular_classification - ยท Tabular regression โ€” predict a number from table columns ยท tabular_regression - ยท Time-series forecasting โ€” predict future values from past ones ยท time_series_forecasting - ยท Time-series classification โ€” predict a class for each whole sequence ยท time_series_classification - ยท Survival analysis โ€” predict how long until an event happens ยท time_to_event_prediction -? Which task? Tabular classification + Step 4 of 5 ยท What kind of machine learning task is this data for? + + tabular_classification predict a class from table columns + tabular_regression predict a number from table columns + time_series_forecasting predict future values from past ones + time_series_classification predict a class for each whole sequence + time_to_event_prediction predict how long until an event happens + +? tabular_classification - The column in your CSV with the answer the model learns to produce. -? Which column holds the class? churned + Step 5 of 5 ยท Which column holds the label? + The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned +? churned - Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR -? Column schema as col:TYPE,... (blank = infer from the CSV) + Column types + We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT. +? Review name: hospital_train @@ -57,38 +59,39 @@ $ tb data ingest # guided ยท tabular classification $ tb data ingest # guided ยท image classification - This ingests a dataset so models can train on it. Your files never leave your - own infrastructure โ€” tracebloc copies them into your secure environment's storage, - checks them, and loads them into a table your training runs read from. Other - collaborators can train against that table without ever seeing the raw files. - Learn more: https://docs.tracebloc.io + Ingest a dataset โ€” your files never leave this machine. + Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset - Let's set up your data ingest - Press Enter to accept a default; Ctrl-C to cancel. + Step 1 of 5 ยท Do you want to ingest training or test data? +? train - Whether this split trains the model or evaluates it. -? Is this training or test data? train + Step 2 of 5 ยท Please name the dataset. +? xray_train - A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train -? What should we call this dataset? xray_train - - The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn -? Where is your data? (file or folder) ~/datasets/hospital + Step 3 of 5 ยท Where is your data? + Give the path to a file or a folder โ€” whichever holds your data: + ยท Tabular one CSV file e.g. ~/data/patients.csv + ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ + ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ +? ~/datasets/hospital โœ” Found labels.csv and an images/ folder โ€” this is image data. - Tasks for image data - Available now: - ยท Image classification โ€” sort images into classes ยท image_classification - ยท Object detection โ€” draw boxes around objects in an image ยท object_detection - ยท Keypoint detection โ€” locate landmark points on an image (e.g. pose) ยท keypoint_detection - ยท Semantic segmentation โ€” label every pixel in an image ยท semantic_segmentation -? Which task? Image classification + Step 4 of 5 ยท What kind of machine learning task is this data for? + + image_classification sort images into classes + object_detection draw boxes around objects in an image + keypoint_detection locate landmark points on an image (e.g. pose) + semantic_segmentation label every pixel in an image + +? image_classification - The column in your CSV with the answer the model learns to produce. -? Which column holds the class? label + Step 5 of 5 ยท Which column holds the label? + The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned +? label - The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224 -? Image resolution as WxH (blank = read it from your first image) 224x224 + Image resolution + The size your images already are, as WxH โ€” tracebloc checks every image matches and never resizes. Press Enter to read it from your first image. e.g. 224x224 +? 224x224 Review name: xray_train diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index 1878a0ed..6ccc0387 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -6,6 +6,8 @@ multi-step flows (ingest steps, login, progress, confirmations) not shown as a screen. %s/%d are runtime placeholders. "\"active\" is this machine's selected client; state is its last reported status to tracebloc." +"%-*s%s" +"%-*s%s (%s)" "%.1f%%" "%.2f GiB" "%.2f KiB" @@ -55,8 +57,6 @@ screen. %s/%d are runtime placeholders. "%s ยท running, but tracebloc hasn't heard from it โ€” run %s" "%s ยท starting up, not ready yet โ€” run %s" "%s ร—%d" -"%s โ€” %s ยท %s" -"%s โ€” %s ยท %s (%s)" "%s, โ€ฆ and %d more" "%s/%s" "%s: %w" @@ -84,7 +84,6 @@ screen. %s/%d are runtime placeholders. "0:%d" "3 GiB" "A dataset named %q already exists โ€” replace it?" -"A name for this dataset โ€” you'll reference it by this name when you start a training run. Start with a letter or underscore, then letters, digits, underscores. e.g. churn_train" "A real run continues with step 2 (copy into your secure environment) and step 3 (validate and load)." "A tracebloc client is already running on this cluster โ€” adopting it. Couldn't read the cluster identity, so its idempotency anchor was left unchanged; point --kubeconfig/--context at a cluster where kube-system is readable to stamp it." "A training run is allocated up to:" @@ -93,7 +92,6 @@ screen. %s/%d are runtime placeholders. "Already signed out." "Applies to your next training run; a run already going keeps its size." "Ask one of these admins (or ask them to grant you access)" -"Available now:" "CPU cores for one run (1โ€“%d)" "CSV %s has no columns" "Can't reach tracebloc from here." @@ -111,7 +109,7 @@ screen. %s/%d are runtime placeholders. "Client status" "Clients in your account" "Cluster teardown reported: %v" -"Column schema as col:TYPE,... (blank = infer from the CSV)" +"Column types" "Connected to tracebloc" "Connecting to your secure environmentโ€ฆ" "Copy into your secure environment" @@ -143,6 +141,7 @@ screen. %s/%d are runtime placeholders. "Details (for support)" "Diagnose auth / cluster problems with: tracebloc doctor" "Do you want to change the allocation? Run `%s resources set` (guided walkthrough on a terminal)." +"Do you want to ingest training or test data?" "Docker and related tools โ€” remove them yourself if you no longer need them" "Dry run โ€” nothing was changed" "Dry-run complete โ€” your data and secure environment check out; nothing was created." @@ -159,12 +158,15 @@ screen. %s/%d are runtime placeholders. "Free some up, or raise the machine's allocation in Docker Desktop โ†’ Resources." "Full log: %s" "GPU access removed โ€” training runs will use CPU only." +"Give the path to a file or a folder โ€” whichever holds your data:" "How many GPUs for one run (1โ€“%d)" -"How many keypoints each sample is annotated with โ€” dataset-specific, no default. e.g. 17 for COCO human pose" +"How many keypoints per sample?" "How much may one training run use?" "How much of this machine a training run may use" "If GPU training is expected, ensure one node has both the compute and the GPU capacity, with its device plugin." -"Image resolution as WxH (blank = read it from your first image)" +"Image resolution" +"Images a folder with labels.csv + images/ e.g. ~/data/xray/" +"Ingest a dataset โ€” your files never leave this machine." "Ingest settings" "Ingestion complete โ€” %s" "Ingestion complete โ€” showing its logs:" @@ -176,21 +178,18 @@ screen. %s/%d are runtime placeholders. "Ingestion summary" "Ingestor SA token" "Ingests a local dataset into your secure environment's storage,\nsubmits the ingestion run, and follows it to completion (streaming\nprogress + the final summary). Your data never leaves your own\ninfrastructure. Supports %[1]d tasks across the image, text, and\ntabular / time-series families; pick one with --task.\n\n is the data itself. What it looks like depends on the task:\n\n tabular / time-series โ€” the dataset is a single CSV. Pass the .csv\n file directly, or a folder holding exactly one .csv:\n\n churn.csv (the .csv file itself)\n or\n churn/\n data.csv (the one .csv in the folder)\n\n image (classification, object/keypoint detection) โ€” a folder with\n labels.csv + an images/ subfolder:\n\n cats_dogs/\n labels.csv (required)\n images/ (required)\n 001.jpg\n ...\n\n text (classification, masked language modeling) โ€” a folder with\n labels.csv + a %[2]s/ subfolder (masked language modeling uses %[3]s/):\n\n reviews/\n labels.csv (required)\n %[2]s/ (required โ€” %[3]s/ for masked language modeling)\n 001.txt\n ...\n\nA bare .csv file is accepted only for the tabular / time-series family;\nimage and text datasets must be a folder.\n\nAccepted image extensions: .jpg, .jpeg, or .png (case-insensitive).\nAll images in one dataset must share a single type โ€” the cluster\nvalidates the type it was told to expect.\n\nv0.1 caps the dataset at 1 GiB total + 500 MiB per file. Larger\ndatasets need the v0.2 cloud-source story (S3/GCS/HTTPS sources) โ€”\nsee tracebloc/client#147 non-goals.\n\nExit codes:\n 0 files staged + ingested successfully (or --detach: just staged + submitted)\n 2 schema validation failed (synthesized spec rejected) or\n v0.1-unsupported task passed\n 3 local-layout or kubeconfig error\n 4 cluster reachable but no tracebloc client / shared storage missing\n 5 ingestor SA token couldn't be obtained, or jobs-manager\n rejected the token (401/403)\n 6 destination table already exists (re-run with --overwrite to\n replace it, or pick a different --name)\n 7 pre-flight succeeded but staging the files failed\n (Pod creation, image pull, exec stream, or remote tar error) โ€”\n or, with --overwrite, removing the old table failed\n 8 jobs-manager rejected the submit (4xx/5xx other than auth)\n 9 ingestion Job exited non-zero, or completed with row-level\n failures the summary panel reports" -"Is this training or test data?" "Kept local data and config (~/.tracebloc); cleared the active-client pointer โ€” --keep-data." "Kept on tracebloc" "Kubeconfig" "Label policy" -"Learn more: https://docs.tracebloc.io" +"Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset" "Left %s in place โ€” it isn't tracebloc's `tb` alias." "Left alone" "Let each training run use up to %s?" -"Let's set up your data ingest" "Local dataset" "Machine credential โ€” needed by the installer to connect this client" "Memory" "Memory for one run in GiB (2โ€“%d)" -"MySQL identifier + PVC subdir; start with a letter or underscore, then letters, digits, underscore" "No client in namespace %q โ€” using the one in %q (override with --namespace)." "No clients yet. Run `tracebloc client create`." "No datasets yet โ€” ingest one with `%s data ingest`." @@ -212,16 +211,14 @@ screen. %s/%d are runtime placeholders. "Not signed in. Run `tracebloc login`." "Not yet in the CLI:" "Note: %d file(s) in images/ have no labels.csv row and won't be part of the dataset: %s" -"Number of keypoints per sample" "Offboarded %q. This machine is no longer connected to tracebloc." "Only tracebloc's small jobs-manager restarts โ€” running training isn't interrupted." "Open" -"Override the column types the CLI would infer. Blank = infer from the CSV. e.g. age:INT,price:FLOAT,city:VARCHAR" "POST %s%s: %w" "PVC is %v, not ReadWriteMany โ€” the stage Pod will co-locate with the existing mounter" "Pending > %s: %v" "Pick this dataset when you set it up." -"Press Enter to accept a default; Ctrl-C to cancel." +"Please name the dataset." "Private image pulls will ImagePullBackOff. Reinstall the chart with valid registry credentials." "Proceed with the ingest?" "Provision this client?" @@ -271,15 +268,15 @@ screen. %s/%d are runtime placeholders. "Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below." "System ยท %d" "Table %q already exists โ€” replacing it (table + files)." +"Tabular one CSV file e.g. ~/data/patients.csv" "Target" "Target cluster" -"Tasks for %s data" +"Text a folder with labels.csv + texts/ e.g. ~/data/reviews/" "The column holding the duration / time-to-event. e.g. time, tenure_days" -"The column in your CSV with the answer the model learns to produce." "The dataset's catalog metadata is kept as a record on tracebloc, marked unavailable โ€” never removed." -"The file or folder holding your data โ€” a single .csv for a table, or labels.csv + an images/ folder for images. e.g. ~/datasets/churn" "The name you provided was only control characters โ€” auto-naming this client instead." -"The resolution your images already are. tracebloc never resizes โ€” it checks every image is exactly this size and rejects any that differ. Blank = read it from your first image. e.g. 224x224" +"The number of landmark points each sample is annotated with โ€” dataset-specific. e.g. 17 for COCO human pose" +"The size your images already are, as WxH โ€” tracebloc checks every image matches and never resizes. Press Enter to read it from your first image. e.g. 224x224" "The tracebloc CLI (your local data & config are kept โ€” --keep-data)" "This CLI is out of date โ€” update it: %s" "This cluster is already registered as client %q (namespace %s) โ€” adopted it." @@ -301,13 +298,12 @@ screen. %s/%d are runtime placeholders. "VARCHAR(%d)" "Validate and load" "Verify the requests-proxy is wired: kubectl set env deploy/-jobs-manager --list | grep PROXY" -"We couldn't tell the data type from what's there โ€” which is it?" +"We couldn't tell from the layout โ€” tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/." +"We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT." "Welcome to your secure environment for AI, %s ๐Ÿ‘‹" "What kind of data is this?" -"What should we call this dataset?" "What's next" -"Where is your data? (file or folder)" -"Whether this split trains the model or evaluates it." +"Where is your data?" "Which task?" "Will delete" "Wrote a support bundle to ./%s" @@ -380,9 +376,9 @@ screen. %s/%d are runtime placeholders. "deleting stage Pod %s/%s: %w" "destination" "dropping %s.%s: %w%s" -"e.g. ./my-data" "e.g. 17 for COCO pose" "e.g. age:INT,price:FLOAT" +"e.g. ~/data/patients.csv or ~/data/xray/" "enter a whole number between %d and %d" "exactly %d" "exec stream against %s/%s: %w" @@ -424,6 +420,7 @@ screen. %s/%d are runtime placeholders. "label column" "label policy" "labels.csv" +"letters, digits, and underscores; start with a letter or underscore e.g. churn_train" "listing Pods for service %s/%s: %w" "listing chart-managed deployments in namespace %s: %w" "listing client deployments to check for an existing client: %w" diff --git a/internal/push/spec.go b/internal/push/spec.go index f7a2c3b0..a6c862d1 100644 --- a/internal/push/spec.go +++ b/internal/push/spec.go @@ -111,13 +111,8 @@ func ValidateTableName(table string) error { } if !tableNamePattern.MatchString(table) { return fmt.Errorf( - "table name %q is invalid: must start with a letter or "+ - "underscore, then letters, digits, and underscores only "+ - "(matches [A-Za-z_][A-Za-z0-9_]*). The table name is "+ - "used both as the MySQL table identifier and as the "+ - "/data/shared/
/ subdirectory on the cluster PVC, "+ - "so a leading digit, slashes, dots, and path-traversal "+ - "sequences are rejected.", + "%q won't work โ€” use letters, digits, and underscores, "+ + "starting with a letter or underscore (e.g. churn_train)", table) } return nil diff --git a/internal/ui/ui.go b/internal/ui/ui.go index 0dfd29ba..4f4b0695 100644 --- a/internal/ui/ui.go +++ b/internal/ui/ui.go @@ -260,6 +260,14 @@ func (p *Printer) Step(n, total int, label string) { p.out("\n%s %s\n", head, p.paint(label, color.Bold)) } +// PromptStep prints the header for one guided-setup question: a blank line, then +// "Step n of total ยท question" in the heading tone (cyan bold) โ€” the dominant +// line. Any supporting hint (Hintf/Infof) and the input prompt render beneath +// it, so the question reads first and the guidance is clearly secondary. +func (p *Printer) PromptStep(n, total int, question string) { + p.out("\n %s\n", p.hue(fmt.Sprintf("Step %d of %d ยท %s", n, total, question), toneHeading)) +} + // Successf prints a completed-item line with a green โœ”. The trailing // `f` + (format, args) signature is Go's convention for "takes a format // string" (cf. fmt.Printf vs fmt.Print). diff --git a/scripts/deadcode-allowlist.txt b/scripts/deadcode-allowlist.txt index c4ae1811..ac232768 100644 --- a/scripts/deadcode-allowlist.txt +++ b/scripts/deadcode-allowlist.txt @@ -15,3 +15,12 @@ internal/submit/watch.go: JobOutcome.String # code), so they are unreachable from main by design. internal/push/preflight.go: ReadLabelValues internal/push/tabular.go: inferColumnType +# Retained UI primitive + registry metadata the ingest redesign stopped +# rendering, both still pinned by tests: +# - PromptHeader (bold label before a prompt): the guided ingest flow moved to +# PromptStep, but this stays available for other prompt surfaces (color_matrix_test). +# - CategorySpec.DisplayName: the task picker now lists raw task IDs (approved +# design), but the friendly gloss-over-label metadata is kept โ€” pinned by +# preview_test.TestDisplayNameGlosses โ€” so re-showing glosses is a one-liner. +internal/ui/ui.go: Printer.PromptHeader +internal/push/category.go: CategorySpec.DisplayName From 2f81139d4b2b6d1d0d08bb976ba82ce9921185ef Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 11:11:30 +0200 Subject: [PATCH 09/14] Ingest: uniform prompt spacing + intro copy tweak MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Applies one consistent spacing rule to the whole guided flow (STYLE.md "Guided-prompt spacing"): header โ†’ blank โ†’ [supporting text โ†’ blank] โ†’ `?` prompt, with the answer-belongs-to-it result (the sniff echo) attached to the answer. Every step, the task-specific refinements, and the confirm now sit the same distance below their question โ€” tabular and image alike. - The confirm renders as its own "Proceed with the ingest?" header + a bare "? Yes" (surveyPrompter.Confirm honours bare), matching the step questions. - Intro copy: "Ingest datasets to your secure environment." / "For help: โ€ฆ". Regenerated 01-data-ingest.golden; the schema step ("Column types") now also carries the blank-before-prompt for consistency. Co-Authored-By: Claude Opus 4.8 --- STYLE.md | 20 +++++++++++ internal/cli/copy_catalog_test.go | 13 +++---- internal/cli/data_ingest_local.go | 4 +-- internal/cli/interactive.go | 30 ++++++++++++++-- .../cli/testdata/golden/01-data-ingest.golden | 34 +++++++++++++++---- .../cli/testdata/golden/zz-all-strings.golden | 4 +-- 6 files changed, 87 insertions(+), 18 deletions(-) diff --git a/STYLE.md b/STYLE.md index d544c880..fcbde6be 100644 --- a/STYLE.md +++ b/STYLE.md @@ -40,6 +40,26 @@ nothing when colour is off (`NO_COLOR` / non-TTY / `TERM=dumb` / `--plain`). The exact brand SGR is pinned by `internal/ui/brand_tones_test.go`, so a drift in the tone table fails CI. The installer mirrors this in `scripts/lib/common.sh`. +## Guided-prompt spacing + +Interactive flows (the `tb data ingest` questionnaire, and any future guided +flow) use one uniform rhythm so every question reads the same: + +- **One blank line before each question header** โ€” a `Step N of M ยท ` + (`PromptStep`) or an unnumbered refinement/confirm header (`Section`). The + header method emits this leading blank itself. +- **One blank line between the header and its supporting text** (the hint / + examples / option list), when there is any. +- **One blank line before the `?` prompt line.** With no supporting text, that + single blank sits directly between the header and the prompt. +- **A result that belongs to an answer attaches to it with no blank** โ€” e.g. the + `โœ” Found a CSV table โ€ฆ` sniff echo sits directly under the path answer. + +So: `header โ†’ blank โ†’ [supporting text โ†’ blank] โ†’ ? prompt`. The prompt line is +answer-only (`? train`); the question lives in the header (the prompter runs +`bare`), never repeated on the `?` line. Keep it uniform โ€” don't hand-tune the +spacing of individual questions. + ## Terminology Source of truth: the docs repo `TERMINOLOGY.md`. In user-facing output: diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 1a370d68..ab462b86 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -126,8 +126,8 @@ func TestCopyCatalog(t *testing.T) { var b bytes.Buffer p := ui.New(&b, ui.WithColor(false)) p.Newline() - p.Para("Ingest a dataset โ€” your files never leave this machine.") - p.Hintf("Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset") + p.Para("Ingest datasets to your secure environment.") + p.Hintf("For help: https://docs.tracebloc.io/create-use-case/prepare-dataset") pr := &catalogPrompter{w: &b, answers: answers} a := &runDataIngestArgs{} if err := runInteractive(p, pr, a, false /*taskSet*/); err != nil { @@ -393,14 +393,15 @@ func (c *catalogPrompter) Select(label, _ string, _ []string, def string) (strin return ans, nil } -// Confirm keeps its label (a short y/n with no header of its own โ€” matches the -// non-bare surveyPrompter Confirm). -func (c *catalogPrompter) Confirm(label string, def bool) (bool, error) { +// Confirm is bare in the guided flow too: the question ("Proceed with the +// ingest?") is printed by the CLI as its own header, so the prompt shows only +// the answer โ€” matching the bare surveyPrompter. +func (c *catalogPrompter) Confirm(_ string, def bool) (bool, error) { ans := "No" if def { ans = "Yes" } - fmt.Fprintf(c.w, "? %s %s\n", label, ans) + c.answerLine(ans) return def, nil } diff --git a/internal/cli/data_ingest_local.go b/internal/cli/data_ingest_local.go index 1a16ab85..5bf5fbd3 100644 --- a/internal/cli/data_ingest_local.go +++ b/internal/cli/data_ingest_local.go @@ -93,8 +93,8 @@ func resolveLocalInput(out, errOut io.Writer, a *runDataIngestArgs) (layout *pus } a.Printer.Newline() - a.Printer.Para("Ingest a dataset โ€” your files never leave this machine.") - a.Printer.Hintf("Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset") + a.Printer.Para("Ingest datasets to your secure environment.") + a.Printer.Hintf("For help: https://docs.tracebloc.io/create-use-case/prepare-dataset") // 0. Guided mode: prompt for any missing core inputs before // validation. Flags already provided win; non-TTY / --no-input diff --git a/internal/cli/interactive.go b/internal/cli/interactive.go index 1c49523f..6d9882db 100644 --- a/internal/cli/interactive.go +++ b/internal/cli/interactive.go @@ -80,9 +80,9 @@ func (s surveyPrompter) Select(label, help string, options []string, def string) return ans, nil } -func (surveyPrompter) Confirm(label string, def bool) (bool, error) { +func (s surveyPrompter) Confirm(label string, def bool) (bool, error) { ans := def - if err := survey.AskOne(&survey.Confirm{Message: label, Default: def}, &ans); err != nil { + if err := survey.AskOne(&survey.Confirm{Message: s.message(label), Default: def}, &ans); err != nil { return false, mapErr(err) } return ans, nil @@ -125,10 +125,17 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // prompt (the prompter runs bare โ€” see surveyPrompter). Task-specific // extras beyond the label (schema, resolution, โ€ฆ) are refinements under // step 5 and aren't separately numbered. + // + // Spacing is uniform (STYLE.md "Guided-prompt spacing"): the header carries + // its own leading blank; then one blank line, the optional supporting text, + // one blank line, and the `?` prompt. With no supporting text the single + // blank sits directly between header and prompt. A result that belongs to an + // answer (the sniff echo) attaches to it with no blank. // Step 1 โ€” intent: what this data is for. if a.Spec.Intent == "" { p.PromptStep(1, 5, "Do you want to ingest training or test data?") + p.Newline() ans, err := pr.Select("Do you want to ingest training or test data?", "which split this data is", []string{"train", "test"}, "train") if err != nil { @@ -142,6 +149,7 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // name is rejected (see ValidateTableName), so the prompt stays clean. if a.Spec.Table == "" { p.PromptStep(2, 5, "Please name the dataset.") + p.Newline() ans, err := pr.Input("Please name the dataset.", "letters, digits, and underscores; start with a letter or underscore e.g. churn_train", "", push.ValidateTableName) @@ -156,10 +164,12 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // detect the family from the layout and echo it back. if a.LocalPath == "" { p.PromptStep(3, 5, "Where is your data?") + p.Newline() p.Hintf("Give the path to a file or a folder โ€” whichever holds your data:") p.Infof("Tabular one CSV file e.g. ~/data/patients.csv") p.Infof("Images a folder with labels.csv + images/ e.g. ~/data/xray/") p.Infof("Text a folder with labels.csv + texts/ e.g. ~/data/reviews/") + p.Newline() ans, err := pr.Input("Where is your data?", "e.g. ~/data/patients.csv or ~/data/xray/", "", validateDatasetPath) if err != nil { return err @@ -214,6 +224,8 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // โ€” an ingest fully specified by flags (on a TTY) isn't nagged. if prompted { renderReview(p, a) + p.Section("Proceed with the ingest?") + p.Newline() ok, err := pr.Confirm("Proceed with the ingest?", true) if err != nil { return err @@ -243,7 +255,9 @@ func resolveFamily(p *ui.Printer, pr prompter, path string) (push.Family, error) p.Warnf("%s", s.Hint) } p.Section("What kind of data is this?") + p.Newline() p.Hintf("We couldn't tell from the layout โ€” tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/.") + p.Newline() opts := push.FamilyNouns() ans, err := pr.Select("What kind of data is this?", "tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/", @@ -346,7 +360,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b desc = "The number the model learns to predict. e.g. price, age, days_to_event" } p.PromptStep(5, 5, question) + p.Newline() p.Hintf("%s", desc) + p.Newline() ans, err := promptLabelColumn(pr, cat, a.LocalPath, question) if err != nil { return prompted, err @@ -362,7 +378,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b case push.IsImage(cat): if cat == "keypoint_detection" && a.Spec.NumberOfKeypoints <= 0 { p.Section("How many keypoints per sample?") + p.Newline() p.Hintf("The number of landmark points each sample is annotated with โ€” dataset-specific. e.g. 17 for COCO human pose") + p.Newline() ans, err := pr.Input("How many keypoints per sample?", "e.g. 17 for COCO pose", "", validatePositiveInt) if err != nil { @@ -374,7 +392,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b } if a.TargetSizeFlag == "" { p.Section("Image resolution") + p.Newline() p.Hintf("The size your images already are, as WxH โ€” tracebloc checks every image matches and never resizes. Press Enter to read it from your first image. e.g. 224x224") + p.Newline() ans, err := pr.Input("Image resolution", "the size your images already are; tracebloc checks it, it never resizes", "", validateOptionalTargetSize) @@ -387,7 +407,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b case push.IsTabular(cat): if a.SchemaFlag == "" { p.Section("Column types") + p.Newline() p.Hintf("We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT.") + p.Newline() ans, err := pr.Input("Column types", "e.g. age:INT,price:FLOAT", "", validateOptionalSchema) if err != nil { return prompted, err @@ -397,7 +419,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b } if push.IsRegressionClass(cat) && a.Spec.LabelPolicy == "" { p.Section("Label policy") + p.Newline() p.Hintf("Regression targets are continuous. 'bucket' groups them into ranges before they leave the cluster; 'passthrough' keeps raw values.") + p.Newline() ans, err := pr.Select("Label policy", "bucket bins the target before it leaves the cluster", []string{"bucket", "passthrough"}, "bucket") @@ -409,7 +433,9 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b } if cat == "time_to_event_prediction" && a.Spec.TimeColumn == "" { p.Section("Time column") + p.Newline() p.Hintf("The column holding the duration / time-to-event. e.g. time, tenure_days") + p.Newline() ans, err := pr.Input("Time column", "the duration/time column name", "time", nil) if err != nil { return prompted, err diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index c115bf17..e0e5d3dc 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -13,20 +13,24 @@ deprecated alias of the verb.) $ tb data ingest # guided ยท tabular classification - Ingest a dataset โ€” your files never leave this machine. - Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset + Ingest datasets to your secure environment. + For help: https://docs.tracebloc.io/create-use-case/prepare-dataset Step 1 of 5 ยท Do you want to ingest training or test data? + ? train Step 2 of 5 ยท Please name the dataset. + ? hospital_train Step 3 of 5 ยท Where is your data? + Give the path to a file or a folder โ€” whichever holds your data: ยท Tabular one CSV file e.g. ~/data/patients.csv ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ + ? ~/datasets/hospital โœ” Found a CSV table โ€” this is tabular data. @@ -41,11 +45,15 @@ $ tb data ingest # guided ยท tabular classification ? tabular_classification Step 5 of 5 ยท Which column holds the label? + The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned + ? churned Column types + We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT. + ? Review @@ -55,24 +63,31 @@ $ tb data ingest # guided ยท tabular classification path: ~/datasets/hospital label column: churned schema: infer from CSV -? Proceed with the ingest? Yes + + Proceed with the ingest? + +? Yes $ tb data ingest # guided ยท image classification - Ingest a dataset โ€” your files never leave this machine. - Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset + Ingest datasets to your secure environment. + For help: https://docs.tracebloc.io/create-use-case/prepare-dataset Step 1 of 5 ยท Do you want to ingest training or test data? + ? train Step 2 of 5 ยท Please name the dataset. + ? xray_train Step 3 of 5 ยท Where is your data? + Give the path to a file or a folder โ€” whichever holds your data: ยท Tabular one CSV file e.g. ~/data/patients.csv ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ + ? ~/datasets/hospital โœ” Found labels.csv and an images/ folder โ€” this is image data. @@ -86,11 +101,15 @@ $ tb data ingest # guided ยท image classification ? image_classification Step 5 of 5 ยท Which column holds the label? + The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned + ? label Image resolution + The size your images already are, as WxH โ€” tracebloc checks every image matches and never resizes. Press Enter to read it from your first image. e.g. 224x224 + ? 224x224 Review @@ -100,7 +119,10 @@ $ tb data ingest # guided ยท image classification path: ~/datasets/hospital label column: label resolution: 224x224 -? Proceed with the ingest? Yes + + Proceed with the ingest? + +? Yes ------------------------------------------------------------ diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index 6ccc0387..ec2a3d97 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -154,6 +154,7 @@ screen. %s/%d are runtime placeholders. "Enter" "Everything looks good โ€” you're ready to run training." "Follow it later with: kubectl logs -f -n %s job/%s" +"For help: https://docs.tracebloc.io/create-use-case/prepare-dataset" "Found labels.csv and a %s folder โ€” this looks like text data." "Free some up, or raise the machine's allocation in Docker Desktop โ†’ Resources." "Full log: %s" @@ -166,7 +167,7 @@ screen. %s/%d are runtime placeholders. "If GPU training is expected, ensure one node has both the compute and the GPU capacity, with its device plugin." "Image resolution" "Images a folder with labels.csv + images/ e.g. ~/data/xray/" -"Ingest a dataset โ€” your files never leave this machine." +"Ingest datasets to your secure environment." "Ingest settings" "Ingestion complete โ€” %s" "Ingestion complete โ€” showing its logs:" @@ -182,7 +183,6 @@ screen. %s/%d are runtime placeholders. "Kept on tracebloc" "Kubeconfig" "Label policy" -"Learn how: https://docs.tracebloc.io/create-use-case/prepare-dataset" "Left %s in place โ€” it isn't tracebloc's `tb` alias." "Left alone" "Let each training run use up to %s?" From 6d326f0d2e5af909d6d4fdf3628102721239a12f Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 11:20:30 +0200 Subject: [PATCH 10/14] Catalog: add the text/NLP family to the ingest flow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 01-data-ingest now drives one task per family โ€” tabular, image, AND text โ€” so the text path (and the full text task list: classification, masked/causal LM, seq2seq, token/sentence-pair classification, embeddings) is reviewable. Each flow now shows a self-consistent example path (~/data/patients, ~/data/xray, ~/data/reviews) instead of a shared placeholder. Self-supervised text (which skips the label step) stays in the backstop. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 22 ++++-- .../cli/testdata/golden/01-data-ingest.golden | 76 ++++++++++++++++--- 2 files changed, 81 insertions(+), 17 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index ab462b86..6386311e 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -122,7 +122,7 @@ func TestCopyCatalog(t *testing.T) { // picker, the task-specific questions, the review, and the confirm. The intro // preamble mirrors data_ingest_local.go (its text is drift-guarded by the // backstop); the temp data dir is normalised to a stable placeholder. - driveIngest := func(dir string, answers map[string]string) string { + driveIngest := func(dir, shownPath string, answers map[string]string) string { var b bytes.Buffer p := ui.New(&b, ui.WithColor(false)) p.Newline() @@ -133,18 +133,19 @@ func TestCopyCatalog(t *testing.T) { if err := runInteractive(p, pr, a, false /*taskSet*/); err != nil { t.Fatalf("driveIngest(%s): %v", dir, err) } - return strings.ReplaceAll(b.String(), dir, "~/datasets/hospital") + return strings.ReplaceAll(b.String(), dir, shownPath) } tabDir := tabularDir(t) imgDir := imageDirLayout(t) - tabularIngest := driveIngest(tabDir, map[string]string{ + txtDir := textDirLayout(t) + tabularIngest := driveIngest(tabDir, "~/data/patients", map[string]string{ "Do you want to ingest training or test data?": "train", "Please name the dataset.": "hospital_train", "Where is your data?": tabDir, "Which task?": "tabular_classification", "Which column holds the label?": "churned", }) - imageIngest := driveIngest(imgDir, map[string]string{ + imageIngest := driveIngest(imgDir, "~/data/xray", map[string]string{ "Do you want to ingest training or test data?": "train", "Please name the dataset.": "xray_train", "Where is your data?": imgDir, @@ -152,12 +153,23 @@ func TestCopyCatalog(t *testing.T) { "Which column holds the label?": "label", "Image resolution": "224x224", }) + // Text family: text_classification shows the label question; the picker lists + // every text task + blurb. (Self-supervised text โ€” masked/causal LM, seq2seq + // โ€” skips the label step; that path is covered by the backstop.) + textIngest := driveIngest(txtDir, "~/data/reviews", map[string]string{ + "Do you want to ingest training or test data?": "train", + "Please name the dataset.": "reviews_train", + "Where is your data?": txtDir, + "Which task?": "text_classification", + "Which column holds the label?": "label", + }) dataIngestFile := doc( "tb data ingest โ€” stage a dataset into your secure environment", - "What you see when you run `tb data ingest` with no flags: a short intro, then a\nfive-step guided setup. Every question is shown below, in order, driven through\nthe real flow for two tasks (tabular + image) so the task-specific questions are\nvisible. Each question prints as a `Step N of 5 ยท โ€ฆ` header (task-specific\nrefinements as their own header); the supporting line sits beneath it, and the\n`?` line shows your answer. Passing flags (--as, --task, a path, โ€ฆ) skips the\nmatching questions. The remaining tasks' extra questions (keypoints, label\npolicy, time column) and every prompt's `?`-help text are in\nzz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a\ndeprecated alias of the verb.)", + "What you see when you run `tb data ingest` with no flags: a short intro, then a\nfive-step guided setup. Every question is shown below, in order, driven through\nthe real flow for one task in each family (tabular, image, text) so the\ntask-specific questions are visible. Each question prints as a `Step N of 5 ยท โ€ฆ`\nheader (task-specific refinements as their own header); the supporting line sits\nbeneath it, and the `?` line shows your answer. Passing flags (--as, --task, a\npath, โ€ฆ) skips the matching questions. The other tasks' extra questions\n(keypoints, label policy, time column) and self-supervised text (which skips the\nlabel step) are in zz-all-strings.golden. (`tb ingest` is a hidden deprecated\nalias; `push` is a deprecated alias of the verb.)", []run{ {"tb data ingest # guided ยท tabular classification", tabularIngest}, {"tb data ingest # guided ยท image classification", imageIngest}, + {"tb data ingest # guided ยท text classification", textIngest}, }, []run{ {"tracebloc data ingest --help", help("data", "ingest")}, diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index e0e5d3dc..45cd2c80 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -2,14 +2,14 @@ tb data ingest โ€” stage a dataset into your secure environment ============================================================= What you see when you run `tb data ingest` with no flags: a short intro, then a five-step guided setup. Every question is shown below, in order, driven through -the real flow for two tasks (tabular + image) so the task-specific questions are -visible. Each question prints as a `Step N of 5 ยท โ€ฆ` header (task-specific -refinements as their own header); the supporting line sits beneath it, and the -`?` line shows your answer. Passing flags (--as, --task, a path, โ€ฆ) skips the -matching questions. The remaining tasks' extra questions (keypoints, label -policy, time column) and every prompt's `?`-help text are in -zz-all-strings.golden. (`tb ingest` is a hidden deprecated alias; `push` is a -deprecated alias of the verb.) +the real flow for one task in each family (tabular, image, text) so the +task-specific questions are visible. Each question prints as a `Step N of 5 ยท โ€ฆ` +header (task-specific refinements as their own header); the supporting line sits +beneath it, and the `?` line shows your answer. Passing flags (--as, --task, a +path, โ€ฆ) skips the matching questions. The other tasks' extra questions +(keypoints, label policy, time column) and self-supervised text (which skips the +label step) are in zz-all-strings.golden. (`tb ingest` is a hidden deprecated +alias; `push` is a deprecated alias of the verb.) $ tb data ingest # guided ยท tabular classification @@ -31,7 +31,7 @@ $ tb data ingest # guided ยท tabular classification ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ -? ~/datasets/hospital +? ~/data/patients โœ” Found a CSV table โ€” this is tabular data. Step 4 of 5 ยท What kind of machine learning task is this data for? @@ -60,7 +60,7 @@ $ tb data ingest # guided ยท tabular classification name: hospital_train task: tabular_classification intent: train - path: ~/datasets/hospital + path: ~/data/patients label column: churned schema: infer from CSV @@ -88,7 +88,7 @@ $ tb data ingest # guided ยท image classification ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ -? ~/datasets/hospital +? ~/data/xray โœ” Found labels.csv and an images/ folder โ€” this is image data. Step 4 of 5 ยท What kind of machine learning task is this data for? @@ -116,7 +116,7 @@ $ tb data ingest # guided ยท image classification name: xray_train task: image_classification intent: train - path: ~/datasets/hospital + path: ~/data/xray label column: label resolution: 224x224 @@ -124,6 +124,58 @@ $ tb data ingest # guided ยท image classification ? Yes +$ tb data ingest # guided ยท text classification + + Ingest datasets to your secure environment. + For help: https://docs.tracebloc.io/create-use-case/prepare-dataset + + Step 1 of 5 ยท Do you want to ingest training or test data? + +? train + + Step 2 of 5 ยท Please name the dataset. + +? reviews_train + + Step 3 of 5 ยท Where is your data? + + Give the path to a file or a folder โ€” whichever holds your data: + ยท Tabular one CSV file e.g. ~/data/patients.csv + ยท Images a folder with labels.csv + images/ e.g. ~/data/xray/ + ยท Text a folder with labels.csv + texts/ e.g. ~/data/reviews/ + +? ~/data/reviews + โœ” Found labels.csv and a texts/ folder โ€” this looks like text data. + + Step 4 of 5 ยท What kind of machine learning task is this data for? + + text_classification sort text snippets into classes + masked_language_modeling predict masked-out words โ€” no labels needed + causal_language_modeling predict the next word in a sequence + seq2seq map an input sequence to an output one + token_classification label each word in a sequence + sentence_pair_classification label how two texts relate + embeddings learn vector representations from text pairs + +? text_classification + + Step 5 of 5 ยท Which column holds the label? + + The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned + +? label + + Review + name: reviews_train + task: text_classification + intent: train + path: ~/data/reviews + label column: label + + Proceed with the ingest? + +? Yes + ------------------------------------------------------------ --help From 6c1d988a2b510c2435d83e497c6a0c67557a6971 Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 11:31:19 +0200 Subject: [PATCH 11/14] Catalog: render the ingest RUN (execution + summary) + coverage guard MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The catalog showed only the guided Q&A โ€” the execution (Check โ†’ Copy โ†’ Validate + the summary) was missing, which is what a user sees on every real run. Now 01-data-ingest also renders the run: the three step headers, the "Local dataset" / "Ingest settings" blocks (printLocalSummary), and the final summary + "What's next" (submit.RenderSummary) โ€” the real renderers, so it's drift-caught. Rendering the CLI's OWN screens excludes the raw ingestor stream (MySQL waits, the ๐Ÿ“Š banner, the doubled-word validator lines) โ€” that's the engine's stdout, not CLI copy โ€” so the catalog shows the clean view. Adds a coverage guard (mustRender): each command's primary path must appear as a rendered screen, so a dropped/half-rendered flow fails the test instead of silently vanishing (which is how the execution went missing in the first place). Every user-facing string is already guaranteed by the zz-all-strings AST harvest; this guards the SCREENS. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 71 ++++++++++++++++++- .../cli/testdata/golden/01-data-ingest.golden | 63 +++++++++++++--- 2 files changed, 124 insertions(+), 10 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 6386311e..701737f6 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -15,6 +15,7 @@ import ( "github.com/tracebloc/cli/internal/doctor" "github.com/tracebloc/cli/internal/push" + "github.com/tracebloc/cli/internal/submit" "github.com/tracebloc/cli/internal/ui" ) @@ -163,13 +164,56 @@ func TestCopyCatalog(t *testing.T) { "Which task?": "text_classification", "Which column holds the label?": "label", }) + // execIngest renders the run that follows the confirm โ€” the three steps and + // the final summary. printLocalSummary + submit.RenderSummary are the REAL + // renderers (drift-caught); the step headers/hints mirror the run + // orchestration (their strings are drift-guarded by zz-all-strings). The raw + // ingestor stream the CLI streams through (MySQL waits, the ๐Ÿ“Š banner, + // per-validator lines) is the *ingestor's* stdout โ€” not CLI copy โ€” so it + // isn't shown: this is the CLI's own view of the run. + execIngest := func() string { + var b bytes.Buffer + p := ui.New(&b, ui.WithColor(false)) + layout := &push.LocalLayout{ + Root: "~/data/patients", + LabelsCSV: "~/data/patients/data.csv", + TotalBytes: 52807, + } + spec := map[string]any{ + "category": "tabular_classification", + "table": "hospital_train", + "intent": "train", + "label": "churned", + "schema": map[string]string{ + "age": "INT", "income": "FLOAT", "tenure": "INT", "balance": "FLOAT", + "products": "INT", "active": "INT", "region": "VARCHAR(16)", + }, + } + p.Step(1, 3, "Check your data") + p.Hintf("Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away.") + printLocalSummary(p, layout, spec) + p.Step(2, 3, "Copy into your secure environment") + p.Hintf("Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you.") + p.Step(3, 3, "Validate and load") + p.Hintf("Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below.") + p.Hintf("This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later).") + submit.RenderSummary(p, &submit.Summary{ + IngestorID: "80c224bd-202b-4a6f-9362-61c84599f334", + TotalRecords: 849, + ProcessedRecords: 849, + InsertedRecords: 849, + APISentRecords: 849, + }) + return b.String() + } dataIngestFile := doc( "tb data ingest โ€” stage a dataset into your secure environment", - "What you see when you run `tb data ingest` with no flags: a short intro, then a\nfive-step guided setup. Every question is shown below, in order, driven through\nthe real flow for one task in each family (tabular, image, text) so the\ntask-specific questions are visible. Each question prints as a `Step N of 5 ยท โ€ฆ`\nheader (task-specific refinements as their own header); the supporting line sits\nbeneath it, and the `?` line shows your answer. Passing flags (--as, --task, a\npath, โ€ฆ) skips the matching questions. The other tasks' extra questions\n(keypoints, label policy, time column) and self-supervised text (which skips the\nlabel step) are in zz-all-strings.golden. (`tb ingest` is a hidden deprecated\nalias; `push` is a deprecated alias of the verb.)", + "What you see when you run `tb data ingest` with no flags: a short intro, a\nfive-step guided setup, then โ€” after you confirm โ€” the run itself. The setup is\ndriven through the real flow for one task in each family (tabular, image, text)\nso the task-specific questions are visible; each question prints as a\n`Step N of 5 ยท โ€ฆ` header (task-specific refinements as their own header), the\nsupporting line beneath it, and the `?` line shows your answer. The run (shown\nonce, for tabular) is the three steps + the final summary as the CLI renders\nthem. Passing flags (--as, --task, a path, โ€ฆ) skips the matching questions. The\nother tasks' extra questions (keypoints, label policy, time column),\nself-supervised text (which skips the label step), and the failure-summary\nwordings are in zz-all-strings.golden. The raw ingestor stream the CLI streams\nthrough (MySQL waits, the ๐Ÿ“Š banner, per-validator lines) is the engine's own\nstdout โ€” not CLI copy โ€” so it isn't shown. (`tb ingest` is a hidden deprecated\nalias; `push` is a deprecated alias of the verb.)", []run{ {"tb data ingest # guided ยท tabular classification", tabularIngest}, {"tb data ingest # guided ยท image classification", imageIngest}, {"tb data ingest # guided ยท text classification", textIngest}, + {"tb data ingest # after you confirm โ€” the run (tabular)", execIngest()}, }, []run{ {"tracebloc data ingest --help", help("data", "ingest")}, @@ -331,6 +375,31 @@ func TestCopyCatalog(t *testing.T) { "screen. %s/%d are runtime placeholders.\n\n" + strings.Join(quoteAll(harvestMessages(t)), "\n") + "\n", } + // Coverage guarantee: a command's PRIMARY PATH must be rendered as a screen, + // not left to the string backstop โ€” so a dropped/half-rendered flow fails the + // test instead of silently vanishing from the catalog (which is how the whole + // ingest execution went missing once). Each entry names markers that must + // appear in that file's rendered content; add one when a file gains a phase. + mustRender := map[string][]string{ + "00-home.golden": {"tracebloc --help"}, + "01-data-ingest.golden": { + "Ingest datasets to your secure environment.", // intro + "Step 1 of 5 ยท", "Step 5 of 5 ยท", // the guided questionnaire + "Review", "Proceed with the ingest?", // review + confirm + "Step 1/3", "Step 3/3", "Ingestion summary", "What's next", // the run + }, + "02-data-list.golden": {"tracebloc data list --help"}, + "05-doctor.golden": {"Connected to tracebloc", "Everything looks good"}, + } + for name, needles := range mustRender { + got := files[name] + for _, n := range needles { + if !strings.Contains(got, n) { + t.Errorf("%s is missing required primary-path copy %q โ€” a screen may have been dropped or half-rendered", name, n) + } + } + } + update := os.Getenv("TB_UPDATE_GOLDEN") != "" for name, content := range files { path := filepath.Join("testdata/golden", name) diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index 45cd2c80..0d59777a 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -1,14 +1,18 @@ tb data ingest โ€” stage a dataset into your secure environment ============================================================= -What you see when you run `tb data ingest` with no flags: a short intro, then a -five-step guided setup. Every question is shown below, in order, driven through -the real flow for one task in each family (tabular, image, text) so the -task-specific questions are visible. Each question prints as a `Step N of 5 ยท โ€ฆ` -header (task-specific refinements as their own header); the supporting line sits -beneath it, and the `?` line shows your answer. Passing flags (--as, --task, a -path, โ€ฆ) skips the matching questions. The other tasks' extra questions -(keypoints, label policy, time column) and self-supervised text (which skips the -label step) are in zz-all-strings.golden. (`tb ingest` is a hidden deprecated +What you see when you run `tb data ingest` with no flags: a short intro, a +five-step guided setup, then โ€” after you confirm โ€” the run itself. The setup is +driven through the real flow for one task in each family (tabular, image, text) +so the task-specific questions are visible; each question prints as a +`Step N of 5 ยท โ€ฆ` header (task-specific refinements as their own header), the +supporting line beneath it, and the `?` line shows your answer. The run (shown +once, for tabular) is the three steps + the final summary as the CLI renders +them. Passing flags (--as, --task, a path, โ€ฆ) skips the matching questions. The +other tasks' extra questions (keypoints, label policy, time column), +self-supervised text (which skips the label step), and the failure-summary +wordings are in zz-all-strings.golden. The raw ingestor stream the CLI streams +through (MySQL waits, the ๐Ÿ“Š banner, per-validator lines) is the engine's own +stdout โ€” not CLI copy โ€” so it isn't shown. (`tb ingest` is a hidden deprecated alias; `push` is a deprecated alias of the verb.) $ tb data ingest # guided ยท tabular classification @@ -176,6 +180,47 @@ $ tb data ingest # guided ยท text classification ? Yes +$ tb data ingest # after you confirm โ€” the run (tabular) + +Step 1/3 Check your data + Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away. + + Local dataset + root: ~/data/patients + data CSV: ~/data/patients/data.csv + columns: 7 + total size: 51.57 KiB + + Ingest settings + name: hospital_train + task: tabular_classification + intent: train + label column: churned + destination: /data/shared/hospital_train + +Step 2/3 Copy into your secure environment + Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you. + +Step 3/3 Validate and load + Submitting the run, then following along as tracebloc validates your data and loads it into the table โ€” progress streams below. + This follows the run for up to an hour; a longer run keeps going on its own (or start it with --detach and check back later). + โœ” Ingestion complete โ€” ingested 849 of 849 records (100.0%) + + Ingestion summary + ingestor ID: 80c224bd-202b-4a6f-9362-61c84599f334 + total records: 849 + inserted: 849 + sent to API: 849 + skipped: 0 + file failures: 0 + DB failures: 0 + success rate: 100.0% + + What's next + ยท Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata + ยท To train or benchmark models on it, create a use case at https://ai.tracebloc.io/my-use-cases + Pick this dataset when you set it up. + ------------------------------------------------------------ --help From b900d52ddaf354918c2a0f401b073fb8df8cd98b Mon Sep 17 00:00:00 2001 From: Lukas Wuttke Date: Wed, 22 Jul 2026 12:04:07 +0200 Subject: [PATCH 12/14] Ingest: lean the summary + drop the duplicate settings block MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two execution-copy fixes from the redesign review: - Ingestion summary is lean: the clean-run headline already says "X of Y records (N%)", so the summary drops "sent to API", the zero-value failure counters, and the redundant success-rate line. Everything below total/inserted is now a SHORTFALL, shown only when it happened โ€” incl. "sent to API" gated on api_sent < inserted (the one failure mode the headline can't show), so no silent data-loss signal is hidden. - The step-1 "Ingest settings" block is suppressed on the guided path: the pre-confirm Review already showed name/task/intent/label, so repeating them was pure duplication. The flag-only path (no Review) still shows it once. Catalog regenerated; the run now reads clean. Step-format unification (the run's "Step N/3" vs the questionnaire's "Step N of 5 ยท") is deferred โ€” Step mirrors the installer's bash step(), so it's a coordinated CLI+installer change, not a CLI-only tweak. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 5 ++++- internal/cli/coverage_test.go | 2 +- internal/cli/data_ingest_local.go | 22 ++++++++++++++----- internal/cli/data_test.go | 2 +- .../cli/testdata/golden/01-data-ingest.golden | 12 ---------- .../cli/testdata/golden/zz-all-strings.golden | 2 -- internal/submit/summary.go | 22 ++++++++++++++----- 7 files changed, 40 insertions(+), 27 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 701737f6..2427b71a 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -191,7 +191,10 @@ func TestCopyCatalog(t *testing.T) { } p.Step(1, 3, "Check your data") p.Hintf("Reading your files locally first โ€” nothing has touched your secure environment yet โ€” so a layout or settings problem shows up right away.") - printLocalSummary(p, layout, spec) + // Guided path: the Review already echoed the settings, so the duplicate + // "Ingest settings" block is suppressed (showSettings=false). The flag-only + // path passes true; that block's copy is in zz-all-strings.golden. + printLocalSummary(p, layout, spec, false) p.Step(2, 3, "Copy into your secure environment") p.Hintf("Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you.") p.Step(3, 3, "Validate and load") diff --git a/internal/cli/coverage_test.go b/internal/cli/coverage_test.go index f07e4612..ac6dd7db 100644 --- a/internal/cli/coverage_test.go +++ b/internal/cli/coverage_test.go @@ -53,7 +53,7 @@ func TestPrintPushPreflight_RendersKeyFacts(t *testing.T) { // TestPrintClusterSummary_VerboseOnly). The local summary shows regardless. var buf bytes.Buffer p := ui.New(&buf, ui.WithColor(false), ui.WithVerbose(true)) - printLocalSummary(p, layout, spec) + printLocalSummary(p, layout, spec, true) printClusterSummary(p, release, pvc) out := buf.String() diff --git a/internal/cli/data_ingest_local.go b/internal/cli/data_ingest_local.go index 5bf5fbd3..50ad9b68 100644 --- a/internal/cli/data_ingest_local.go +++ b/internal/cli/data_ingest_local.go @@ -480,15 +480,23 @@ func resolveLocalInput(out, errOut io.Writer, a *runDataIngestArgs) (layout *pus return nil, nil, nil, false, perr } - printLocalSummary(a.Printer, layout, spec) + // Interactive runs already echoed name/task/intent/label in the pre-confirm + // Review, so suppress the duplicate "Ingest settings" block here; the + // flag-only path (no Review) still shows it. + printLocalSummary(a.Printer, layout, spec, a.Prompter == nil) return layout, spec, specBytes, false, nil } -// printLocalSummary shows what the CLI found on disk plus the ingest -// settings it assembled โ€” the detail under step 1 ("Check your data"). -// Mirrors `cluster info`'s section/Field layout. -func printLocalSummary(p *ui.Printer, layout *push.LocalLayout, spec map[string]any) { +// printLocalSummary shows what the CLI found on disk plus (for the flag-only +// path) the ingest settings it assembled โ€” the detail under step 1 ("Check your +// data"). Mirrors `cluster info`'s section/Field layout. +// +// showSettings gates the "Ingest settings" block: the guided flow already +// showed those exact fields in its pre-confirm Review, so repeating them here +// is noise โ€” the caller passes false when interactive. The flag-only path +// (no Review) passes true, so those users still see the resolved settings once. +func printLocalSummary(p *ui.Printer, layout *push.LocalLayout, spec map[string]any, showSettings bool) { cat, _ := spec["category"].(string) p.Section("Local dataset") @@ -523,6 +531,10 @@ func printLocalSummary(p *ui.Printer, layout *push.LocalLayout, spec map[string] } p.Field("total size", push.HumanBytes(layout.TotalBytes)) + if !showSettings { + return + } + p.Section("Ingest settings") p.Field("name", fmt.Sprintf("%v", spec["table"])) p.Field("task", fmt.Sprintf("%v", spec["category"])) diff --git a/internal/cli/data_test.go b/internal/cli/data_test.go index a4ceb419..257f564d 100644 --- a/internal/cli/data_test.go +++ b/internal/cli/data_test.go @@ -589,7 +589,7 @@ func TestPrintLocalSummary_ShowsDetectedExtension(t *testing.T) { "table": "t", "category": "image_classification", "intent": "train", "spec": map[string]any{"file_options": map[string]any{"extension": ".png"}}, } - printLocalSummary(p, layout, spec) + printLocalSummary(p, layout, spec, true) if !strings.Contains(buf.String(), "1 files (.png)") { t.Errorf("summary missing detected extension:\n%s", buf.String()) } diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index 0d59777a..e7fd412f 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -191,13 +191,6 @@ Step 1/3 Check your data columns: 7 total size: 51.57 KiB - Ingest settings - name: hospital_train - task: tabular_classification - intent: train - label column: churned - destination: /data/shared/hospital_train - Step 2/3 Copy into your secure environment Your files are copied securely into your secure environment's storage โ€” set up and cleaned up for you. @@ -210,11 +203,6 @@ Step 3/3 Validate and load ingestor ID: 80c224bd-202b-4a6f-9362-61c84599f334 total records: 849 inserted: 849 - sent to API: 849 - skipped: 0 - file failures: 0 - DB failures: 0 - success rate: 100.0% What's next ยท Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index ec2a3d97..39d951ed 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -8,7 +8,6 @@ screen. %s/%d are runtime placeholders. "\"active\" is this machine's selected client; state is its last reported status to tracebloc." "%-*s%s" "%-*s%s (%s)" -"%.1f%%" "%.2f GiB" "%.2f KiB" "%.2f MiB" @@ -547,7 +546,6 @@ screen. %s/%d are runtime placeholders. "streaming logs from Pod %s/%s: %w" "submit response missing job_name (got body %q)" "submit response missing namespace (got body %q)" -"success rate" "synthesized spec failed schema validation; check the flag values above" "tabular = a CSV table; image = labels.csv + images/; text = labels.csv + texts/" "task" diff --git a/internal/submit/summary.go b/internal/submit/summary.go index a52f14b5..f2793401 100644 --- a/internal/submit/summary.go +++ b/internal/submit/summary.go @@ -425,11 +425,23 @@ func RenderSummary(p *ui.Printer, s *Summary) { } p.Field("total records", commaSep(s.TotalRecords)) p.Field("inserted", commaSep(s.InsertedRecords)) - p.Field("sent to API", commaSep(s.APISentRecords)) - p.Field("skipped", commaSep(s.SkippedRecords)) - p.Field("file failures", commaSep(s.FileTransferFailures)) - p.Field("DB failures", commaSep(s.FailedRecords)) - p.Field("success rate", fmt.Sprintf("%.1f%%", s.SuccessRate())) + // Everything below is a SHORTFALL โ€” shown only when it actually happened. + // A clean run says it all in the headline (X of Y, %), so zero-row failure + // counters, a "sent to API" that equals inserted, and a redundant + // success-rate line would just be noise. The api-sync shortfall is kept + // because it's the one failure mode the headline (inserted/total) can't show. + if s.APISentRecords < s.InsertedRecords { + p.Field("sent to API", commaSep(s.APISentRecords)) + } + if s.SkippedRecords > 0 { + p.Field("skipped", commaSep(s.SkippedRecords)) + } + if s.FileTransferFailures > 0 { + p.Field("file failures", commaSep(s.FileTransferFailures)) + } + if s.FailedRecords > 0 { + p.Field("DB failures", commaSep(s.FailedRecords)) + } p.Section("What's next") p.Infof("Your data is registered as a dataset. View it at https://ai.tracebloc.io/metadata") From 7cf71e835d9648840daa9900174a17e38b071ebf Mon Sep 17 00:00:00 2001 From: Asad Iqbal Date: Wed, 22 Jul 2026 15:19:34 +0500 Subject: [PATCH 13/14] fix(cli): address bugbot findings on the guided ingest flow Three verified bugbot findings on PR #385: - Blank overwrite confirm (High): surveyPrompter.Confirm applied `bare` via message(), so the label was dropped. The overwrite-replace confirm in existingTableAction fires during the cluster phase with no header of its own, rendering a label-less `? (y/N)` before a destructive replace. Confirm now always keeps its label (matching its documented contract); drop the now-redundant Section header before "Proceed with the ingest?". - Settings hidden without Review (Medium): printLocalSummary gated the "Ingest settings" block on `Prompter == nil`, but a fully flagged run on a TTY sets a prompter yet prompts nothing, so it skipped the Review AND dropped settings that non-interactive runs still show. Gate on a new ReviewShown flag (set only when the Review actually renders) instead. - PromptStep omitted from string harvest (Medium): harvestMessages missed the new PromptStep method, so copy that only appears there (e.g. "What kind of machine learning task is this data for?") never landed in zz-all-strings.golden and the completeness backstop couldn't catch drift. Regenerated the golden catalog; catalogPrompter.Confirm now echoes its label to match the non-bare surveyPrompter. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 13 +++++++------ internal/cli/data_ingest_cmd.go | 8 ++++++++ internal/cli/data_ingest_local.go | 19 +++++++++++-------- internal/cli/interactive.go | 11 +++++++++-- .../cli/testdata/golden/01-data-ingest.golden | 12 +++--------- .../cli/testdata/golden/zz-all-strings.golden | 1 + 6 files changed, 39 insertions(+), 25 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 2427b71a..76bde32c 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -477,15 +477,15 @@ func (c *catalogPrompter) Select(label, _ string, _ []string, def string) (strin return ans, nil } -// Confirm is bare in the guided flow too: the question ("Proceed with the -// ingest?") is printed by the CLI as its own header, so the prompt shows only -// the answer โ€” matching the bare surveyPrompter. -func (c *catalogPrompter) Confirm(_ string, def bool) (bool, error) { +// Confirm keeps its label (never bare โ€” see surveyPrompter.Confirm): a y/N +// prompt has no header of its own, so survey draws "? ". +// Mirror that here so the catalog shows the confirm question. +func (c *catalogPrompter) Confirm(label string, def bool) (bool, error) { ans := "No" if def { ans = "Yes" } - c.answerLine(ans) + fmt.Fprintf(c.w, "? %s %s\n", label, ans) return def, nil } @@ -501,7 +501,8 @@ func harvestMessages(t *testing.T) []string { methods := map[string]bool{ "Successf": true, "Warnf": true, "Errorf": true, "Infof": true, "Hintf": true, "Detailf": true, "Para": true, "Section": true, "PromptHint": true, "PromptHeader": true, - "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, + "PromptStep": true, + "WarnLine": true, "CrossLine": true, "CheckLine": true, "Step": true, "Action": true, "Stat": true, "Field": true, "MenuRow": true, "Banner": true, "Command": true, // prompter seam (survey) โ€” question labels + help text for every guided // flow (ingest, client create, delete), incl. flows not driven as a screen. diff --git a/internal/cli/data_ingest_cmd.go b/internal/cli/data_ingest_cmd.go index 52e9d73d..0f3d8543 100644 --- a/internal/cli/data_ingest_cmd.go +++ b/internal/cli/data_ingest_cmd.go @@ -347,6 +347,14 @@ type runDataIngestArgs struct { Prompter prompter TaskSet bool + // ReviewShown records whether the guided flow rendered the pre-confirm + // Review (it only does when it actually prompted for something). It gates + // the duplicate "Ingest settings" block in printLocalSummary: suppress it + // only when the Review already showed those fields. A fully flagged run โ€” + // even on a TTY with a Prompter set โ€” prompts nothing, shows no Review, so + // it must still print the settings once (like the non-interactive path). + ReviewShown bool + // ChangedFlags records which CLI flags were EXPLICITLY set // (cmd.Flags().Changed), decoupling "was it passed" from "is its value // non-zero" โ€” the value alone can't tell `--number-of-keypoints 0` (an diff --git a/internal/cli/data_ingest_local.go b/internal/cli/data_ingest_local.go index 50ad9b68..a4a394a5 100644 --- a/internal/cli/data_ingest_local.go +++ b/internal/cli/data_ingest_local.go @@ -480,10 +480,12 @@ func resolveLocalInput(out, errOut io.Writer, a *runDataIngestArgs) (layout *pus return nil, nil, nil, false, perr } - // Interactive runs already echoed name/task/intent/label in the pre-confirm - // Review, so suppress the duplicate "Ingest settings" block here; the - // flag-only path (no Review) still shows it. - printLocalSummary(a.Printer, layout, spec, a.Prompter == nil) + // Interactive runs that prompted already echoed name/task/intent/label in + // the pre-confirm Review, so suppress the duplicate "Ingest settings" block + // here. A run that showed no Review (non-interactive, OR a fully flagged + // run on a TTY) still shows it โ€” gate on whether the Review rendered, not on + // whether a Prompter exists. + printLocalSummary(a.Printer, layout, spec, !a.ReviewShown) return layout, spec, specBytes, false, nil } @@ -492,10 +494,11 @@ func resolveLocalInput(out, errOut io.Writer, a *runDataIngestArgs) (layout *pus // path) the ingest settings it assembled โ€” the detail under step 1 ("Check your // data"). Mirrors `cluster info`'s section/Field layout. // -// showSettings gates the "Ingest settings" block: the guided flow already -// showed those exact fields in its pre-confirm Review, so repeating them here -// is noise โ€” the caller passes false when interactive. The flag-only path -// (no Review) passes true, so those users still see the resolved settings once. +// showSettings gates the "Ingest settings" block: when the guided flow already +// showed those exact fields in its pre-confirm Review, repeating them here is +// noise โ€” the caller passes false in that case. Any run without a Review (a +// non-interactive run, or a fully flagged run on a TTY that prompted nothing) +// passes true, so those users still see the resolved settings once. func printLocalSummary(p *ui.Printer, layout *push.LocalLayout, spec map[string]any, showSettings bool) { cat, _ := spec["category"].(string) diff --git a/internal/cli/interactive.go b/internal/cli/interactive.go index 6d9882db..d21a4cb4 100644 --- a/internal/cli/interactive.go +++ b/internal/cli/interactive.go @@ -81,8 +81,12 @@ func (s surveyPrompter) Select(label, help string, options []string, def string) } func (s surveyPrompter) Confirm(label string, def bool) (bool, error) { + // Confirm always keeps its label (never bare): a y/N prompt has no step + // header of its own, and the overwrite-replace confirm fires later, during + // the cluster phase, with nothing printed before it โ€” a bare "? (y/N)" + // there would be a label-less destructive prompt. ans := def - if err := survey.AskOne(&survey.Confirm{Message: s.message(label), Default: def}, &ans); err != nil { + if err := survey.AskOne(&survey.Confirm{Message: label, Default: def}, &ans); err != nil { return false, mapErr(err) } return ans, nil @@ -224,7 +228,10 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // โ€” an ingest fully specified by flags (on a TTY) isn't nagged. if prompted { renderReview(p, a) - p.Section("Proceed with the ingest?") + a.ReviewShown = true + // No header here: Confirm keeps its own label ("Proceed with the + // ingest?"), so a Section would just duplicate it. One blank line for + // breathing room between the Review block and the y/N prompt. p.Newline() ok, err := pr.Confirm("Proceed with the ingest?", true) if err != nil { diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index e7fd412f..797fcd5f 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -68,9 +68,7 @@ $ tb data ingest # guided ยท tabular classification label column: churned schema: infer from CSV - Proceed with the ingest? - -? Yes +? Proceed with the ingest? Yes $ tb data ingest # guided ยท image classification @@ -124,9 +122,7 @@ $ tb data ingest # guided ยท image classification label column: label resolution: 224x224 - Proceed with the ingest? - -? Yes +? Proceed with the ingest? Yes $ tb data ingest # guided ยท text classification @@ -176,9 +172,7 @@ $ tb data ingest # guided ยท text classification path: ~/data/reviews label column: label - Proceed with the ingest? - -? Yes +? Proceed with the ingest? Yes $ tb data ingest # after you confirm โ€” the run (tabular) diff --git a/internal/cli/testdata/golden/zz-all-strings.golden b/internal/cli/testdata/golden/zz-all-strings.golden index 39d951ed..009e7131 100644 --- a/internal/cli/testdata/golden/zz-all-strings.golden +++ b/internal/cli/testdata/golden/zz-all-strings.golden @@ -301,6 +301,7 @@ screen. %s/%d are runtime placeholders. "We infer each column's type from your CSV. Press Enter to accept, or type overrides like age:INT,price:FLOAT." "Welcome to your secure environment for AI, %s ๐Ÿ‘‹" "What kind of data is this?" +"What kind of machine learning task is this data for?" "What's next" "Where is your data?" "Which task?" From 6bf1328ed2b3ec3f1d505f20a85944ef84064bfe Mon Sep 17 00:00:00 2001 From: Asad Iqbal Date: Wed, 22 Jul 2026 15:31:13 +0500 Subject: [PATCH 14/14] fix(cli): number only the 4 core ingest steps; label is a task detail MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the remaining bugbot finding (#3, "Step total wrong without label"): self-supervised text (MLM/CLM) has no label column, so the guided flow showed "Step 4 of 5" and then jumped straight to Review with no fifth step. Root cause: the label column was numbered as a universal "step 5 of 5", but it isn't universal โ€” it's task-specific, exactly like keypoints (image) or schema (tabular), none of which are numbered. Fix at that level: the four core steps (intent โ†’ name โ†’ path โ†’ task) are the only numbered ones ("of 4"), and the label joins the other task-dependent inputs as an unnumbered Section header. The count is now honest for every task, including the ones that have no label. Regenerated the golden catalog + updated the catalog's required-copy assertions and screen description to match. Co-Authored-By: Claude Opus 4.8 --- internal/cli/copy_catalog_test.go | 4 +- internal/cli/interactive.go | 46 +++++++++++-------- .../cli/testdata/golden/01-data-ingest.golden | 40 ++++++++-------- 3 files changed, 49 insertions(+), 41 deletions(-) diff --git a/internal/cli/copy_catalog_test.go b/internal/cli/copy_catalog_test.go index 76bde32c..433fabde 100644 --- a/internal/cli/copy_catalog_test.go +++ b/internal/cli/copy_catalog_test.go @@ -211,7 +211,7 @@ func TestCopyCatalog(t *testing.T) { } dataIngestFile := doc( "tb data ingest โ€” stage a dataset into your secure environment", - "What you see when you run `tb data ingest` with no flags: a short intro, a\nfive-step guided setup, then โ€” after you confirm โ€” the run itself. The setup is\ndriven through the real flow for one task in each family (tabular, image, text)\nso the task-specific questions are visible; each question prints as a\n`Step N of 5 ยท โ€ฆ` header (task-specific refinements as their own header), the\nsupporting line beneath it, and the `?` line shows your answer. The run (shown\nonce, for tabular) is the three steps + the final summary as the CLI renders\nthem. Passing flags (--as, --task, a path, โ€ฆ) skips the matching questions. The\nother tasks' extra questions (keypoints, label policy, time column),\nself-supervised text (which skips the label step), and the failure-summary\nwordings are in zz-all-strings.golden. The raw ingestor stream the CLI streams\nthrough (MySQL waits, the ๐Ÿ“Š banner, per-validator lines) is the engine's own\nstdout โ€” not CLI copy โ€” so it isn't shown. (`tb ingest` is a hidden deprecated\nalias; `push` is a deprecated alias of the verb.)", + "What you see when you run `tb data ingest` with no flags: a short intro, a\nfour-step guided setup (intent, name, path, task) then the task-specific\nquestions, and โ€” after you confirm โ€” the run itself. The setup is\ndriven through the real flow for one task in each family (tabular, image, text)\nso the task-specific questions are visible; each core question prints as a\n`Step N of 4 ยท โ€ฆ` header, the task-specific ones (the label column, and extras\nlike resolution or schema) as their own header, the\nsupporting line beneath it, and the `?` line shows your answer. The run (shown\nonce, for tabular) is the three steps + the final summary as the CLI renders\nthem. Passing flags (--as, --task, a path, โ€ฆ) skips the matching questions. The\nother tasks' extra questions (keypoints, label policy, time column),\nself-supervised text (which skips the label question), and the failure-summary\nwordings are in zz-all-strings.golden. The raw ingestor stream the CLI streams\nthrough (MySQL waits, the ๐Ÿ“Š banner, per-validator lines) is the engine's own\nstdout โ€” not CLI copy โ€” so it isn't shown. (`tb ingest` is a hidden deprecated\nalias; `push` is a deprecated alias of the verb.)", []run{ {"tb data ingest # guided ยท tabular classification", tabularIngest}, {"tb data ingest # guided ยท image classification", imageIngest}, @@ -387,7 +387,7 @@ func TestCopyCatalog(t *testing.T) { "00-home.golden": {"tracebloc --help"}, "01-data-ingest.golden": { "Ingest datasets to your secure environment.", // intro - "Step 1 of 5 ยท", "Step 5 of 5 ยท", // the guided questionnaire + "Step 1 of 4 ยท", "Step 4 of 4 ยท", // the guided questionnaire "Review", "Proceed with the ingest?", // review + confirm "Step 1/3", "Step 3/3", "Ingestion summary", "What's next", // the run }, diff --git a/internal/cli/interactive.go b/internal/cli/interactive.go index d21a4cb4..1c56c729 100644 --- a/internal/cli/interactive.go +++ b/internal/cli/interactive.go @@ -123,12 +123,15 @@ func isInteractiveTTY() bool { func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bool) error { prompted := false - // The guided flow is a five-step setup: intent โ†’ name โ†’ path โ†’ task โ†’ - // task-specific details. Each question prints as its own step header - // (PromptStep), with any supporting line beneath it and an answer-only - // prompt (the prompter runs bare โ€” see surveyPrompter). Task-specific - // extras beyond the label (schema, resolution, โ€ฆ) are refinements under - // step 5 and aren't separately numbered. + // The guided flow is a four-step setup: intent โ†’ name โ†’ path โ†’ task. Each + // question prints as its own step header (PromptStep), with any supporting + // line beneath it and an answer-only prompt (the prompter runs bare โ€” see + // surveyPrompter). Everything task-dependent comes AFTER the four steps as + // unnumbered refinements (Section headers): the label column, and per-task + // extras like schema, resolution, keypoints. These aren't numbered because + // which ones apply โ€” and whether any apply at all โ€” depends on the task + // picked at step 4 (self-supervised text has no label and no extras), so a + // fixed "of N" couldn't be honest about them. // // Spacing is uniform (STYLE.md "Guided-prompt spacing"): the header carries // its own leading blank; then one blank line, the optional supporting text, @@ -138,7 +141,7 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // Step 1 โ€” intent: what this data is for. if a.Spec.Intent == "" { - p.PromptStep(1, 5, "Do you want to ingest training or test data?") + p.PromptStep(1, 4, "Do you want to ingest training or test data?") p.Newline() ans, err := pr.Select("Do you want to ingest training or test data?", "which split this data is", []string{"train", "test"}, "train") @@ -152,7 +155,7 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // Step 2 โ€” name. No auto-fill; the character rules surface only if the // name is rejected (see ValidateTableName), so the prompt stays clean. if a.Spec.Table == "" { - p.PromptStep(2, 5, "Please name the dataset.") + p.PromptStep(2, 4, "Please name the dataset.") p.Newline() ans, err := pr.Input("Please name the dataset.", "letters, digits, and underscores; start with a letter or underscore e.g. churn_train", "", @@ -167,7 +170,7 @@ func runInteractive(p *ui.Printer, pr prompter, a *runDataIngestArgs, taskSet bo // Step 3 โ€” path. Show what "file or folder" means per modality, then // detect the family from the layout and echo it back. if a.LocalPath == "" { - p.PromptStep(3, 5, "Where is your data?") + p.PromptStep(3, 4, "Where is your data?") p.Newline() p.Hintf("Give the path to a file or a folder โ€” whichever holds your data:") p.Infof("Tabular one CSV file e.g. ~/data/patients.csv") @@ -310,7 +313,7 @@ func pickTask(p *ui.Printer, pr prompter, fam push.Family) (string, error) { } width += 3 - p.PromptStep(4, 5, "What kind of machine learning task is this data for?") + p.PromptStep(4, 4, "What kind of machine learning task is this data for?") p.Newline() for _, s := range available { p.Para(fmt.Sprintf(" %-*s%s", width, s.ID, s.Blurb)) @@ -352,13 +355,16 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b cat := a.Spec.Category prompted := false - // Label column โ€” the answer the model learns to produce. Skipped for - // self-supervised text (MLM/CLM: the target comes from the text itself, - // there's no label column). Interactive picks from the REAL CSV header - // row so the choice exact-matches a column that exists โ€” killing the - // case-mismatch silent-null-label class (data-ingestors#340) that - // free-typing "Label" against a "label" header would cause. Wording is - // per-task: a class to sort into vs a numeric value to predict (ยง8). + // Label column โ€” the answer the model learns to produce. The first + // task-specific refinement (unnumbered Section, like the extras below), not + // a numbered core step: it's skipped for self-supervised text (MLM/CLM: the + // target comes from the text itself, there's no label column), so numbering + // it "of N" would promise a step that flow never reaches. Interactive picks + // from the REAL CSV header row so the choice exact-matches a column that + // exists โ€” killing the case-mismatch silent-null-label class + // (data-ingestors#340) that free-typing "Label" against a "label" header + // would cause. Wording is per-task: a class to sort into vs a numeric value + // to predict (ยง8). if !push.SelfSupervisedText(cat) && a.Spec.LabelColumn == "" { question := "Which column holds the label?" desc := "The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned" @@ -366,7 +372,7 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b question = "Which column holds the value to predict?" desc = "The number the model learns to predict. e.g. price, age, days_to_event" } - p.PromptStep(5, 5, question) + p.Section(question) p.Newline() p.Hintf("%s", desc) p.Newline() @@ -378,8 +384,8 @@ func promptCategorySpecific(p *ui.Printer, pr prompter, a *runDataIngestArgs) (b prompted = true } - // Task-specific refinements โ€” shown under step 5, each with its own cyan - // header (Section) rather than a step number, since which ones appear + // Further task-specific refinements โ€” like the label above, each gets its + // own Section header rather than a step number, since which ones appear // depends on the task. switch { case push.IsImage(cat): diff --git a/internal/cli/testdata/golden/01-data-ingest.golden b/internal/cli/testdata/golden/01-data-ingest.golden index 797fcd5f..678512a0 100644 --- a/internal/cli/testdata/golden/01-data-ingest.golden +++ b/internal/cli/testdata/golden/01-data-ingest.golden @@ -1,15 +1,17 @@ tb data ingest โ€” stage a dataset into your secure environment ============================================================= What you see when you run `tb data ingest` with no flags: a short intro, a -five-step guided setup, then โ€” after you confirm โ€” the run itself. The setup is +four-step guided setup (intent, name, path, task) then the task-specific +questions, and โ€” after you confirm โ€” the run itself. The setup is driven through the real flow for one task in each family (tabular, image, text) -so the task-specific questions are visible; each question prints as a -`Step N of 5 ยท โ€ฆ` header (task-specific refinements as their own header), the +so the task-specific questions are visible; each core question prints as a +`Step N of 4 ยท โ€ฆ` header, the task-specific ones (the label column, and extras +like resolution or schema) as their own header, the supporting line beneath it, and the `?` line shows your answer. The run (shown once, for tabular) is the three steps + the final summary as the CLI renders them. Passing flags (--as, --task, a path, โ€ฆ) skips the matching questions. The other tasks' extra questions (keypoints, label policy, time column), -self-supervised text (which skips the label step), and the failure-summary +self-supervised text (which skips the label question), and the failure-summary wordings are in zz-all-strings.golden. The raw ingestor stream the CLI streams through (MySQL waits, the ๐Ÿ“Š banner, per-validator lines) is the engine's own stdout โ€” not CLI copy โ€” so it isn't shown. (`tb ingest` is a hidden deprecated @@ -20,15 +22,15 @@ $ tb data ingest # guided ยท tabular classification Ingest datasets to your secure environment. For help: https://docs.tracebloc.io/create-use-case/prepare-dataset - Step 1 of 5 ยท Do you want to ingest training or test data? + Step 1 of 4 ยท Do you want to ingest training or test data? ? train - Step 2 of 5 ยท Please name the dataset. + Step 2 of 4 ยท Please name the dataset. ? hospital_train - Step 3 of 5 ยท Where is your data? + Step 3 of 4 ยท Where is your data? Give the path to a file or a folder โ€” whichever holds your data: ยท Tabular one CSV file e.g. ~/data/patients.csv @@ -38,7 +40,7 @@ $ tb data ingest # guided ยท tabular classification ? ~/data/patients โœ” Found a CSV table โ€” this is tabular data. - Step 4 of 5 ยท What kind of machine learning task is this data for? + Step 4 of 4 ยท What kind of machine learning task is this data for? tabular_classification predict a class from table columns tabular_regression predict a number from table columns @@ -48,7 +50,7 @@ $ tb data ingest # guided ยท tabular classification ? tabular_classification - Step 5 of 5 ยท Which column holds the label? + Which column holds the label? The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned @@ -75,15 +77,15 @@ $ tb data ingest # guided ยท image classification Ingest datasets to your secure environment. For help: https://docs.tracebloc.io/create-use-case/prepare-dataset - Step 1 of 5 ยท Do you want to ingest training or test data? + Step 1 of 4 ยท Do you want to ingest training or test data? ? train - Step 2 of 5 ยท Please name the dataset. + Step 2 of 4 ยท Please name the dataset. ? xray_train - Step 3 of 5 ยท Where is your data? + Step 3 of 4 ยท Where is your data? Give the path to a file or a folder โ€” whichever holds your data: ยท Tabular one CSV file e.g. ~/data/patients.csv @@ -93,7 +95,7 @@ $ tb data ingest # guided ยท image classification ? ~/data/xray โœ” Found labels.csv and an images/ folder โ€” this is image data. - Step 4 of 5 ยท What kind of machine learning task is this data for? + Step 4 of 4 ยท What kind of machine learning task is this data for? image_classification sort images into classes object_detection draw boxes around objects in an image @@ -102,7 +104,7 @@ $ tb data ingest # guided ยท image classification ? image_classification - Step 5 of 5 ยท Which column holds the label? + Which column holds the label? The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned @@ -129,15 +131,15 @@ $ tb data ingest # guided ยท text classification Ingest datasets to your secure environment. For help: https://docs.tracebloc.io/create-use-case/prepare-dataset - Step 1 of 5 ยท Do you want to ingest training or test data? + Step 1 of 4 ยท Do you want to ingest training or test data? ? train - Step 2 of 5 ยท Please name the dataset. + Step 2 of 4 ยท Please name the dataset. ? reviews_train - Step 3 of 5 ยท Where is your data? + Step 3 of 4 ยท Where is your data? Give the path to a file or a folder โ€” whichever holds your data: ยท Tabular one CSV file e.g. ~/data/patients.csv @@ -147,7 +149,7 @@ $ tb data ingest # guided ยท text classification ? ~/data/reviews โœ” Found labels.csv and a texts/ folder โ€” this looks like text data. - Step 4 of 5 ยท What kind of machine learning task is this data for? + Step 4 of 4 ยท What kind of machine learning task is this data for? text_classification sort text snippets into classes masked_language_modeling predict masked-out words โ€” no labels needed @@ -159,7 +161,7 @@ $ tb data ingest # guided ยท text classification ? text_classification - Step 5 of 5 ยท Which column holds the label? + Which column holds the label? The answer the model learns to produce โ€” for classification, the class. e.g. diagnosis, churned