Uh oh!
There was an error while loading. Please reload this page.
feat(data ingest): flexible file-or-folder input + path fixes (#181) - #202
Merged
Conversation
…181) RFC-0002 phase 3: flexible file-or-folder input, clearer instructions, and two path bugs. - Accept a bare .csv for the tabular / time-series family. DiscoverTabular now takes either a directory (its existing exactly-one-CSV rule) or a bare .csv file — both resolve to the SAME staged layout (the CSV staged as the one labels.csv under the dataset), so the ingestor's contract is unchanged. This is a CLI-side input convenience only; no data-ingestors change. Media/label families (image, text) stay directory-only and error clearly ("… is not a directory"). - Restore the sniff + copy #180b softened: SniffFamily is confident-tabular for a bare .csv again (mirroring the now file-or-folder-capable walk), and the interactive path prompt goes back to "Where is your data? (file or folder)" with a matching hint. - Per-family layout help: the ingest Long help + the data group help now show tabular (a .csv OR a folder with one .csv), image (labels.csv + images/), and text (labels.csv + texts/) instead of only image. - Rename the positional arg <local-path> → <dataset> in Use/help. - expandHome now expands "~user" / "~user/…" via user.Lookup, not just "~" / "~/…"; an unknown/unlookupable user is left literal so the path check reports it plainly. - Check path existence BEFORE spec/schema/family validation, so a typo'd path fails on the path (exit 3, "no such file or directory") instead of a confusing downstream error (e.g. the task gate). Tests: bare .csv discovers + stages identically to a one-CSV dir; bare non-.csv and image/text bare files rejected; sniff confident-tabular on a bare .csv; "file or folder" copy restored; ~user expansion; nonexistent path beats the task gate with exit 3; <dataset> arg name. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s + path-check ordering Three verified review findings on the #181 flexible-input work: - expandHome mangled a ~user whose passwd home field is blank: user.Lookup succeeded with an empty HomeDir, so filepath.Join("", rest) produced a relative path instead of the documented clear literal-path error. Guard u.HomeDir == "" the same way the current-user branch already guards an empty $HOME. - The "path existence FIRST" invariant held only on the flag-only route. In guided mode runInteractive sniffed the family and previewed the label header at a non-existent path before the 0b guard ever fired, so the user answered the whole questionnaire against a bad path. Extract the guard into statDatasetPath and run it in runInteractive before the family sniff; a typed exitError from a guided step now propagates unwrapped (clean message, not "interactive setup: ..."). - expandHome (data ingest) and cluster.expandPath had drifted: #181 added ~user resolution to only one, so `--kubeconfig ~alice/.kube/config` silently treated alice as a subdir of the current user's home. Consolidate both into a shared internal/pathutil.ExpandHome (the promote-to-pathutil plan the old comments kept deferring) so ~-expansion is identical across every subcommand. Coverage floor unaffected (internal/cli 73.6% >= 68%). New pathutil package carries its own contract tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodkaforce-pushed
the
feat/181-flexible-input
branch
from
July 9, 2026 19:09
e01f011 to
c946085CompareThere was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c946085. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
…st key Review polish on the #181 flexible-input work: - SniffFamily no longer confidently sniffs a symlinked .csv as tabular. os.Stat follows the link, but DiscoverTabular Lstats + rejectSymlinks the CSV, so a symlinked .csv is a layout the walk REFUSES. The sniff would lock the guided flow to tabular and then hard-fail on the walk's symlink guard — breaking SniffFamily's "never claims more than the matching Discover* would accept" contract. The bare-file branch now Lstats and stays ambiguous for a symlink, so sniff and walk agree. - Extract isCSV(name) and route all four .csv-extension checks through it (findSingleCSV, DiscoverTabular's bare-file branch, and both SniffFamily sites). The sniff and walk bare-file rules were copy-pasted case-fold checks the comments insist must stay in lockstep; one helper removes the drift risk. - Fix a stale fakePrompter key in TestRunInteractive_TrimsPath: the path prompt was renamed to "(file or folder)" everywhere except this one answer-map key, which still read "(the folder holding it)", so the fake returned "" and the test failed with "a dataset path is required". Tests: new SniffFamily case asserts sniff + walk both refuse a symlinked .csv. go build ./... && go test ./... && gofmt -l . && go vet ./... green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saadqbal
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
RFC-0002 phase 3: let
data ingesttake a bare.csvas easily as a folder, make the per-family layout instructions concrete, and fix two~/path-handling bugs. Data stays on-prem throughout — this only changes what the CLI accepts as input and how cleanly it fails on a typo; the ingestor contract is untouched.Closes#181.
What changed
.csvinput for tabular / time-series.DiscoverTabularnow accepts either a directory (its existing exactly-one-CSV rule) or a bare.csvfile. Both resolve to the same staged layout — the CSV staged as the onelabels.csvunder the dataset — so the ingestor sees an identical contract. Media/label families (image, text) stay directory-only and reject a bare file with a clear message.SniffFamilyis confident-tabular for a bare.csvagain (mirroring the now file-or-folder-capable walk), and the interactive prompt is back to "Where is your data? (file or folder)".Longhelp and thedatagroup help now spell out tabular (a.csvor a folder with one.csv), image (labels.csv+images/), and text (labels.csv+texts/) instead of only image. Positional arg renamed<local-path>to<dataset>.~userexpansion + shared expander.expandHomeresolves~user/~user/…viauser.Lookup, not just~/~/…. Both this andcluster.expandPathnow delegate to a new leafinternal/pathutil.ExpandHome, so~-expansion (incl.~alice/.kube/config) is identical across every subcommand.Review-finding fixes (second commit)
Three verified findings from review of the feature commit:
expandHomeno longer mangles a~userwhose passwd home field is blank (user.Lookupsucceeds with an emptyHomeDir) — it returns the literal path, matching the documented contract and the current-user branch's existing empty-$HOMEguard.runInteractivesniffed the family and previewed the label header at a non-existent path before the guard fired, so the user answered the whole questionnaire against a bad path. The guard is extracted tostatDatasetPathand runs before the family sniff; a typedexitErrorfrom a guided step propagates unwrapped (clean message, not "interactive setup: …").expandHomeandcluster.expandPathhad drifted (~useron only one) — consolidated intopathutil.ExpandHome, the promote-to-pathutil plan the old comments kept deferring.Test plan
go build ./... && go test ./... && gofmt -l . && go vet ./...— all green locally..csvdiscovers + stages identically to a one-CSV dir; bare non-.csvand image/text bare files rejected; sniff confident-tabular on a bare.csv; "file or folder" copy restored;~userexpansion; nonexistent path beats the task gate with exit 3 on both routes;<dataset>arg name; newinternal/pathutilcontract tests.internal/cli73.6% ≥ 68%,internal/submit74.9% ≥ 72%.tracebloc data ingest /nope/typoed-path.csv --no-input …exits 3 withno such file or directory: "/nope/typoed-path.csv" — check the path to your dataset.Stacked on #198 — merge after it
Base is
feat/180b-data-first-inversion(PR #198), notdevelop. Merge #198 first, then this retargets todevelopautomatically.Decisions to ratify
.csvis staged as the onelabels.csvunder the dataset — byte-identical to the one-CSV-directory case — so no ingestor-side change is needed. The pre-flight "root" stays a directory (the file's parent) so the summary reads the same for both shapes. Confirm this CLI-side-convenience framing is what we want long-term (vs. teaching the ingestor about bare files).~userresolution is best-effort. An unknown/unlookupable user (incl. a static CGO-less build that can't read/etc/passwd) or a blank passwd home leaves the literal path so the existence check surfaces it plainly. Confirm we're fine with~usersilently not-expanding on a CGO-less binary rather than erroring.pathutil.ExpandHome.cluster.expandPathnow also resolves~user(previously it treated~alice/…as a subdir of the current home). This is a behavior change for--kubeconfig ~alice/…— intended, but worth a nod.Note
Low Risk
Changes are localized to CLI input validation, path expansion, and tabular discovery/staging parity; ingestor wire format is unchanged, with broad test coverage for ordering and layout edge cases.
Overview
data ingestnow accepts tabular/time-series data as a bare.csvfile or a folder with exactly one CSV. Staging still produces the samelabels.csvlayout as before, so the ingestor contract is unchanged; image and text tasks remain directory-only.Path handling is tightened: missing paths fail first with exit 3 (
statDatasetPath) before task validation or guided sniffing, and interactiveexitErrors from that guard propagate without aninteractive setup:wrapper.~userpaths resolve via sharedinternal/pathutil.ExpandHome, used by both data ingest and--kubeconfig(replacing duplicate expanders incliandcluster).UX/docs: positional arg renamed to
<dataset>, help describes tabular vs image vs text layouts, and the guided path prompt again says “file or folder.”SniffFamilytreats a lone.csvas confident tabular (but not symlinked CSVs), aligned withDiscoverTabularand sharedisCSV.Reviewed by Cursor Bugbot for commit b0d0244. Bugbot is set up for automated code reviews on this repo. Configure here.