Skip to content

data ingest: mis-cased media folder + labels.csv sniffs as confident tabular → silent mis-ingestion #203

Description

@saadqbal

Found while reviewing #198 / following up on #180b.

Problem

When a dataset directory has labels.csv plus a mis-cased media folder (e.g. Images/, Texts/, Sequences/ instead of the literal images//texts//sequences/), SniffFamily (internal/push/preview.go) reports a confident tabular layout:

  • the subdir switch is case-sensitive, so ImagesimageshasImages stays false;
  • labels.csv counts toward csvCount (== 1);
  • so it hits the !hasImages && !hasText && csvCount == 1 branch → confident tabular, echoes "Found a CSV table — this is tabular data."

The guided data-first flow then skips the family question (resolveFamily returns on Confident) and offers only tabular tasks. The user picks a tabular task, DiscoverTabular runs, findSingleCSVignores subdirectories and finds the one CSV (labels.csv) → it succeeds. Result: an image/text dataset is silently ingested as a standalone table, the media files dropped, no error.

Why the existing test misses it

TestSniffFamily has a mis-cased Images/ is not confident image case — but it only asserts the sniff isn't confident image. It never checks it isn't confidently tabular, which is the actual footgun.

Proposed fix

When the lone CSV sits next to a subdirectory whose name matches a marker case-insensitively but not exactly (a likely mis-cased media folder), stay ambiguous and ask the family question instead of confidently claiming tabular. Narrow — an unrelated subdir (backup/, raw/) still sniffs confident tabular, matching DiscoverTabular, which ignores it.

Acceptance

  • labels.csv + Images/ (or Texts//Sequences/) → sniff NOT confident (asks family).
  • labels.csv + images/ (exact) → still confident image.
  • single CSV + unrelated subdir → still confident tabular.
  • Regression test pinning the mis-cased-tabular case.

Rolls out of the #180 / #198 line.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions