fix(preflight): resolve labels.csv filename by column name, not position

[LukasWodka]

Summary

CrossCheckLabels (the image_classification preflight that verifies each labels.csv row's image exists under images/) read the filename from rec[0] — the first column, positionally. But the ingestor reads it by name: record.get("filename") over the header-keyed record (file_transfer.py / record_processor.py), order-independent, and every template's labels.csv names the column filename.

So a label,filename header (filename present but not first — a layout the cluster ingests cleanly) made the CLI read the label value ("cat") as the filename, miss every row, and reject with exit 3 ("N labels.csv row(s) reference images that aren't in images/") — a false reject whose message also misleadingly lists label values as missing images. Violated the "never stricter than the ingestor" preflight contract.

(Found by an adversarial bug-hunt; verified against the ingestor's actual read path + templates.)

Fix

Resolve the filename column by name — exact filename first, then case-insensitive + whitespace-trimmed (the ingestor's _match_column rule) — and read that column's index. Falls back to column 0 when there's no filename-ish column (a malformed layout the ingestor fails on regardless — not this check's job to diagnose); short/ragged rows without that cell are skipped.

Tests

• Existing test's unrealistic image_id,label fixture corrected to the real filename,label.
• TestCrossCheckLabels_FilenameColumnNotFirst — label,Filename (not first, mixed case) resolves correctly and does not read labels as filenames (this fails against the old rec[0]).
• TestFilenameColIndex — first / not-first / case / whitespace / no-filename fallback.

Full go test ./... green; gofmt -s / errcheck / ineffassign / misspell clean.

Type

Bug fix (preflight false-reject; same parity class as #340) · cli · HIGH.

---

Note

Low Risk
Scoped to CLI preflight parity for CrossCheckLabels; behavior is tightened to match ingestor rules and is covered by new unit tests, with no upload or cluster path changes.

Overview
Fixes false preflight rejects for image_classification when labels.csv does not put the image key in the first column. CrossCheckLabels used to read rec[0]; it now picks the image column the same way the ingestor does: filename by name (exact, then case/whitespace-insensitive), else data_id, with column 0 only when neither exists.

That restores the never stricter than the ingestor contract for layouts like label,filename or label,data_id, where the old check treated label values as filenames and failed with exit 3. Short rows without a filename cell are skipped instead of mis-read.

Tests cover non-first filename, data_id fallback, and imageFileColIndex header matching; the main fixture uses a realistic filename,label header.

^{Reviewed by Cursor Bugbot for commit 21258c9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[saadqbal]

Good fix — reading by column name is right. One gap though: the ingestor resolves filename OR data_id, and we only handle filename. Details inline.

[LukasWodka]

Thanks @saadqbal — fixed in 21258c9.

filename-else-data_id (the gap).imageFileColIndex (renamed from filenameColIndex) now mirrors the ingestor's precedence: filename if present, else data_id, each matched exact-then-case-insensitive-trimmed, filename winning when both resolve. So a valid label,data_id CSV resolves the data_id column instead of falling back to index 0 and false-rejecting. Fallback to 0 stays only for the neither-column case (which the ingestor rejects at validate_data regardless — as you noted, fine to leave). Added TestCrossCheckLabels_DataIDColumn (the label,data_id regression) plus data_id / Data_ID / filename-wins cases to the index test.

Truncated-row skip (preflight.go:290) — left, on purpose. A row too short to reach the image-file cell references no image, so there's nothing for CrossCheckLabels (an image-existence check) to verify. The record's own invalidity is IngestableRecordsValidator's concern — the ingestor pads it to NaN and fails that record with a clear per-record error (the exit-9 path), which is a different check than "do the referenced images exist." Flagging it here would mean inventing a synthetic entry in the filename list or a signature change, for a ragged-CSV edge the cluster already reports cleanly. Happy to add a ragged-row guard if you'd rather it fail locally too — your call.

go build / go test ./... / gofmt / go vet green.