feat(data ingest): wire semantic_segmentation (RFC-0002 phase 4, #182)

[LukasWodka]

Summary

Wires semantic_segmentation into data ingest — the last CLI-pending task, closing out #182 / RFC-0002 phase 4. semseg is an existing, fully-registered category (backend/engine/SDK/zoo/ingestor all have it — the category blueprint §4.2 names it the donor for "CSV link-column" file-bearing categories); this is only the CLI-push slice (§3.2). Its blockers landed: di#358 shipped the ingestor's mask_id require-and-enforce (v0.7.0) and backend#816 closed.

What changed

• category.go — semantic_segmentation → CLISupported: true (all 16 tasks now wired).
• DiscoverSemanticSegmentation — labels.csv + images/ + masks/*.png (required), mirroring DiscoverObjectDetection.
• spec.go — emits masks: + declares schema: {mask_id: VARCHAR(255)} so the ingestor storesmask_id (the backend#816 crux — an undeclared one is dropped and training can't find masks), matching the canonical example manifest.
• Preflight mirrors the ingestor's semseg validators (modalities/validators.py):
  • CheckMaskPairing — images↔masks by the _mask suffix (FilePairingValidator, feat(data ingest): plain-language copy + hide k8s ceremony + progress on every wait (#179) #196).
  • CheckMaskIdColumn — mask_id declared exact-lowercase + populated every row, NA-sentinel-aware (MaskIdColumnValidator).
  • CrossCheckLabels — labels.csv rows ↔ images/ (the same fail-fast image_classification runs).
• data.go — friendly "needs --label-column" pre-check extended to semseg (the schema's allOf requires label for it, but the per-image label reads as vestigial beside the masks, so the flag is easy to forget → without this, an opaque schema dump).
• No vendored-schema change (the contract already declared semseg's masks sidecar — drift check green).

Faithfulness (RFC-0002 Principle 6 — mirror the ingestor, never over-reject)

Reviewed adversarially (a 3-angle /code-review + a walk of the blueprint §3.2/§7). Findings fixed:

• Padded-NA over-rejection (flagged by all 3 finders): the empty-mask_id scan now tests the raw cell against the NA set + a separate whitespace-only clause, so a padded token like " NULL " (a real value in-cluster — pandas keeps the spaces) isn't falsely flagged. This is the documented cli#218/TSC preflight parity: sequence_id null set uses coercion sentinels, not pandas' default NA — 'none' ids over-reject, '#NA' under-rejects #239 parity trap.
• Dotfile over-rejection: CheckMaskPairing skips hidden files (macOS AppleDouble ._x), mirroring FilePairingValidator._stems, so a stray ._x.png can't fake a mismatch.
• Fail-fast gap: added CrossCheckLabels (rows→images) to the semseg preflight.

Deliberately not added (would over-reject / out of scope): CheckLabelColumn for semseg (the ingestor omits LabelColumnValidator — labels come from the masks).

Deferred (noted for follow-up, not blockers)

• Mask resolution / PNG-readability preview — the ingestor runs a 2nd ImageResolutionValidator + FileTypeValidator on masks/; the CLI previews the high-value checks and leaves these to the in-cluster pass (an under-rejection, not over-rejection).
• CheckAnnotationPairing (object_detection) has the same pre-existing dotfile gap; extending the #214 friendly-label guard to all label-requiring image categories would help imgc/OD/keypoint too.
• Sidecar-validator parity cases (testdata/parity/) — object_detection (the file-bearing donor) has none either; covered here by unit tests.

Test plan

• make ci green (build, -race tests, gofmt, lint, schema drift).
• New unit tests, mutation-proven: discovery (valid / missing masks), CheckMaskPairing (paired / image-without-mask / mask-without-image / non-conforming / hidden-file), CheckMaskIdColumn (valid / missing / wrong-case / empty / NA / padded-NA), TestBuild_SemanticSegmentation (masks + schema{mask_id} emission + schema validation — guards the #816 crux), TestSemsegSidecarMirrorsContract (pins the hardcoded masks/*.png/mask_id to the vendored contract).
• Updated the registry / picker / gate tests that pinned "semseg pending" (all 16 categories CLI-supported now).

Checklist

• Targets develop
• make ci green
• Mirrors the ingestor (blueprint §3.2 CLI touchpoints all covered; faithful to the semseg validators)
• Tests added + mutation-checked
• Human review

Closes#182.

🤖 Generated with Claude Code

---

Note

Medium Risk
Adds a new file-bearing ingest path and CSV/mask contract checks; behavior is scoped to one task and heavily tested for ingestor parity, but mistakes could still cause late cluster failures or incorrect staging.

Overview
Enables data ingest for semantic_segmentation, completing CLI support for all 16 ingest schema tasks. The category registry marks it CLI-supported (no pending note); docs and gates now describe full task coverage instead of semseg as blocked.

Local layout follows object detection: labels.csv + images/ plus required masks/*.png, discovered via DiscoverSemanticSegmentation and staged through the existing sidecar/tar path. Ingest specs add masks/ and declare schema: { mask_id: VARCHAR(255) } so the ingestor stores the mask link column.

Preflight mirrors ingestor validators: CheckMaskPairing (<image>_mask.png), CheckMaskIdColumn (exact mask_id, row population, NA-sentinel parity), and CrossCheckLabels for rows vs images—without CheckLabelColumn, which the ingestor omits for semseg. --label-column gets the same friendly missing-label guard as tabular tasks; the pre-flight summary shows mask file counts.

Tests drop “semseg pending” expectations, add discovery/preflight/spec/contract coverage, and assert the image task picker lists semantic segmentation under “Available now.”

^{Reviewed by Cursor Bugbot for commit 51b61d3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 51b61d3. Configure here.}

[saadqbal]

Solid PR 👍 The padded-NA handling in CheckMaskIdColumn (raw cell vs naSentinels before trim) is exactly right — that's the parity trap that bit us on #218/#239. Mask pairing + the mask_id schema declaration all check out against the vendored contract. LGTM.