chore(schema): bump data-ingestors pin to 8f89aec — adopt di#365, audit di#358 (cli#286)

[LukasWodka]

Fixes#286. Part of epic tracebloc/backend#1106 (WS-C.1).

What

Bumps scripts/.data-ingestors-ref from 7b4ecac (the di#359 merge) to data-ingestors develop HEAD 8f89aec (the di#368 merge), adopting the 6 intervening commits:

upstream	what	CLI impact
di#358	require-and-enforce semseg mask_id validator (backend#816)	audited — CheckMaskIdColumn already mirrors it exactly (see below)
di#365	min_size values validated as positive ints at construction (#348)	adopted — no code change needed, contract documented + test pinned
di#369	orphan-row reconcile before retry ingest	write path only — nothing the CLI previews
di#366	content-comparison e2e	tests only
di#368	ingest correlation id	write path only
di#362	0.7.0 version bump	none

Sync scripts re-run (pin-bump doctrine, backend#1009)

• scripts/sync-schema.sh — ingest.v1.json + layout.v1.jsonbyte-identical at the new pin (no upstream schema change in the range), so no diff.
• scripts/sync-validator-goldens.sh — regenerated against the REAL validators at 8f89aec (local data-ingestors checkout at that SHA, its venv: pandas 3.0.3 + Pillow 12.2.0). Goldens byte-identical — no verdict or value-level drift for the existing corpus: di#358 only adds the semseg validator (no semseg parity cases yet — those land with the WS-C corpus ticket), and di#365 only changes verdicts for malformed min_size shapes the corpus never emits. --check mode passes for both scripts.

di#365 adoption (ValidateImages/spec)

Upstream now rejects a min_size whose sides aren't positive integers at validator construction (config error) instead of mislabeling it a corrupt image mid-scan. The CLI can't emit those shapes: ParseMinSize (WxH grammar, Atoi, both sides > 0) is the equivalent gate at flag-parse time, and SpecArgs.MinSize is []int by type. Adopted as doc blocks on SpecArgs.MinSize + ValidateImages, plus two new TestParseMinSize cases pinning the float boundary (16.5x32, 32.0x32 → rejected). Note the CLI is deliberately stricter than upstream on integer-valued floats (32.0): upstream coerces, the flag grammar rejects — safe direction (the user just types 32).

di#358 audit (CheckMaskIdColumn vs merged MaskIdColumnValidator)

Read the full upstream validator at 8f89aec and compared point for point (the #239/#240 padded-NA null-set bug being the cautionary precedent). No divergence found:

• exact-lowercase mask_id required after whitespace strip (ReadCSVHeader trims like CSVIngestor's columns.str.strip())
• case/whitespace variant → rename hint (matchColumnIndex ≙ BaseValidator._match_column)
• empty scan matches the RAW untrimmed cell against NA_SENTINELS (naSentinels verified byte-identical, 12 tokens) plus whitespace-only/missing — a padded " NULL " counts as populated on both sides
• mid-read error fails closed on both sides
• duplicate stripped-equal headers: both sides inspect the FIRST exact-match column
• the validator's schema-declaration half is satisfied by construction on the CLI side (buildImage always declares {"mask_id": "VARCHAR(255)"}); its csv_options dialect threading is N/A (the CLI stages comma-separated UTF-8 manifests, no custom dialect)

Audit recorded as a doc block on CheckMaskIdColumn.

Verification (what actually ran locally)

• bash scripts/sync-schema.sh + --check — no drift
• bash scripts/sync-validator-goldens.sh + --check against a checkout at 8f89aec — regenerated, byte-identical, check green
• gofmt -l internal/push/ clean; go build ./... green
• go test ./internal/push/ green (3.5s)

One reality note: cli#286 / the epic describe data-ingestors as private; it is currently public (gh repo view → PUBLIC) — which is also why the unauthenticated raw.githubusercontent.com curl in sync-schema.sh works.

🤖 Generated with Claude Code

---

Note

Low Risk
Pin bump plus comments and two test cases; existing ParseMinSize/CheckMaskIdColumn behavior is unchanged, with sync scripts reported clean.

Overview
Bumps the pinned data-ingestors commit in scripts/.data-ingestors-ref from 7b4ecac to 8f89aec, adopting upstream changes including semseg mask_id enforcement (di#358), min_size positive-integer validation at construction (di#365), orphan-row reconcile, correlation id, and the 0.7.0 version bump—without regenerating schema or validator goldens (reported byte-identical at the new pin).

di#365 is documented on SpecArgs.MinSize, ValidateImages, and enforced in tests: TestParseMinSize now rejects float forms (16.5x32, 32.0x32), noting the CLI flag grammar is stricter than upstream on integer-valued floats.

di#358 parity for CheckMaskIdColumn is recorded in a comment block on CheckMaskIdColumn in preflight.go (no logic changes in this diff).

^{Reviewed by Cursor Bugbot for commit bc0a902. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

👋 Heads-up — Code review queue is at 42 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• averaging-service#181 — feat(weights): normalize-on-read — mixed cycles average instead of rejecting (SEC-03 §8 step 2) · author: @shujaatTracebloc · no reviewer assigned
• averaging-service#182 — feat(weights): averaging writes SafeTensors (TF + PyTorch) — SEC-03 §8 step 4 · author: @shujaatTracebloc · no reviewer assigned
• backend#1079 — feat(global_meta): edge dataset_meta exposure + attributes contract at ingest (#924 G4a) · author: @divyasinghds · no reviewer assigned
• backend#1086 — docs(rfc): SafeTensors weight-format migration — SEC-03 Phase 1 (RFC 0004) · author: @shujaatTracebloc · no reviewer assigned
• backend#1093 — chore(deps): bump django from 5.2.14 to 5.2.15 · author: @dependabot · no reviewer assigned
• backend#1095 — feat(experiment): configurable preprocessing knobs incl. tabular scaler — RFC 0003 L1 + L1b (#1094) · author: @LukasWodka · no reviewer assigned
• backend#1100 — fix(boot): pin SDK install to tracebloc==0.11.2, drop 404 dev line (#1098) · author: @LukasWodka · reviewer: @saqlainsyed007
• backend#1105 — perf(api): query micro-fixes — notifications N+1, cached data_scientist, composite index, sampling (#975) · author: @aptracebloc · no reviewer assigned
• cli#266 — main - > enhance CLI features and tests · author: @saadqbal · no reviewer assigned
• cli#278 — fix(deps): toolchain go1.26.5 + x/net v0.57.0 — clear 6 reachable vulns; govulncheck CI gate · author: @LukasWodka · reviewer: @saadqbal

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

[LukasWodka]

@BugBot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bc0a902. Configure here.}

[saadqbal]

👍 Careful pin bump — Schema drift check is green (confirms the 'no schema/verdict changes' claim across 7b4ecac→8f89aec), the CLI already satisfies the new di#358/di#365 value contracts (ParseMinSize rejects the floats, and the two new cases pin it), and the CheckMaskIdColumn parity audit matches what I'd independently confirmed against di#358. Good audit trail in the ref file.