Skip to content

data ingest: staged source copy is never reclaimed after a successful ingest (~2x PVC usage) #166

Description

@LukasWodka

Found by the ingest-flow UX audit (2026-07-07), verified against code.

Problem

After a successfultracebloc data ingest, the staged source copy at SharedRoot/.tracebloc-staging/<table> is never removed. The CLI streams a full copy of the dataset there; the in-cluster ingestor copies it (shutil.copy, data-ingestors/tracebloc_ingestor/file_transfer.py:61 — never a move) into the final table dir SharedRoot/<table>. So on success both copies live on the shared PVC.

The staged copy is only ever removed by push.Teardown — called from data delete and the --overwrite pre-stage branch. There is no cleanup on the normal success path (internal/cli/data.go, internal/push/teardown.go:42).

Impact

Every successful ingest of a file-bearing dataset (image / detection / segmentation) silently leaves ~2× the dataset on the shared PVC, indefinitely. Users hit no space left on device on later ingests with no signal that a duplicate source copy is the cause and no "reclaim staging only" path. (Pure-tabular datasets carry no sidecar files under the staging prefix, so the doubling is smaller there — but the leak is the same.)

Fix (PR incoming)

On a clean success only (classifyPushOutcome == "succeeded"), best-effort rm -rf of onlyStagedPrefix(table) — never the final table dir, never the MySQL table — via the same ephemeral stage-identity pod Teardown already uses (it owns the uid-65532 staging files). Gated so it never fires on --detach (Job still reading the source), completed_with_failures, or any failure. A failed reclaim logs a warning and does not fail the ingest.

Longer term, the cleaner fix is server-side: have the ingestor move-not-copy (or rm its SRC_PATH after a verified load) so no second pod is needed — will file a data-ingestors follow-up.

Parent QA epic: #67.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions