Skip to content

data ingest: add --append for existing destination tables (needs server-side support) #156

Description

@LukasWodka

Spun out of #70 when it closed. The silent-duplication bug is fixed on develop (#149): re-ingesting into an existing table now stops with exit 6, and --overwrite (teardown + re-ingest) is shipped. --append was deliberately NOT shipped because it can't be done client-side:

  1. The ingestor has no append modeDuplicateValidator fails on a non-empty destination directory, and the staging/ingest path assumes a fresh table. Appending needs defined server-side semantics first (duplicate-key handling: skip / replace / error).
  2. No Job↔table link exists, so the CLI can't guard against appending while another ingest run into the same table is still in flight — a required safety rail before append is offered.

Scope: server-side append semantics in tracebloc/data-ingestors, then the CLI flag + preflight wiring. Until then, exit 6 + --overwrite cover the known workflows.

Parent QA epic: #67. Context: #70, PR #149.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions