Describe the feature
Add a first-class Rust API for batch incremental reads over a snapshot range, starting with Delta mode and Auto mode selection.
Motivation
Right now you can plan a full table scan for a snapshot, but there isn't a clean public API for "give me files between snapshot A (exclusive) and B (inclusive)" — which is the building block for incremental ETL and later SQL helpers.
Proposed scope (this issue)
IncrementalScanMode / IncrementalScan / IncrementalPlan- Snapshot range checks (start exclusive, end inclusive; empty + out-of-range)
- Delta planning over APPEND snapshots in the range
- Auto:
changelog-producer=none → Delta; otherwise resolve to Changelog (implementation of Changelog itself can be a follow-up) - Reuse existing
TableScan / TableRead instead of reimplementing manifest readers
Follow-ups (separate issues/PRs): reading existing changelog manifests, Diff mode, DataFusion TVF.
Additional context
I already have a draft implementation I'm cleaning up against current main.
Describe the feature
Add a first-class Rust API for batch incremental reads over a snapshot range, starting with Delta mode and Auto mode selection.
Motivation
Right now you can plan a full table scan for a snapshot, but there isn't a clean public API for "give me files between snapshot A (exclusive) and B (inclusive)" — which is the building block for incremental ETL and later SQL helpers.
Proposed scope (this issue)
IncrementalScanMode/IncrementalScan/IncrementalPlanchangelog-producer=none→ Delta; otherwise resolve to Changelog (implementation of Changelog itself can be a follow-up)TableScan/TableReadinstead of reimplementing manifest readersFollow-ups (separate issues/PRs): reading existing changelog manifests, Diff mode, DataFusion TVF.
Additional context
I already have a draft implementation I'm cleaning up against current
main.