Uh oh!
There was an error while loading. Please reload this page.
feat: Add MERGE INTO support to FFI TableProvider - #23867
Draft
wirybeaver wants to merge 6 commits into
Draft
Conversation
wirybeaver
commented
Jul 24, 2026
ContributorAuthor
Maintainer note: this PR extends the public |
Add merge_into async method to TableProvider trait for MERGE INTO DML support. The method accepts: - source: ExecutionPlan representing the USING clause - on: Expr representing the ON join condition - clauses: Vec<MergeIntoClause> for WHEN MATCHED/NOT MATCHED actions Default implementation returns not_impl_err for tables that don't support MERGE INTO operations.
Implement merge_to_plan and merge_clause_to_plan in SQL planner: - Parse Statement::Merge into LogicalPlan::Dml with WriteOp::MergeInto - Resolve target table and plan source (USING clause) as LogicalPlan - Build combined schema for target + source to resolve ON and WHEN expressions - Convert ON condition and WHEN clauses to DataFusion Expr - Handle UPDATE, INSERT, and DELETE actions in WHEN clauses Add physical planner dispatch for WriteOp::MergeInto: - Use source_as_provider() to recover the TableProvider from the TableSource - Extract source ExecutionPlan from children - Call TableProvider::merge_into with source plan, ON condition, and clauses - Wrap errors with MERGE INTO operation context Wire MergeInto's expressions through LogicalPlan tree-traversal so optimizers can rewrite them: add MergeIntoOp::exprs() (stable iteration order: on, then per-clause predicate + action value Exprs) and MergeIntoOp::with_new_exprs() to rebuild the op from a transformed expr vector. Branch LogicalPlan::apply_expressions, map_expressions, and with_new_exprs on WriteOp::MergeInto to use these helpers; other WriteOp variants continue to expose no expressions as before.
wirybeaverforce-pushed
the
feature/mergeinto-ffi
branch
from
July 24, 2026 15:32
aa088b2 to
6f18ee8CompareThank you for opening this pull request! Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch). Details |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@## main #23867 +/- ##
========================================
Coverage 80.71% 80.72% ========================================
Files 1090 1090 Lines 370339 371121 +782 Branches 370339 371121 +782 ========================================
+ Hits 298926 299580 +654 - Misses 53603 53662 +59 - Partials 17810 17879 +69 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
wirybeaverforce-pushed
the
feature/mergeinto-ffi
branch
from
July 24, 2026 17:05
6f18ee8 to
ba1d668Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This draft depends on #22988. Its branch is stacked on that PR, so GitHub will temporarily show the parent commits too. I will rebase it onto
mainafter #22988 merges.Which issue does this PR close?
Follow-up to #22988.
Rationale for this change
#22988 adds
TableProvider::merge_into. Without matching FFI plumbing, a foreignTableProvidercannot expose an implementation of that method: the consumer side would use the trait's default unsupported behavior instead.What changes are included in this PR?
merge_intofunction pointer toFFI_TableProviderand dispatch it fromForeignTableProvider.MergeIntoOpwith the logical extension codec and pass the source as anFFI_ExecutionPlan.MergeIntoOpat the FFI boundary.Are these changes tested?
cargo fmt --allcargo clippy --all-targets --all-features -- -D warningscargo test -p datafusion-fficargo test -p datafusion-ffi --features integration-testsAGENTS.mdon the identical combined tree before the splitAre there any user-facing changes?
Yes. This adds a function-pointer field to the public
#[repr(C)] FFI_TableProviderlayout, so it is an ABI change and requires theapi changelabel. FFI consumers must rebuild against the corresponding DataFusion major version.