feat: add basic MemTable MERGE INTO support

[wirybeaver]

Which issue does this PR close?

Follow-up to the MERGE INTO review in #22988. No separate issue.

Rationale for this change

MERGE INTO could be planned, but the built-in in-memory table provider could not execute it, and EXPLAIN only showed op=[MergeInto] without the ON condition or WHEN actions. This made the new SQL surface hard to exercise end-to-end with the default provider.

What changes are included in this PR?

• Adds display helpers for MERGE INTO operations, clauses, and actions, and includes ON / WHEN ... THEN ... details in text and JSON logical plan output.
• Implements basic MemTableMERGE INTO execution for matched update/delete, not-matched insert, and not-matched-by-source update/delete.
• Handles first matching clause semantics, NULL predicates as false, duplicate source matches for a target row as an error before mutation, insert column subsets with defaults or typed NULLs, expression casts to target column types, affected-row counts, and sort-order reset.
• Documents MERGE INTO syntax, provider caveats, and currently unsupported planner syntax.
• Extends sqllogictests and Rust tests for planning, display, execution, NULL/no-op predicates, defaults, and duplicate-match errors.

Are these changes tested?

Yes:

• cargo fmt --all
• cargo test -p datafusion-expr
• cargo test -p datafusion-catalog
• cargo test -p datafusion --test core_integration merge_into -- --nocapture
• cargo test -p datafusion-sql plan_merge_into_canonicalizes_qualifiers_and_preserves_quoted_columns -- --nocapture
• cargo test --profile=ci --test sqllogictests -- merge_into.slt --nocapture
• ./ci/scripts/doc_prettier_check.sh --write --allow-dirty
• cargo clippy --all-targets --all-features -- -D warnings

Note: the plan mentioned cargo test -p datafusion --test sql, but this repository does not currently have a sql test target; the relevant MERGE SQL tests live under core_integration.

Are there any user-facing changes?

Yes. MemTable now supports basic MERGE INTO, and EXPLAIN output for MERGE INTO includes the merge condition and clauses. There are no public API signature changes.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 80.55010% with 99 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.23%. Comparing base (bbf739a) to head (24aa97f).

Files with missing lines	Patch %	Lines
datafusion/catalog/src/memory/table.rs	81.92%	44 Missing and 35 partials ⚠️
datafusion/expr/src/logical_plan/dml.rs	69.09%	3 Missing and 14 partials ⚠️
datafusion/expr/src/logical_plan/plan.rs	60.00%	0 Missing and 2 partials ⚠️
datafusion/expr/src/logical_plan/display.rs	91.66%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #24195 +/- ##
==========================================
- Coverage 81.23% 81.23% -0.01% 
==========================================
Files 1111 1111 Lines 390208 390715 +507 Branches 390208 390715 +507 ==========================================
+ Hits 316990 317383 +393 - Misses 54591 54653 +62 - Partials 18627 18679 +52 

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[timsaucer]

It looks like this will mutate the table even when we don't execute. If you run these commands in datafusion-cli you'll see:

CREATETABLEdestination(id INT, val VARCHAR);
INSERT INTO destination VALUES (1, 'a'), (2, 'b'), (3, 'c');
CREATETABLEsource(id INT, val VARCHAR);
INSERT INTO source VALUES (2, 'B');
SELECT*FROM destination ORDER BY id;
EXPLAIN MERGE INTO destination USING source ONdestination.id=source.id
WHEN MATCHED THEN UPDATESET val =source.val;
SELECT*FROM destination ORDER BY id;

Yields this output:

0 row(s) fetched.
Elapsed 0.024 seconds.
+-------+
| count |
+-------+
| 3 |
+-------+
1 row(s) fetched.
Elapsed 0.018 seconds.
0 row(s) fetched.
Elapsed 0.000 seconds.
+-------+
| count |
+-------+
| 1 |
+-------+
1 row(s) fetched.
Elapsed 0.000 seconds.
+----+-----+
| id | val |
+----+-----+
| 1 | a |
| 2 | b |
| 3 | c |
+----+-----+
3 row(s) fetched.
Elapsed 0.006 seconds.
+---------------+-------------------------------+
| plan_type | plan |
+---------------+-------------------------------+
| physical_plan | ┌───────────────────────────┐ |
| | │ CooperativeExec │ |
| | │ -------------------- │ |
| | │ CooperativeExec │ |
| | └─────────────┬─────────────┘ |
| | ┌─────────────┴─────────────┐ |
| | │ DmlResultExec │ |
| | │ -------------------- │ |
| | │ DmlResultExec: │ |
| | │ rows_affected │ |
| | │ : │ |
| | │ 1 │ |
| | └───────────────────────────┘ |
| | |
+---------------+-------------------------------+
1 row(s) fetched.
Elapsed 0.003 seconds.
+----+-----+
| id | val |
+----+-----+
| 1 | a |
| 2 | B |
| 3 | c |
+----+-----+
3 row(s) fetched.
Elapsed 0.000 seconds.

The big difference I want to point out is in the two select commands. In the first one we have values a, b, c and in the second we have a, B, c so the table was mutated even though we only did an explain plan. I didn't expect explain to actually do the mutation.

Now I think this is a pre-existing issue. In fact if you run an insert command you have similar problems. Starting in a fresh cli:

INSERT INTO t2 VALUES (1), (2);
SELECT*FROM t2;
EXPLAIN UPDATE t2 SET a =99;
SELECT*FROM t2;

Yields:

0 row(s) fetched.
Elapsed 0.024 seconds.
+-------+
| count |
+-------+
| 2 |
+-------+
1 row(s) fetched.
Elapsed 0.018 seconds.
+---+
| a |
+---+
| 1 |
| 2 |
+---+
2 row(s) fetched.
Elapsed 0.002 seconds.
+---------------+-------------------------------+
| plan_type | plan |
+---------------+-------------------------------+
| physical_plan | ┌───────────────────────────┐ |
| | │ CooperativeExec │ |
| | │ -------------------- │ |
| | │ CooperativeExec │ |
| | └─────────────┬─────────────┘ |
| | ┌─────────────┴─────────────┐ |
| | │ DmlResultExec │ |
| | │ -------------------- │ |
| | │ DmlResultExec: │ |
| | │ rows_affected │ |
| | │ : │ |
| | │ 2 │ |
| | └───────────────────────────┘ |
| | |
+---------------+-------------------------------+
1 row(s) fetched.
Elapsed 0.003 seconds.
+----+
| a |
+----+
| 99 |
| 99 |
+----+
2 row(s) fetched.
Elapsed 0.000 seconds.

So again we're getting mutation of the data during an explain. I'm not sure if we want to merge as is, make a change, or open an issue. Do others have thoughts? Is this a well known and accepted approach - mutation on explain?