fix: Pick correct columns in Sort Merge Equijoin

[tglanz]

Which issue does this PR close?

• ClosesPanic in Sort Merge Equijoin when tables have different columns count #18804.

Rationale for this change

What changes are included in this PR?

Take correct columns

Are these changes tested?

Yes,

• added rust tests to hit invalid code paths
• sqltests
• fuzz tests enhancement to fuzzify columns count

Fuzz tests are taken from @rluvaton 's #18788 , excluding those this PR doesn't fix:

 fuzz_cases::join_fuzz::test_right_anti_join_1k_binary_filtered
fuzz_cases::join_fuzz::test_right_anti_join_1k_filtered
fuzz_cases::join_fuzz::test_right_semi_join_1k_filtered

Are there any user-facing changes?

[rluvaton]

I think this should also be changed to:

Suggested change

	.take(left_columns_length)

[rluvaton]

I was able to reproduce the bug when changing the sort_merge_join.slt to have 3 columns in t2 rather than 2 like in t1 (below is the diff I changed in the sort_merge_join.slt file).

Can you please update the description and update/add tests (I would update the sort_merge_join.slt like in the diff below to make sure all tests are testing that. make sure to add a comment on why 3 columns and t1 2).

Updated the slt file to reproduce the error

diff --git a/datafusion/sqllogictest/test_files/sort_merge_join.slt b/datafusion/sqllogictest/test_files/sort_merge_join.slt--- a/datafusion/sqllogictest/test_files/sort_merge_join.slt	(revision f3980641660997345af6061dc3b34f365020bd07)+++ b/datafusion/sqllogictest/test_files/sort_merge_join.slt	(date 1763411346050)@@ -26,7 +26,7 @@
CREATE TABLE t1(a text, b int) AS VALUES ('Alice', 50), ('Alice', 100), ('Bob', 1);
statement ok
-CREATE TABLE t2(a text, b int) AS VALUES ('Alice', 2), ('Alice', 1);+CREATE TABLE t2(a text, b int, c int) AS VALUES ('Alice', 2, 77), ('Alice', 1, 66);
# inner join query plan with join filter
query TT
@@ -64,83 +64,83 @@
----
# left join without join filter
-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Alice 50 Alice 2-Bob 1 NULL NULL+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Alice 50 Alice 2 77+Bob 1 NULL NULL NULL
# left join with join filter
-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a AND t2.b * 50 <= t1.b
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Bob 1 NULL NULL+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Bob 1 NULL NULL NULL-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a AND t2.b < t1.b
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Alice 50 Alice 2-Bob 1 NULL NULL+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Alice 50 Alice 2 77+Bob 1 NULL NULL NULL
# right join without join filter
-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 RIGHT JOIN t2 ON t1.a = t2.a
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Alice 50 Alice 2+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Alice 50 Alice 2 77
# right join with join filter
-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 RIGHT JOIN t2 ON t1.a = t2.a AND t2.b * 50 <= t1.b
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 RIGHT JOIN t2 ON t1.a = t2.a AND t1.b > t2.b
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Alice 50 Alice 2+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Alice 50 Alice 2 77
# full join without join filter
-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 FULL JOIN t2 ON t1.a = t2.a
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 Alice 1-Alice 50 Alice 2-Bob 1 NULL NULL+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 Alice 1 66+Alice 50 Alice 2 77+Bob 1 NULL NULL NULL-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 FULL JOIN t2 ON t1.a = t2.a AND t2.b * 50 > t1.b
----
-Alice 100 NULL NULL-Alice 50 Alice 2-Bob 1 NULL NULL-NULL NULL Alice 1+Alice 100 NULL NULL NULL+Alice 50 Alice 2 77+Bob 1 NULL NULL NULL+NULL NULL Alice 1 66-query TITI rowsort+query TITII rowsort
SELECT * FROM t1 FULL JOIN t2 ON t1.a = t2.a AND t1.b > t2.b + 50
----
-Alice 100 Alice 1-Alice 100 Alice 2-Alice 50 NULL NULL-Bob 1 NULL NULL+Alice 100 Alice 1 66+Alice 100 Alice 2 77+Alice 50 NULL NULL NULL+Bob 1 NULL NULL NULL
statement ok
DROP TABLE t1;

[tglanz]

Thanks @rluvaton, Sure

[tglanz]

Fuzz tests are taken from @rluvaton 's #18788 , excluding those this PR doesn't fix:

 fuzz_cases::join_fuzz::test_right_anti_join_1k_binary_filtered
fuzz_cases::join_fuzz::test_right_anti_join_1k_filtered
fuzz_cases::join_fuzz::test_right_semi_join_1k_filtered

[rluvaton]

Thank you @tglanz LGTM.

Hope to see you contributing again soon