joins::nested_loop_join::tests::join_maintains_right_order tests take over 60 seconds

[alamb]

Describe the bug

Some CI tests are taking over 60 seconds to complete, which results in slower CI runs slower local development

To Reproduce

Using cargo nextest https://nexte.st/ shows this problem pretty well

cargo nextest run 
....
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 PASS [ 80.763s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 PASS [ 79.821s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 PASS [ 83.138s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 PASS [ 83.531s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_1_JoinType__Inner::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 PASS [ 78.561s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 SLOW [> 60.000s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 PASS [ 78.559s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 PASS [ 81.219s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_2_JoinType__Right::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 PASS [ 73.597s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 PASS [ 82.230s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_1_1::right_batch_size_3_1000::batch_size_1_2
 PASS [ 75.764s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_3_JoinType__RightAnti::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
 PASS [ 74.621s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_2_100::right_batch_size_3_1000::batch_size_1_2
 PASS [ 75.116s] datafusion-physical-plan joins::nested_loop_join::tests::join_maintains_right_order::join_type_4_JoinType__RightSemi::left_batch_size_3_1000::right_batch_size_3_1000::batch_size_1_2
────────────
 Summary [ 175.130s] 6405 tests run: 6405 passed (12 slow), 9 skipped

Expected behavior

I would like those tests to take significantly less time (ideally a few seconds at most)

Additional context

No response