ARROW-11064: [Rust][DataFusion] Speed up hash join on smaller batches

[Dandandan]

FYI @andygrove@jorgecarleitao

This is related to findings here: #9036
This is a ~20% win on smaller batch sizes, but also has an effect on bigger sizes (about 10% on size of 16k locally)

Query 12 with batch size 4096

PR:

Query 12 iteration 0 took 484.0 ms
Query 12 iteration 1 took 482.5 ms
Query 12 iteration 2 took 496.9 ms
Query 12 iteration 3 took 488.1 ms
Query 12 iteration 4 took 504.9 ms
Query 12 iteration 5 took 490.4 ms
Query 12 iteration 6 took 486.8 ms
Query 12 iteration 7 took 499.3 ms
Query 12 iteration 8 took 485.0 ms
Query 12 iteration 9 took 488.8 ms
Query 12 avg time: 490.67 ms

Master:

Query 12 iteration 0 took 669.6 ms
Query 12 iteration 1 took 764.8 ms
Query 12 iteration 2 took 705.8 ms
Query 12 iteration 3 took 645.1 ms
Query 12 iteration 4 took 640.9 ms
Query 12 iteration 5 took 639.2 ms
Query 12 iteration 6 took 658.9 ms
Query 12 iteration 7 took 643.2 ms
Query 12 iteration 8 took 639.0 ms
Query 12 iteration 9 took 639.2 ms
Query 12 avg time: 664.57 ms

[github-actions]

https://issues.apache.org/jira/browse/ARROW-11064

[andygrove]

At SF=100 I see a small improvement from 191 seconds to 176 seconds (at batch size 16384). Thanks @Dandandan

[jorgecarleitao]

Thanks a lot @Dandandan , how these pisky clones are affecting performance :(

[Dandandan]

@jorgecarleitao Yes, small things can have a big effect...

The difference is a bit "magnified" though because of the n*n behavior in the join.
If you have ideas of how to solve this issue, please share.

What looks like is the problem here:

• for the left side of the join we .iter() on the left side batches iteration (a couple of times, in the join implementation itself and in MutableDataArray) when we process a probe-side batch.
• if we halve the batch size, the number of items in the build-side array grows by two, but also the number of probe-side batches we process in the join, thus the n*n slowdown.