ARROW-15498: [C++][Compute] Implement Bloom filter pushdown between hash joins

[save-buffer]

This adds Bloom filter pushdown between hash join nodes.

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW

Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - How to contribute patches

[github-actions]

https://issues.apache.org/jira/browse/ARROW-15498

[github-actions]

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

[westonpace]

Starting to review this. Still need to go through hash_join.cc but I have some initial comments from the periphery.

[westonpace]

Why are we padding here?

[save-buffer]

It's weird if you Init the TempVectorStack with one size and then it segfaults if you try to alloc that much memory. That's because alloc bumps the stack by PaddedAllocationSize(size) + 2 * sizeof(uint64_t)

[westonpace]

Is this still needed after we fixed other TSAN related issues?

[save-buffer]

Yes, I had to make blocks atomic to make TSAN go away, which we don't want to do

[westonpace]

Why ignore it? Return it if it isn't ok.

[save-buffer]

The lambda expression needs to return void here. I can maybe DCHECK_OK it.

[westonpace]

So is the idea here to measure the overhead cost of building the bloom filter?

[save-buffer]

Well we benchmark to see what kind of performance impact the Bloom filter has.
But since we currently only do early-elimination of rows, and only build on the build side, this disqualifies some types of joins, so we check that here.

[westonpace]

Since we operate on these pairs everywhere why not create:

struct BloomFilterTarget {
HashJoinImpl* join_impl;
std::vector<int> column_map;
};

It's also takes a bit of reading to figure out what the purpose of column_map is so this could be a place to briefly describe that.

[save-buffer]

I only use the pair in one spot as far as I can tell. I just use it so that I can use std::tie on whoever calls GetPushdownTarget. I did add a big comment though

[westonpace]

Some minor suggestions but overall I think this is pretty much ready to go once #13091 merges.

[westonpace]

Let's rebase on top of master now that the thread scheduler issue is in. Then, assuming CI passes, I think this is ready to go.

[save-buffer]

OK I think this is good now. The various failures seem to be because of :

tests/test_plasma.py::TestPlasmaClient::test_connection_failure_raises_exception

And

685 CMake Error at cmake_modules/ThirdpartyToolchain.cmake:239 (find_package):
686 Could not find a package configuration file provided by "GTest" (requested
687 version 1.10.0) with any of the following names:
688
689 GTestConfig.cmake
690 gtest-config.cmake
691
692 Add the installation prefix of "GTest" to CMAKE_PREFIX_PATH or set
693 "GTest_DIR" to a directory containing one of the above files. If "GTest"
694 provides a separate development package or SDK, be sure it has been
695 installed.
696

The latter of which is being addressed in #13101

[westonpace]

I played around with this today and saw similar. Some bloom filter combinations were just very slow on my laptop. In general though I didn't see any true deadlock though I can never fully rule that out.

[save-buffer]

Just to be clear, it's not the Bloom filter that's slow (the slow parts tend to be full outer joins, where Bloom filter is disabled). It seems to be related to residual filters being slow, in particular KeyEncoder::DecodeNulls

[westonpace]

Yes. The slowdown happened in ProbeQueuedBatches which would make sense (I think) if it was related to residual filters.

[ursabot]

Benchmark runs are scheduled for baseline = 6faee47 and contender = 0742f78. 0742f78 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️1.98% ⬆️1.63%] test-mac-arm
[Failed ⬇️0.0% ⬆️11.79%] ursa-i9-9960x
[Finished ⬇️1.66% ⬆️0.32%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 0742f78a ec2-t3-xlarge-us-east-2
[Failed] 0742f78a test-mac-arm
[Failed] 0742f78a ursa-i9-9960x
[Finished] 0742f78a ursa-thinkcentre-m75q
[Finished] 6faee474 ec2-t3-xlarge-us-east-2
[Finished] 6faee474 test-mac-arm
[Finished] 6faee474 ursa-i9-9960x
[Finished] 6faee474 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

[ursabot]

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm