ARROW-13764: [C++] Support CountOptions in grouped count distinct

[lidavidm]

This works by filtering the values (trading off time for memory usage).

[github-actions]

https://issues.apache.org/jira/browse/ARROW-13764

[lidavidm]

I'm still digging into this, but I noticed the unit tests are especially slow in debug mode. This is because we make a 2 GiB size allocation in GrouperFastImpl::GetUniques, which also makes me worry that these tests might use too much memory for CI.

[lidavidm]

Also, filtering is not the right tradeoff to make: I think we don't care too much about the ONLY_NULL case and in the other cases, counting nulls is not that big a deal. So I'll update this to do the filtering in Finalize().

[lidavidm]

Changed to filter the results after consumption. For whatever reason, not filtering before feeding batches into the grouper doesn't cause the runtime/memory blowup.

[pitrou]

Thank you. A couple comments, but looks mostly good to me.

[pitrou]

values->null_bitmap() may be null here, I think.

[pitrou]

Perhaps add a test case where none of the values is null?

[pitrou]

Also add that the setting doesn't apply to the grouping keys?

[pitrou]

This is because we make a 2 GiB size allocation in GrouperFastImpl::GetUniques

Really? Why does this happen?

[lidavidm]

This is because we make a 2 GiB size allocation in GrouperFastImpl::GetUniques

Really? Why does this happen?

I was trying to figure out why, but didn't get very far in untangling the internals before I decided that filtering before grouping was pointless anyways. This doesn't happen the current implementation, I might poke at this further later though.

[lidavidm]

Thanks for the suggestions. I believe I've addressed everything.

[pitrou]

+1, thank you!