GH-40592: [C++][Parquet] Implement SizeStatistics

[wgtmac]

Rationale for this change

Parquet format 2.10.0 has introduced SizeStatistics. parquet-mr has also implemented this: apache/parquet-java#1177. Now it is time for parquet-cpp to pick the ball.

What changes are included in this PR?

Implement reading and writing size statistics for parquet-cpp.

Are these changes tested?

Yes, a bunch of test cases have been added.

Are there any user-facing changes?

Yes, now parquet users are able to read and write size statistics.

• GitHub Issue: [C++][Parquet] Implement SizeStatistics #40592

[emkornfield]

A few high level questions/suggestions.

[emkornfield]

Took another quick pass through and didn't see any major blockers, as long as @pitrou is happy with changes I'm OK to merge.

[pitrou]

Thanks for the PR @wgtmac ! The approach looks generally fine, some assorted comments below.

[wgtmac]

Thanks for your thorough review! I think I have addressed all comments except for the benchmark. Could you please take a look again? @pitrou

[wgtmac]

Gentle ping @pitrou :)

[pitrou]

Thanks a lot @wgtmac and sorry for having overlooked this. LGTM, I only have minor comments below.

Should we look into benchmarks to change the default size_statistics_level, or should we leave this for another issue/PR?

[wgtmac]

Thanks for the review! @pitrou I've created #45045 to follow up with the benchmark and default value of size_statistics_level.

[wgtmac]

Damn, I also hit the ORC timeout issue from the failed CI:

 --- LOG END ---
error: downloading 'https://archive.apache.org/dist/orc/orc-format-1.0.0/orc-format-1.0.0.tar.gz' failed
status_code: 28
status_string: "Timeout was reached"
log:
--- LOG BEGIN ---
Host archive.apache.org:443 was resolved.

[wgtmac]

Rebased and got following (unrelated) failure from the MacOS CI:

[549/1117] Building CXX object src/arrow/CMakeFiles/arrow_testing_objlib.dir/testing/process.cc.o
FAILED: src/arrow/CMakeFiles/arrow_testing_objlib.dir/testing/process.cc.o /usr/local/bin/ccache /Applications/Xcode_15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DARROW_HAVE_RUNTIME_AVX2 -DARROW_HAVE_RUNTIME_AVX512 -DARROW_HAVE_RUNTIME_BMI2 -DARROW_HAVE_RUNTIME_SSE4_2 -DARROW_HAVE_SSE4_2 -DARROW_TESTING_EXPORTING -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_DATE_TIME_DYN_LINK -DBOOST_DATE_TIME_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROCESS_DYN_LINK -DBOOST_PROCESS_HAVE_V1 -DBOOST_PROCESS_HAVE_V2 -DBOOST_PROCESS_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -I/Users/runner/work/arrow/arrow/build/cpp/src -I/Users/runner/work/arrow/arrow/cpp/src -I/Users/runner/work/arrow/arrow/cpp/src/generated -isystem /Users/runner/work/arrow/arrow/build/cpp/_deps/googletest-src/googletest/include -isystem /Users/runner/work/arrow/arrow/build/cpp/_deps/googletest-src/googletest -isystem /Users/runner/work/arrow/arrow/build/cpp/_deps/googletest-src/googlemock/include -isystem /Users/runner/work/arrow/arrow/build/cpp/_deps/googletest-src/googlemock -isystem /Users/runner/work/arrow/arrow/cpp/thirdparty/flatbuffers/include -isystem /usr/local/Cellar/rapidjson/1.1.0/include -isystem /usr/local/include -fno-aligned-new -Qunused-arguments -fcolor-diagnostics -Wall -Wextra -Wdocumentation -DARROW_WARN_DOCUMENTATION -Wshorten-64-to-32 -Wno-missing-braces -Wno-unused-parameter -Wno-constant-logical-operand -Wno-return-stack-address -Wdate-time -Wno-unknown-warning-option -Wno-pass-failed -msse4.2 -g -Werror -O0 -ggdb -g1 -std=c++17 -isysroot /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -mmacosx-version-min=13.7 -fPIC -MD -MT src/arrow/CMakeFiles/arrow_testing_objlib.dir/testing/process.cc.o -MF src/arrow/CMakeFiles/arrow_testing_objlib.dir/testing/process.cc.o.d -o src/arrow/CMakeFiles/arrow_testing_objlib.dir/testing/process.cc.o -c /Users/runner/work/arrow/arrow/cpp/src/arrow/testing/process.cc
/Users/runner/work/arrow/arrow/cpp/src/arrow/testing/process.cc:88:18: error: expected namespace name
namespace asio = BOOST_PROCESS_V2_ASIO_NAMESPACE;
^
/Users/runner/work/arrow/arrow/cpp/src/arrow/testing/process.cc:246:3: error: use of undeclared identifier 'asio'; did you mean 'boost::asio'?
asio::io_context ctx_;
^~~~
boost::asio
/usr/local/include/boost/asio/writable_pipe.hpp:26:11: note: 'boost::asio' declared here
namespace asio {
^
2 errors generated.

[pitrou]

The macOS failure was fixed in #45057, so you can rebase another time 😁

[wgtmac]

Here we go!

[pitrou]

Thanks a lot @wgtmac , we can merge now.

[wgtmac]

Thank you for your thorough review! Let me work on the benchmark.

[mapleFU]

Grats!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit f93004f.

There was 1 benchmark result indicating a performance regression:

• Commit Run on amd64-m5-4xlarge-linux at 2025-01-10 19:38:55Z
  • file-write (Python) with compression=uncompressed, dataset=fanniemae_2016Q4, file_type=feather, input_type=dataframe

The full Conbench report has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them.