Skip to content

GH-34785: [C++][Parquet] Add bloom filter write support#37400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pitrou merged 98 commits into from
Jan 14, 2026
Merged

GH-34785: [C++][Parquet] Add bloom filter write support #37400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
f1c6dc0
Parquet: Implement skeleton for BloomFilter
mapleFU Aug 26, 2023
6ebd6da
tiny fixing
mapleFU Aug 26, 2023
70c9267
tiny update test
mapleFU Aug 26, 2023
48350d8
trying to fix ci
mapleFU Aug 26, 2023
d2a659e
fix lint
mapleFU Aug 26, 2023
41236d8
fix some style problem
mapleFU Aug 26, 2023
8afba81
add file roundtrip test
mapleFU Aug 26, 2023
96c6691
add file roundtrip test
mapleFU Aug 26, 2023
c131341
fix document and ci
mapleFU Aug 26, 2023
220b58e
Update: tiny style fix
mapleFU Aug 26, 2023
ad96c48
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Sep 2, 2023
b756241
Bloom Filter Resolve comments:
mapleFU Sep 2, 2023
f43505b
make space writing a batched writing
mapleFU Sep 2, 2023
3497f4a
update bloom_filter builder interface
mapleFU Sep 2, 2023
fecd0f0
update BloomFilterBuilder arguments
mapleFU Sep 2, 2023
29cc1c1
fix compile
mapleFU Sep 2, 2023
ffbb491
try to satisfy win compiler
mapleFU Sep 2, 2023
4d63428
change all to vector
mapleFU Sep 2, 2023
f689716
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Sep 11, 2023
8e9cb16
resolve comment
mapleFU Sep 11, 2023
7fd47be
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Oct 2, 2023
7c4ff4e
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Oct 10, 2023
feccee9
fix some comment
mapleFU Oct 10, 2023
90245e7
add cached version test
mapleFU Oct 10, 2023
d924e36
cleaning the code for column-props
mapleFU Oct 10, 2023
0340193
optimize get bf
mapleFU Oct 10, 2023
b78eed0
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Mar 16, 2024
23828e1
comment minor fix
mapleFU Mar 16, 2024
6fd57dc
fix comment and add bloom-filter-length
mapleFU Mar 16, 2024
86a8760
Fix a bf bug
mapleFU Mar 16, 2024
f8e724c
trying to use std::map for RowGroup filter
mapleFU Mar 17, 2024
447badf
trying to fix msvc compile
mapleFU Mar 17, 2024
0c1065c
fix comment
mapleFU Mar 17, 2024
5225e08
add test case for 2 row-groups
mapleFU Mar 17, 2024
a779982
add test case for dictionary
mapleFU Mar 17, 2024
4195406
minor update style for file_writer.cc
mapleFU Mar 17, 2024
ed267bd
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Mar 26, 2024
478889d
resolve comment
mapleFU Mar 26, 2024
2992072
fix comment for boolean col, and add test
mapleFU Mar 26, 2024
4852261
trying to add bloom boolean test
mapleFU Mar 26, 2024
add1afd
fix test
mapleFU Mar 26, 2024
f627e30
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Apr 8, 2024
bb8d4a5
fix some comments
mapleFU Apr 8, 2024
ad0f1af
Merge branch 'parquet/support-write-bloom-filter' of github.com:maple…
mapleFU Apr 8, 2024
e1de5bc
fix lint
mapleFU Apr 8, 2024
430742a
switch to anonymous namespace
mapleFU Apr 9, 2024
00f176e
fix comment for column_writer.cc
mapleFU Apr 26, 2024
17f4951
fix comment in other parts
mapleFU Apr 26, 2024
de27ce4
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Apr 26, 2024
259f15b
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Apr 26, 2024
057b542
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Jun 10, 2024
34a4c28
trying to fix the ci build
mapleFU Jun 10, 2024
70e3508
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Jul 3, 2024
c587568
resolve comments
mapleFU Jul 3, 2024
2223423
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Nov 11, 2024
22030db
change the bloom filter from vector to map
mapleFU Nov 11, 2024
e9c550a
fix lint
mapleFU Nov 11, 2024
23fb3fa
fix lint
mapleFU Nov 14, 2024
d892819
fix comment
mapleFU Nov 15, 2024
ef3291d
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Dec 20, 2024
7aee7dd
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Jan 13, 2025
c5b1fb1
Resolve comments
mapleFU Jan 13, 2025
0898466
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Feb 5, 2025
71f5906
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Feb 7, 2025
d57ceea
minor fix
mapleFU Feb 7, 2025
26c2d07
address some comments
mapleFU Feb 7, 2025
d422ffa
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Mar 10, 2025
e6bc6e1
Minor fix
mapleFU Mar 10, 2025
dfaf0e8
try to fix lint
mapleFU Mar 10, 2025
0bafe78
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Mar 26, 2025
ce30ebc
Resolve comment part1
mapleFU Apr 24, 2025
8286783
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Apr 27, 2025
b079acb
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU May 28, 2025
3a5a491
Extract a BloomFilterWriterImpl, and supports binary-view type
mapleFU May 28, 2025
cccb9a8
test for string_view type
mapleFU May 28, 2025
3cf9425
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Jun 3, 2025
aac454e
Fix comment part1
mapleFU Jun 3, 2025
83999cd
add writer_internal for bf
mapleFU Jun 3, 2025
fa0c9b1
try to fix ci
mapleFU Jun 3, 2025
351da07
Trying to fix lint
mapleFU Jun 4, 2025
12364d0
Remove duplicate code
mapleFU Jun 6, 2025
8dec902
Merge branch 'parquet/support-write-bloom-filter' of https://github.c…
mapleFU Jun 6, 2025
40c9079
Merge branch 'main' into parquet/support-write-bloom-filter
mapleFU Jun 19, 2025
d32c40b
Apply suggestions
mapleFU Jun 20, 2025
a662563
apply suggestions
mapleFU Jun 20, 2025
61b6dff
fix lint
mapleFU Jun 20, 2025
18f1a47
Update: remove some , and fix include
mapleFU Jun 20, 2025
2bfa278
Merge remote-tracking branch 'origin' into parquet/support-write-bloo…
wgtmac Nov 20, 2025
f03a327
address comments
wgtmac Nov 20, 2025
4aeff8b
Merge remote-tracking branch 'origin' into parquet/support-write-bloo…
wgtmac Nov 26, 2025
0f50418
address comments
wgtmac Nov 26, 2025
789d130
address comments
wgtmac Dec 5, 2025
6dc8d88
add template definition back
wgtmac Dec 7, 2025
0940cd8
Merge branch 'main' into parquet/support-write-bloom-filter
wgtmac Jan 14, 2026
a126e03
polish test and address feedback
wgtmac Jan 14, 2026
e560a28
address more comments
wgtmac Jan 14, 2026
bf5e859
update table
wgtmac Jan 14, 2026
0638b11
update table
wgtmac Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion cpp/src/parquet/CMakeLists.txt
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -161,6 +161,7 @@ set(PARQUET_SRCS
arrow/writer.cc
bloom_filter.cc
bloom_filter_reader.cc
bloom_filter_writer.cc
chunker_internal.cc
column_reader.cc
column_scanner.cc
Expand DownExpand Up@@ -373,7 +374,7 @@ set_source_files_properties(public_api_test.cc PROPERTIES SKIP_UNITY_BUILD_INCLU

add_parquet_test(internals-test
SOURCES
bloom_filter_reader_test.cc
bloom_filter_reader_writer_test.cc
Comment thread
wgtmac marked this conversation as resolved.
bloom_filter_test.cc
geospatial/statistics_test.cc
geospatial/util_internal_test.cc
Expand DownExpand Up@@ -409,6 +410,8 @@ add_parquet_test(arrow-reader-writer-test
arrow/arrow_statistics_test.cc
arrow/variant_test.cc)

add_parquet_test(arrow-index-test SOURCES arrow/index_test.cc)

add_parquet_test(arrow-internals-test SOURCES arrow/path_internal_test.cc
arrow/reconstruct_internal_test.cc)

Expand Down
387 changes: 3 additions & 384 deletions cpp/src/parquet/arrow/arrow_reader_writer_test.cc
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading