Uh oh!
There was an error while loading. Please reload this page.
refact: Reuse RowGroupPageIndexReader across columns to improve page-level predicate pushdown performance regression on wide tables - #316
Conversation
…edRowGroupReader::ReadFilteredRowGroup to avoid performance drop on wide tables
cc @liangjie3138 . This PR optimizes the RowGroupPageIndexReader usage introduced in #232. Please take a look when you have a chance. I'm wondering if you have any plans to push bitmap filters down to the page level. If not, I'd like to pick this up — it would be a natural next step and could further improve point-query performance on wide tables. |
liangjie3138
commented
Jun 1, 2026
LGTM 👍 Go ahead with the page-level bitmap pushdown. |
Purpose
Linked issue: close#137
In
PageFilteredRowGroupReader::ReadFilteredRowGroup, the original implementation passed PageIndexReader into ReadFilteredColumn, where each column calledpage_index_reader->RowGroup(row_group_index)to reconstruct aRowGroupPageIndexReader. TheRowGroup()call reads and parses all ColumnIndex and OffsetIndex metadata for every column in that row group. For wide tables (hundreds or even thousands of columns), the same metadata was redundantly read N times (N = number of columns being read), causing page index read time to scale linearly and resulting in a significant performance drop for point-query scenarios where page-level predicate pushdown was supposed to help.Fix: Simply Hoist the RowGroupPageIndexReader creation out of the per-column loop in ReadFilteredRowGroup — construct it once and reuse it across all columns. Change the ReadFilteredColumn parameter type from PageIndexReader to RowGroupPageIndexReader, eliminating redundant metadata reads.
Tests
On a test case with a wide table containing over 1,000 columns, the original code took significantly longer to execute queries. The flamegraph clearly shows that RowGroup() consumed a substantial portion of the total execution time:


After applying the changes in this PR, range query performance on the wide table shows a significant improvement:

The redundant RowGroup() calls are no longer visible in the flamegraph:

API and Format
Not API and format changes.
Documentation
No new features were introduced.
Generative AI tooling
Claude code (Opus 4.6)