Uh oh!
There was an error while loading. Please reload this page.
WIP: Test DataFusion with experimental Parquet Filter Pushdown - #16222
WIP: Test DataFusion with experimental Parquet Filter Pushdown#16222alamb wants to merge 4 commits into
Conversation
alamb
commented
Jun 1, 2025
🤖 |
alamb
commented
Jun 1, 2025
🤖: Benchmark completed Details |
zhuqi-lucas
commented
Jun 1, 2025
The clickbench only has several cases with real regression > 20%, and i believe those cases can be improved by combined with adaptive, i think we are at good state. |
alamb
commented
Jun 1, 2025
I agree -- thank you @zhuqi-lucas I have a few other optimization ideas on #16208 (comment) that will help this case too. It would also be super helpful to profile / review the queries where the performance slows down , like Q14 and Q21 and see if those are the ones where the adaptive filtering would help Q14: SELECT"SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM hits WHERE"SearchPhrase"<>''GROUP BY"SearchEngineID", "SearchPhrase"ORDER BY c DESCLIMIT10;Q21: SELECT"SearchPhrase", MIN("URL"), COUNT(*) AS c FROM hits WHERE"URL"LIKE'%google%'AND"SearchPhrase"<>''GROUP BY"SearchPhrase"ORDER BY c DESCLIMIT10; |
zhuqi-lucas
commented
Jun 1, 2025
apache/arrow-rs#7524 (comment) Thank you @alamb , from previous result, it will help Q14 Q24 Q30 Q31 , which are the major regression from this PR benchmark result, but it seems not help Q21/22.
|
alamb
commented
Jun 26, 2025
Next version here: #16562 |
This PR is for testing DataFusion with the code in the following PR
This is the second of 2 experiments:
pushdown_filtersenabled?The first experiment is in