Skip to content

Not equal predicate in physical_planning pruning - #544

Merged
alamb merged 2 commits into
apache:masterfrom
jgoday:ne_predicate_in_pruning
Jun 13, 2021
Merged

Not equal predicate in physical_planning pruning#544
alamb merged 2 commits into
apache:masterfrom
jgoday:ne_predicate_in_pruning

Conversation

@jgoday

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes#420.

Rationale for this change

Pruning not equal predicates if the value is between min and max bounds.

What changes are included in this PR?

Are there any user-facing changes?

Change datafusion/src/physical_optimizer/pruning.rs build_predicate_expression to include Operator::NotEq
and define test case 'row_group_predicate_not_eq'

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

Merging #544 (c4e91d9) into master (ad70a1e) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@ Coverage Diff @@## master #544 +/- ##
==========================================
+ Coverage 76.13% 76.15% +0.01% 
==========================================
Files 156 156 Lines 27032 27047 +15 ==========================================
+ Hits 20582 20597 +15 
Misses 6450 6450 
Impacted FilesCoverage Δ
datafusion/src/physical_optimizer/pruning.rs92.52% <100.00%> (+0.24%)⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad70a1e...c4e91d9. Read the comment docs.

@DandandanDandandan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome -- thank you @jgoday !

@alamb
alamb merged commit 2568323 into apache:masterJun 13, 2021
@alamb

alamb commented Jun 14, 2021

Copy link
Copy Markdown
Contributor

I think we got the logic a bit too aggressive see -- see #560. FYI @jgoday

alamb added a commit to alamb/datafusion that referenced this pull request Jun 14, 2021
@houqphouqp added datafusion enhancement New feature or request labels Jul 30, 2021
unkloud pushed a commit to unkloud/datafusion that referenced this pull request Mar 23, 2025
* fix error message
Co-authored-by: Andy Grove <andygrove73@gmail.com>
---------
Co-authored-by: Andy Grove <andygrove73@gmail.com>
HairstonE pushed a commit to HairstonE/datafusion that referenced this pull request Oct 7, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for != predicate in pruning predicates

5 participants

@jgoday@codecov-commenter@alamb@Dandandan@houqp