Expand LIKE simplification: cover `NULL` pattern/expression and constant by findepi · Pull Request #13260 · apache/datafusion

findepi · 2024-11-05T12:51:11Z

cover expression known not to be null
cover NULL pattern
cover repeated '%%' in pattern
currently also includes changes from Simplify EXPR LIKE 'constant' to expr = 'constant' #13061 to resolve conflicts

findepi · 2024-11-05T13:39:35Z

currently depends on #13259

findepi · 2024-11-05T15:04:19Z

CI is green, so ready to review
but this will conflict with @adriangb's #13061, because this changes code structure a little bit
conflict resolution will be trivial though.
@adriangb if you want, i can do this, adding your changes to this PR

findepi · 2024-11-05T15:12:34Z

seemed easy enough, done.

adriangb · 2024-11-05T15:15:40Z

I’m happy with my changes being included in this PR :)

findepi · 2024-11-05T15:22:34Z

draft - to be rebased after #13259 lands

still ready to review
cc @crepererum @goldmedal

goldmedal

Just roughly review now. Overall looks to me. I will check the test cases tomorrow.

goldmedal · 2024-11-06T02:51:07Z

draft - to be rebased after #13259 lands
still ready to review cc @crepererum @goldmedal

#13259 has been merged. 👍

findepi · 2024-11-06T07:31:35Z

@goldmedal rebased, thanks!

alamb

Thank you @findepi and @goldmedal

This looks like a great change to me except for the handling of %% which I am not sure about. Otherwise 👍

alamb · 2024-11-06T11:38:51Z

 }

- fn like(expr: Expr, pattern: &str) -> Expr {
+ fn like(expr: Expr, pattern: impl Into<Expr>) -> Expr {


I think we could use Expr::like https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.Expr.html#method.like and similar here instead of these functions
This is likely left over from when the Expr API was less expressive

crepererum · 2024-11-06T15:43:55Z

since there are already two reviewers on it, I'll gonna skip this PR. However if you need my input, ping me.

alamb

Thank you @findepi and @goldmedal

I had some small testing suggestions but I think we can add that coverage as a follow on PR as well

findepi · 2024-11-07T09:56:29Z

FYI, when testing LIKE patterns with potential escapes, beware of difference between sqllogictests and CLI when it comes to backslash (like implicit escape) -- #13286

alamb · 2024-11-07T18:06:35Z

Looks like this branch has some conflicts now to resolve but then I think it will be ready to go

- cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern

findepi · 2024-11-08T14:58:00Z

The conflicts are because #13288 is now merged. Let me rebase.

findepi · 2024-11-08T14:59:05Z

(just rebased)

findepi · 2024-11-08T15:22:47Z

Applied comments and fixed handling of implicit escape.

alamb

🚀

alamb · 2024-11-08T18:45:22Z

Thanks @findepi @adriangb and @goldmedal for this PR and the reviews

…ant (apache#13260) * Expand LIKE simplification - cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern * Simplify `EXPR LIKE 'constant'` to `expr = 'constant'` * Correctness and style fixes * fix typo --------- Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>

github-actionsBot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Nov 5, 2024

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from c6285bf to be26107Compare November 5, 2024 13:39

findepi mentioned this pull request Nov 5, 2024
Incorrect LIKE and ILIKE result for NULL input, `%', and '%%' pattern #12637
Closed

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from 88d6df1 to 6c57af7Compare November 5, 2024 15:14

findepi marked this pull request as draft November 5, 2024 15:21

goldmedal reviewed Nov 5, 2024
View reviewed changes

Comment threaddatafusion/optimizer/Cargo.toml Outdated

adriangb mentioned this pull request Nov 5, 2024
Simplify EXPR LIKE 'constant' to expr = 'constant'#13061
Closed

goldmedal mentioned this pull request Nov 6, 2024
Fix incorrect ... LIKE '%' simplification with NULLs #13259
Merged

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e4eae46 to 520ad2bCompare November 6, 2024 07:31

findepi marked this pull request as ready for review November 6, 2024 07:31

alamb reviewed Nov 6, 2024
View reviewed changes

alamb changed the title ~~Expand LIKE simplification~~ Expand LIKE simplification: cover NULL pattern/expression and constantNov 6, 2024

goldmedal reviewed Nov 6, 2024
View reviewed changes

Comment threaddatafusion/optimizer/src/simplify_expressions/expr_simplifier.rs
Comment threaddatafusion/sqllogictest/test_files/string/string_view.slt Outdated

alamb approved these changes Nov 6, 2024
View reviewed changes

findepi commented Nov 7, 2024
View reviewed changes

Comment threaddatafusion/sqllogictest/test_files/string/string_query.slt.part Outdated

findepi mentioned this pull request Nov 7, 2024
Test LIKE with implicit \ escape #13288
Merged

goldmedal reviewed Nov 7, 2024
View reviewed changes

Comment threaddatafusion/sqllogictest/test_files/string/string_query.slt.part Outdated

findepiand others added 2 commits November 8, 2024 15:57

Expand LIKE simplification
96a6cd2
- cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern

Simplify EXPR LIKE 'constant' to expr = 'constant'
5a03823

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e05417a to 5a03823Compare November 8, 2024 14:58

Correctness and style fixes
0517b5d

fix typo
2585591

alamb approved these changes Nov 8, 2024
View reviewed changes

alamb merged commit 667b302 into apache:mainNov 8, 2024

findepi deleted the findepi/expand-like-simplification-e96eca branch November 8, 2024 18:54

findepi mentioned this pull request Nov 8, 2024
chore: remove unnecessary test helpers #13317
Merged

alamb mentioned this pull request Nov 20, 2024
Nov 20. 2024: This week in DataFusion #13503
Closed
3 tasks

ngli-me mentioned this pull request Feb 19, 2025
Simplify EXPR LIKE 'constant' to expr = 'constant'#13192
Closed

Uh oh!

Conversation

findepi commented Nov 5, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

adriangb commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

goldmedal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

goldmedal commented Nov 6, 2024

Uh oh!

findepi commented Nov 6, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alambNov 6, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

crepererum commented Nov 6, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

findepi commented Nov 7, 2024

Uh oh!

Uh oh!

alamb commented Nov 7, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Nov 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

findepi commented Nov 5, 2024•
edited
Loading

alamb commented Nov 7, 2024•
edited
Loading