Skip to content

Coerce and simplify FixedSizeBinary equality to literal binary - #15726

Merged
jayzhan211 merged 6 commits into
apache:mainfrom
leoyvens:coerce-and-simplify-fixed-size-binary-eq-lit-binary
Apr 18, 2025
Merged

Coerce and simplify FixedSizeBinary equality to literal binary#15726
jayzhan211 merged 6 commits into
apache:mainfrom
leoyvens:coerce-and-simplify-fixed-size-binary-eq-lit-binary

Conversation

@leoyvens

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

See #15686 (comment).

What changes are included in this PR?

The two initial commits have two sequential changes:

  • Expand type coercion to coerce FixedSizeBinary to Binary when necessary, as we already do for lists.
  • Introduce a case to the ExprSimplifier to optimize away the coercion cast when comparing to a literal.

Are these changes tested?

There is an .slt test. An existing one that previously failed to coerce will now evaluate the comparison.

Are there any user-facing changes?

No, this should be very transparent.

@github-actionsgithub-actionsBot added logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Apr 15, 2025
@leoyvens
leoyvensforce-pushed the coerce-and-simplify-fixed-size-binary-eq-lit-binary branch from c56e280 to c72a478CompareApril 15, 2025 20:48
@leoyvens

leoyvens commented Apr 16, 2025

Copy link
Copy Markdown
ContributorAuthor

I now realize I the ExprSimplifier case I added was entirely redundant with the existing unwrap_cast_in_comparison case. I've removed it and just kept the coercion and test cases.

@leoyvens

Copy link
Copy Markdown
ContributorAuthor

cc @jayzhan211, could I ask you for a review?

@jayzhan211jayzhan211 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good -- thank you @leoyvens and @jayzhan211

I pushed one more slt test that verifies the cast is removed via an EXPLAIN. This is a very nice improvement I think

}
(Binary, Utf8) | (Utf8, Binary) => Some(Binary),

// Cast FixedSizeBinary to Binary

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a follow on PR it might be good to also support BinaryView as well

@jayzhan211
jayzhan211 merged commit 57a99e4 into apache:mainApr 18, 2025
@jayzhan211

Copy link
Copy Markdown
Contributor

Thanks @leoyvens@alamb

nirnayroy pushed a commit to nirnayroy/datafusion that referenced this pull request May 2, 2025
…e#15726)
* coerce FixedSizeBinary to Binary
* simplify FixedSizeBytes equality to literal
* fix clippy
* remove redundant ExprSimplifier case
* Add explain test to make sure unwrapping is working correctly
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-exprLogical plan and expressionsoptimizerOptimizer rulessqllogictestSQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow parsing byte literals as FixedSizeBinary

3 participants

@leoyvens@jayzhan211@alamb