Skip to content

Avoid treating incomparable scalars as equal - #16624

Merged
findepi merged 4 commits into
apache:mainfrom
findepi:findepi/avoid-treating-incomparable-scalars-as-equal-bed985
Jul 1, 2025
Merged

Avoid treating incomparable scalars as equal#16624
findepi merged 4 commits into
apache:mainfrom
findepi:findepi/avoid-treating-incomparable-scalars-as-equal-bed985

Conversation

@findepi

@findepifindepi commented Jun 30, 2025

Copy link
Copy Markdown
Member

Fix calls to ScalarValue::partial_cmp that treat None return value as equivalent to equality. The partial_cmp returns None when values cannot be compared, for example they have incompatible types. Apparently, in all these places it is an error condition when values cannot be compared.

@github-actionsgithub-actionsBot added optimizer Optimizer rules core Core DataFusion crate common Related to common crate functions Changes to functions implementation labels Jun 30, 2025
@findepi
Avoid treating incomparable scalars as equal
251945e
Fix calls to `ScalarValue::partial_cmp` that treat `None` return value
as equivalent to equality. The `partial_cmp` returns `None` when values
cannot be compared, for example they have incompatible types.
Apparently, in all these places it is an error condition when values
cannot be compared.
@findepi
Remove doc mention of self from a function that doesn't take self
6d1654f
@findepi
findepiforce-pushed the findepi/avoid-treating-incomparable-scalars-as-equal-bed985 branch from 6843dbb to 6d1654fCompareJune 30, 2025 12:51
alamb
alamb approved these changes Jun 30, 2025
View reviewed changes

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @findepi -- I think this will improve our ability to find and fix bugs 👍

Comment threaddatafusion/common/src/scalar/mod.rs Outdated
Comment threaddatafusion/common/src/scalar/mod.rs
findepiand others added 2 commits July 1, 2025 00:41
@findepi@alamb
Improve method doc
284ff4f
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@findepi
fix trait name
64bc4b9
@findepi
findepi merged commit 47f75ef into apache:mainJul 1, 2025
27 checks passed
@findepi
findepi deleted the findepi/avoid-treating-incomparable-scalars-as-equal-bed985 branch July 1, 2025 09:42
@adriangbadriangb mentioned this pull request Sep 10, 2025
Closed
jcsherin added a commit to jcsherin/datafusion that referenced this pull request Nov 23, 2025
@jcsherin
fix: uses ScalarValue::try_cmp
08cdacc
See apache#16624
@jcsherinjcsherin mentioned this pull request Nov 24, 2025
Merged
7 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alambalambalamb approved these changes

Assignees

No one assigned

Labels

commonRelated to common cratecoreCore DataFusion cratefunctionsChanges to functions implementationoptimizerOptimizer rules

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@findepi@alamb