Uh oh!
There was an error while loading. Please reload this page.
[SPARK-51420][SQL] Get minutes of TIME datatype - #50296
Conversation
the-sakthi
commented
Mar 17, 2025
@MaxGekk looking forward to know your thoughts on this one! |
MaxGekk
left a comment
There was a problem hiding this comment.
For sure, it would be nice to re-use the existing code for getting minutes, but I believe it would be better to implement minute from a TIME value as a separate expression.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
the-sakthi
commented
Mar 17, 2025
Thanks for the review @MaxGekk |
@the-sakthi Let's leave it as is.
Could you introduce new expression |
Uh oh!
There was an error while loading. Please reload this page.
the-sakthi
commented
Mar 18, 2025
Let me know if this updated PR aligns better with your suggestions, @MaxGekk |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
MaxGekk
commented
Mar 20, 2025
@the-sakthi Could you update PR's description: This PR #50299 fixed the column name. |
the-sakthi
commented
Mar 20, 2025
Thanks for the review @MaxGekk , I'll shortly in few mins update the PR. Major changes:
|
the-sakthi
commented
Mar 20, 2025
Updated the revision with the suggested changes @MaxGekk ! Let me know how this one looks. |
There was a problem hiding this comment.
Moved the function usage/desc/example to the builder to fix a failing test which was complaining about this.
Uh oh!
There was an error while loading. Please reload this page.
Rebased with current main (master) branch! |
MaxGekk
commented
Mar 22, 2025
+1, LGTM. Merging to master. |
the-sakthi
commented
Mar 22, 2025
Thank you very much for helping on this one and merging @MaxGekk |
Uh oh!
There was an error while loading. Please reload this page.
… minute function ### What changes were proposed in this pull request? - Followup to the original PR: #50296 - Extend the minute(...) function (MinutesOfTime) to handle TIME types of any precision from 0 to 6. - Add tests verifying that minute(...) works for all valid TIME precisions. ### Why are the changes needed? - Previously, minute(...) did not consistently support TIME type inputs with arbitrary precision. - Users need the minute function to handle TIME(0) through TIME(6). ### Does this PR introduce _any_ user-facing change? - Yes. Users can now call minute(...) on TIME(p) columns or literals with any valid precision. ### How was this patch tested? By running new tests: ``` $ build/sbt "test:testOnly *TimeExpressionsSuite.scala" ``` By manual tests: ``` scala> spark.sql("select minute(cast('12:30' as time(0)));").show() +------------------------------+ |minute(CAST(12:30 AS TIME(0)))| +------------------------------+ | 30| +------------------------------+ scala> spark.sql("select minute(cast('12:30' as time(2)));").show() +------------------------------+ |minute(CAST(12:30 AS TIME(2)))| +------------------------------+ | 30| +------------------------------+ scala> spark.sql("select minute(cast('12:30' as time(5)));").show() +------------------------------+ |minute(CAST(12:30 AS TIME(5)))| +------------------------------+ | 30| +------------------------------+ ``` ### Was this patch authored or co-authored using generative AI tooling? No Closes#50551 from the-sakthi/SPARK-51420-FOLLOWUP. Authored-by: Sakthi <sakthi@apache.org> Signed-off-by: Max Gekk <max.gekk@gmail.com>
What changes were proposed in this pull request?
This PR adds support for extracting the minute component from TIME (TimeType) values in Spark SQL.
Why are the changes needed?
minute(TIME'HH:MM:SS.######')behaves correctly without unnecessary type coercion.Does this PR introduce any user-facing change?
Yes
minute(TIME'HH:MM:SS.######')resulted in a type mismatch error or an implicit cast attempt to TIMESTAMP, which was incorrect.minute(TIME'HH:MM:SS.######')now works correctly for TIME values without implicit casting.How was this patch tested?
By running new tests:
Was this patch authored or co-authored using generative AI tooling?
No