Uh oh!
There was an error while loading. Please reload this page.
[GLUTEN-12538][VL] Support cast between DATE and TIMESTAMP_NTZ - #12966
[GLUTEN-12538][VL] Support cast between DATE and TIMESTAMP_NTZ#12966felipepessoto wants to merge 2 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
felipepessoto
commented
Sep 3, 2026
/delta-test |
🔄 Delta Spark UT started by @felipepessoto (~2.5 h). View run |
felipepessoto
commented
Sep 4, 2026
@Mariamalmesfer@rui-mo I think you two have worked on other timestamp_ntz PRs. Could you take a look, please? |
There was a problem hiding this comment.
🟢 Approval recommended
The change is small and localized, and it adds explicit native execution test coverage for both cast directions.
Pull request overview
This PR enables Velox backend offload validation for casts between DATE and TIMESTAMP_NTZ (Substrait precision_timestamp) now that native support is available, and adds Spark UT coverage to ensure both cast directions execute natively (including a non-UTC session timezone case for DATE -> TIMESTAMP_NTZ).
Changes:
- Extend
SubstraitToVeloxPlanValidator::isAllowedCastto permitTIMESTAMP_NTZ <-> DATEcasts during Substrait-to-Velox validation. - Add Spark SQL validation tests to ensure native execution for
cast(timestamp_ntz as date)andcast(date as timestamp_ntz). - Add a non-UTC session timezone coverage case for
DATE -> TIMESTAMP_NTZnative execution.
File summaries
| File | Description |
|---|---|
| cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc | Allows DATE casts to/from TIMESTAMP_NTZ in the native plan validator. |
| backends-velox/src/test/scala/org/apache/gluten/functions/DateFunctionsValidateSuite.scala | Adds native execution tests for both cast directions, including a non-UTC timezone case. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟢 Approval recommended
The change is narrowly scoped to cast validation plus targeted UT coverage, and no correctness or maintainability issues were found in the modified hunks.
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
- Review effort level: Lite
felipepessoto
commented
Sep 5, 2026
/delta-test |
🔄 Delta Spark UT started by @felipepessoto (~2.5 h). View run |
felipepessoto
commented
Sep 5, 2026
Delta CI is green |
Smallfu666
commented
Sep 5, 2026
I rebuilt against the correct Velox pin and verified both cast directions under non-UTC session timezones. The results match Spark and the change LGTM. Thanks! |
What changes are proposed in this pull request?
Allow casts between
DATEandTIMESTAMP_NTZto pass Gluten's Substrait-to-Velox validation now that native support is available in facebookincubator/velox#17961 and included by #12961.Add native execution coverage for both cast directions, including
DATEtoTIMESTAMP_NTZunder a non-UTC session timezone.This addresses the DATE/TIMESTAMP_NTZ case discussed in #12538 and is related to #11622.
How was this patch tested?
cpp/build/velox/tests/velox_plan_conversion_test --gtest_filter='Substrait2VeloxPlanValidatorTest.*'./build/mvn test -Pspark-ut -Pbackends-velox -Pspark-3.5 -Pjava-17 -DargLine="-Dspark.test.home=/opt/shims/spark35/spark_home/" -DwildcardSuites=org.apache.gluten.functions.DateFunctionsValidateSuiteWas this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.82