Skip to content

[GLUTEN-12538][VL] Support cast between DATE and TIMESTAMP_NTZ - #12966

Open
felipepessoto wants to merge 2 commits into
apache:mainfrom
felipepessoto:gluten-12538-date-timestamp-ntz-cast
Open

[GLUTEN-12538][VL] Support cast between DATE and TIMESTAMP_NTZ#12966
felipepessoto wants to merge 2 commits into
apache:mainfrom
felipepessoto:gluten-12538-date-timestamp-ntz-cast

Conversation

@felipepessoto

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Allow casts between DATE and TIMESTAMP_NTZ to pass Gluten's Substrait-to-Velox validation now that native support is available in facebookincubator/velox#17961 and included by #12961.

Add native execution coverage for both cast directions, including DATE to TIMESTAMP_NTZ under a non-UTC session timezone.

This addresses the DATE/TIMESTAMP_NTZ case discussed in #12538 and is related to #11622.

How was this patch tested?

  • Rebuilt the pinned Velox dependency and Gluten native backend with tests enabled.
  • cpp/build/velox/tests/velox_plan_conversion_test --gtest_filter='Substrait2VeloxPlanValidatorTest.*'
  • ./build/mvn test -Pspark-ut -Pbackends-velox -Pspark-3.5 -Pjava-17 -DargLine="-Dspark.test.home=/opt/shims/spark35/spark_home/" -DwildcardSuites=org.apache.gluten.functions.DateFunctionsValidateSuite

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot CLI 1.0.82

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@felipepessoto

Copy link
Copy Markdown
ContributorAuthor

/delta-test

@github-actions

Copy link
Copy Markdown

🔄 Delta Spark UT started by @felipepessoto (~2.5 h). View run

@felipepessoto
felipepessoto marked this pull request as ready for review September 4, 2026 20:06
CopilotAI lite review requested due to automatic review settings September 4, 2026 20:06
@felipepessoto

Copy link
Copy Markdown
ContributorAuthor

@Mariamalmesfer@rui-mo I think you two have worked on other timestamp_ntz PRs. Could you take a look, please?

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is small and localized, and it adds explicit native execution test coverage for both cast directions.

Pull request overview

This PR enables Velox backend offload validation for casts between DATE and TIMESTAMP_NTZ (Substrait precision_timestamp) now that native support is available, and adds Spark UT coverage to ensure both cast directions execute natively (including a non-UTC session timezone case for DATE -> TIMESTAMP_NTZ).

Changes:

  • Extend SubstraitToVeloxPlanValidator::isAllowedCast to permit TIMESTAMP_NTZ <-> DATE casts during Substrait-to-Velox validation.
  • Add Spark SQL validation tests to ensure native execution for cast(timestamp_ntz as date) and cast(date as timestamp_ntz).
  • Add a non-UTC session timezone coverage case for DATE -> TIMESTAMP_NTZ native execution.
File summaries
FileDescription
cpp/velox/substrait/SubstraitToVeloxPlanValidator.ccAllows DATE casts to/from TIMESTAMP_NTZ in the native plan validator.
backends-velox/src/test/scala/org/apache/gluten/functions/DateFunctionsValidateSuite.scalaAdds native execution tests for both cast directions, including a non-UTC timezone case.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
CopilotAI review requested due to automatic review settings September 4, 2026 20:23

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped to cast validation plus targeted UT coverage, and no correctness or maintainability issues were found in the modified hunks.

Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@felipepessoto

Copy link
Copy Markdown
ContributorAuthor

/delta-test

@github-actions

Copy link
Copy Markdown

🔄 Delta Spark UT started by @felipepessoto (~2.5 h). View run

@felipepessoto

Copy link
Copy Markdown
ContributorAuthor

Delta CI is green

@Smallfu666

Copy link
Copy Markdown
Contributor

I rebuilt against the correct Velox pin and verified both cast directions under non-UTC session timezones. The results match Spark and the change LGTM. Thanks!

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@felipepessoto@Smallfu666