Timezone aware extract SQL expression by codetyri0n · Pull Request #18990 · apache/datafusion

codetyri0n · 2025-11-29T11:28:45Z

Which issue does this PR close?

Closes#18228

What changes are included in this PR?

timezone config options support and subsequent results produced aware of the set timezone

Are these changes tested?

slt

Are there any user-facing changes?

yes

codetyri0n · 2025-11-29T11:32:45Z

 pub mod to_timestamp;
 pub mod to_unixtime;

+// Adjusts a timestamp to local time by applying the timezone offset.


adjust_to_local_time refactored to mod.rs as a common helper function for both date_part and to_local_time
CC: @martin-g

Might want to put in common.rs instead.

codetyri0n · 2025-11-29T11:34:06Z

+ EXTRACT(SECOND FROM TIMESTAMP '2024-01-01 03:05:59+08:45');
+----
+2023 12 31 18 20 59
+


tested with postive and negative offset timezones such as Asia/Kolkata and America/New_York

codetyri0n · 2025-11-29T11:38:41Z

Really sorry that the previous PR was closed because I forgot to git add via the IDE before force pushing the conflicts that were resolved with main 😭 💀

BUT HOWEVER:

all previous comments and objective of the issue are addressed with this and hope it is fit for approval/merge (pending latest comments, if any)

@Omega359 @martin-g

codetyri0n · 2025-11-29T14:44:06Z

thank you for the improvements! @martin-g

Omega359 · 2025-11-29T15:29:34Z

I'll try and do a full review of this today or tomorrow, thanks for the work on it

Omega359 · 2025-12-02T14:09:22Z

Apologies on the review delay though I hope to get to it in the next few days - honest!

Omega359 · 2025-12-02T18:00:50Z

+}
+
 fn is_epoch(part: &str) -> bool {
 let part = part_normalization(part);


This work seems to be done twice - once before this function is called and once in the function. Perhaps we should just remove this part_normalization() call and pass the result of the previous call on line 209.

Omega359 · 2025-12-02T21:37:14Z

 pub mod to_timestamp;
 pub mod to_unixtime;

+// Adjusts a timestamp to local time by applying the timezone offset.


Might want to put in common.rs instead.

Omega359 · 2025-12-03T18:29:20Z

I believe this is ready to be merged, thanks! I'll try and find a committer to do the last review and put it into the merge queue.

Omega359 · 2025-12-12T03:27:39Z

@alamb @comphead - would either of you have the time to take a look at this PR and merge it if you think it's acceptable?

alamb · 2025-12-16T14:17:28Z

I'll try and review this in the next day or two.

@codetyri0n there appears to be some conflicts with this PR -- could you please resolve them?

codetyri0n · 2025-12-22T12:05:26Z

I'll try and review this in the next day or two.
@codetyri0n there appears to be some conflicts with this PR -- could you please resolve them?

hey @alamb , I've resolved the conflicts a few days back (icymi!)

Omega359 · 2026-01-09T15:06:57Z

@codetyri0n I see there is conflicts on this PR now - would you be able to resolve those?

Omega359 · 2026-01-11T16:49:49Z

Still failing ci tests ...

codetyri0n · 2026-01-11T17:20:47Z

sorry for this... ill resolve it properly first thing tomorrow

codetyri0n · 2026-01-12T21:32:04Z

after a series of conflict resolving clippy tells me this is unused, any idea @Omega359 ?

Omega359 · 2026-01-20T22:27:26Z

after a series of conflict resolving clippy tells me this is unused, any idea @Omega359 ?

I'll try and take a look this week but I'm rather swamped at the moment with non-df related work :(

Omega359 · 2026-01-31T21:31:08Z

+ // Get the timezone offset for this datetime
+ let tz_offset = tz.offset_from_utc_datetime(&date_time.naive_utc());
+ // Convert offset to seconds - offset is formatted like "+01:00" or "-05:00"
+ let offset_str = format!("{tz_offset}");


When working on merging updates from main I noticed that this function in to_local_time is much simplier than this:
let offset_seconds:i64 = tz .offset_from_utc_datetime(&date_time.naive_utc()).fix().local_minus_utc()asi64;
Did this function come from somewhere else or did you make this change for some other reason?

Submitted a PR to your branch @codetyri0n

Omega359 · 2026-02-10T19:10:52Z

@codetyri0n are you still willing to continue working on this to help push it across the line? If not I think I'll submit another PR based on the work done for this comment.

codetyri0n · 2026-02-11T14:25:02Z

this comment

Sorry, haven't had bandwidth to be involved lately! I can look at this during the weekend although if you can/would be willing to do the needful earlier please feel free to do so! : )

github-actions · 2026-04-13T02:16:30Z

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

github-actionsBot added documentation Improvements or additions to documentation sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Nov 29, 2025

codetyri0n commented Nov 29, 2025
View reviewed changes

martin-g reviewed Nov 29, 2025
View reviewed changes

Comment threaddatafusion/functions/src/datetime/date_part.rs Outdated
Comment threaddatafusion/functions/src/datetime/date_part.rs Outdated

Omega359 reviewed Dec 2, 2025
View reviewed changes

codetyri0n force-pushed the extract_withtz branch from d38d0b2 to ad2beefCompare December 3, 2025 17:17

codetyri0n requested a review from Omega359 December 3, 2025 17:50

codetyri0n mentioned this pull request Dec 5, 2025
[Docs] Update SQL examples in documentation for timestamp/datetime functions #19104
Closed

alamb mentioned this pull request Dec 16, 2025
Andrew Lamb Weekly-ish Open Source plan - 2025-12-08 #19210
Closed
40 tasks

alamb mentioned this pull request Jan 5, 2026
Andrew Lamb Weekly-ish Open Source plan - 2026-01-05 #19652
Closed
44 tasks

codetyri0n force-pushed the extract_withtz branch from 15bf77a to 0794846Compare January 10, 2026 20:40

codetyri0n force-pushed the extract_withtz branch 2 times, most recently from d2f0dc6 to e80984dCompare January 12, 2026 20:03

codetyri0n added 4 commits January 13, 2026 02:14

resolve conflicts with main
400e4c5

make extract timezone aware
a3d9e53

cargo fmt
b488c1a

refactoring and setting volatility to stable
1ff4726

codetyri0n added 4 commits January 13, 2026 02:27

minor refactoring and test addition
7ae1503

resolve conflicts
e34d403

resolve conflicts
b4d206e

resolve conflicts
8123ac6

resolve conflicts (ii)
b008a2a

codetyri0n force-pushed the extract_withtz branch from e80984d to b008a2aCompare January 13, 2026 18:38

Omega359 reviewed Jan 31, 2026
View reviewed changes

github-actionsBot added the Stale PR has not had any activity for some time label Apr 13, 2026

github-actionsBot closed this Apr 21, 2026

Uh oh!

Conversation

codetyri0n commented Nov 29, 2025

Which issue does this PR close?

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

codetyri0nNov 29, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Omega359Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

codetyri0nNov 29, 2025

Choose a reason for hiding this comment

Uh oh!

codetyri0n commented Nov 29, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codetyri0n commented Nov 29, 2025

Uh oh!

Omega359 commented Nov 29, 2025

Uh oh!

Omega359 commented Dec 2, 2025

Uh oh!

Omega359Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Omega359Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Omega359 commented Dec 3, 2025

Uh oh!

Omega359 commented Dec 12, 2025

Uh oh!

alamb commented Dec 16, 2025

Uh oh!

codetyri0n commented Dec 22, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Omega359 commented Jan 9, 2026

Uh oh!

Omega359 commented Jan 11, 2026

Uh oh!

codetyri0n commented Jan 11, 2026

Uh oh!

codetyri0n commented Jan 12, 2026

Uh oh!

Omega359 commented Jan 20, 2026

Uh oh!

Omega359Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Omega359Jan 31, 2026

Choose a reason for hiding this comment

Uh oh!

Omega359 commented Feb 10, 2026

Uh oh!

codetyri0n commented Feb 11, 2026

Uh oh!

github-actionsBot commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codetyri0nNov 29, 2025•
edited
Loading

codetyri0n commented Nov 29, 2025•
edited
Loading

codetyri0n commented Dec 22, 2025•
edited
Loading