Skip to content

ARROW-16415: [R] Update strptime binding signature with the tz argument - #13190

Merged
jonkeane merged 12 commits into
apache:masterfrom
dragosmg:strptime_tz_update
Jun 7, 2022
Merged

ARROW-16415: [R] Update strptime binding signature with the tz argument#13190
jonkeane merged 12 commits into
apache:masterfrom
dragosmg:strptime_tz_update

Conversation

@dragosmg

@dragosmgdragosmg commented May 18, 2022

Copy link
Copy Markdown
Contributor

This PR enables the tz argument for the strptime binding by using the assume_timezone compute function.

Current behaviour:

library(arrow, warn.conflicts=FALSE)
library(dplyr, warn.conflicts=FALSE)
test_df<-tibble::tibble(
timestamp_naive_string= c("2018-10-07 19:04:05", NA)
)
test_df %>% arrow_table() %>% mutate(
timestamp= strptime(
timestamp_naive_string, format="%Y-%m-%d %H:%M:%S", tz="Pacific/Marquesas"
)
) %>%
collect()
#> Warning: In strptime(timestamp_naive_string, format = "%Y-%m-%d %H:%M:%S", ...,#> Time zone argument not supported in Arrow; pulling data into R#> # A tibble: 2 × 2#> timestamp_naive_string timestamp #> <chr> <dttm> #> 1 2018-10-07 19:04:05 2018-10-07 19:04:05#> 2 <NA> NA

Created on 2022-05-18 by the reprex package (v2.0.1)

Future/desired behaviour:

library(arrow, warn.conflicts=FALSE)
library(dplyr, warn.conflicts=FALSE)
test_df<-tibble::tibble(
timestamp_naive_string= c("2018-10-07 19:04:05", NA)
)
a<-test_df %>% arrow_table() %>% mutate(
timestamp= strptime(
timestamp_naive_string, format="%Y-%m-%d %H:%M:%S", tz="Pacific/Marquesas"
)
) %>%
collect()
a#> # A tibble: 2 × 2#> timestamp_naive_string timestamp #> <chr> <dttm> #> 1 2018-10-07 19:04:05 2018-10-07 19:04:05#> 2 <NA> NA
attributes(a$timestamp)
#> $class#> [1] "POSIXlt" "POSIXt" #> #> $tzone#> [1] "Pacific/Marquesas"

Created on 2022-05-19 by the reprex package (v2.0.1)

@github-actions

Copy link
Copy Markdown

@dragosmgdragosmg changed the title ARROW-16415: [R] Update strptime bindings to use tzARROW-16415: [R] Update strptime binding signature with the tz argumentMay 18, 2022
@dragosmg
dragosmg marked this pull request as ready for review May 18, 2022 20:53

@thisisnicthisisnic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might have some issues where the default argument of tz in strptime() in R is different to the one here. I tried running this test I wrote, but got an error:

> t_string %>%
+ arrow_table() %>%
+ mutate(
+ x = strptime(x, format = "%Y-%m-%d %H:%M:%S", tz = "")
+ ) %>%
+ collect() -> out
Error in `collect()`:
! Invalid: Cannot locate timezone '': not found in timezone database
/home/nic2/arrow/cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc:128 LocateZone(options.timezone)
/home/nic2/arrow/cpp/src/arrow/compute/exec.cc:701 kernel_->exec(kernel_ctx_, batch, &out)
/home/nic2/arrow/cpp/src/arrow/compute/exec.cc:642 ExecuteBatch(batch, listener)
/home/nic2/arrow/cpp/src/arrow/compute/exec/expression.cc:597 executor->Execute(arguments, &listener)
/home/nic2/arrow/cpp/src/arrow/compute/exec/project_node.cc:90 ExecuteScalarExpression(simplified_expr, target, plan()->exec_context())
/home/nic2/arrow/cpp/src/arrow/compute/exec/exec_plan.cc:467 iterator_.Next()
/home/nic2/arrow/cpp/src/arrow/record_batch.cc:337 ReadNext(&batch)
/home/nic2/arrow/cpp/src/arrow/record_batch.cc:351 ToRecordBatches()

@dragosmg
dragosmg requested a review from thisisnicMay 25, 2022 12:41
Comment threadr/tests/testthat/test-dplyr-funcs-datetime.R

@thisisnicthisisnic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the comment it'd be great to see added, I think this is good to go. @jonkeane - mind taking a look when you have a chance, in case I've missed anything?

@thisisnic
thisisnic requested a review from jonkeaneMay 26, 2022 11:11

@jonkeanejonkeane left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good — a comment about an additional test + a few more comments that would be helpful

Comment threadr/tests/testthat/test-dplyr-funcs-datetime.R
Comment threadr/R/dplyr-funcs-datetime.R
@dragosmg
dragosmg requested a review from jonkeaneJune 6, 2022 09:42
Comment threadr/R/dplyr-funcs-datetime.R Outdated
@dragosmg
dragosmg requested a review from jonkeaneJune 7, 2022 09:02
Comment threadr/tests/testthat/test-dplyr-funcs-datetime.R

@jonkeanejonkeane left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for sticking with it

kou pushed a commit that referenced this pull request Feb 20, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@dragosmg@jonkeane@thisisnic