Skip to content

ARROW-16439: [R] Implement binding for lubridate::fast_strptime - #13174

Closed
dragosmg wants to merge 8 commits into
apache:masterfrom
dragosmg:fast_strptime_binding
Closed

ARROW-16439: [R] Implement binding for lubridate::fast_strptime#13174
dragosmg wants to merge 8 commits into
apache:masterfrom
dragosmg:fast_strptime_binding

Conversation

@dragosmg

@dragosmgdragosmg commented May 17, 2022

Copy link
Copy Markdown
Contributor

This PR adds:

  • a binding emulating lubridate's fast_strptime functionality

This PR does not support the following arguments:

  • lt = TRUE - this returns a POSIXlt object (a list) which cannot be easily used in a dplyr pipeline (currently it actually errors) => we have a different default lt = FALSE for the Arrow binding, and
  • cutoff_2000 = 68L - for the y% format two-digit numbers smaller or equal to cutoff_2000 are parsed as though starting with 20, otherwise parsed as though starting with 19. It would be nice to have this, so I raised ARROW-16596. We can always suggest users they manipulate the strings before parsing, so I don't think this is crucial functionality.

The following code will be possible once the PR is merged:

library(lubridate, warn.conflicts=FALSE)
library(dplyr, warn.conflicts=FALSE)
library(arrow, warn.conflicts=FALSE)
dates_table<- tibble(
string_with_short_year= c("68-05-17", "69-05-17", "55-05-17")
)
dates_table %>% mutate(
date= fast_strptime(
string_with_short_year, format="%y-%m-%d",
lt=FALSE
)
)
#> # A tibble: 3 × 2#> string_with_short_year date #> <chr> <dttm> #> 1 68-05-17 2068-05-17 00:00:00#> 2 69-05-17 1969-05-17 00:00:00#> 3 55-05-17 2055-05-17 00:00:00dates_table %>% arrow_table() %>% mutate(
date= fast_strptime(
string_with_short_year, format="%y-%m-%d",
lt=FALSE
)
) %>%
collect()
#> # A tibble: 3 × 2#> string_with_short_year date #> <chr> <dttm> #> 1 68-05-17 2068-05-17 00:00:00#> 2 69-05-17 1969-05-17 00:00:00#> 3 55-05-17 2055-05-17 00:00:00

Created on 2022-05-18 by the reprex package (v2.0.1)

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@dragosmg
dragosmg marked this pull request as ready for review May 17, 2022 20:25
Comment threadr/R/dplyr-funcs-datetime.R
Comment threadr/R/dplyr-funcs-datetime.R
Comment threadr/tests/testthat/test-dplyr-funcs-datetime.R

@thisisnicthisisnic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@dragosmgdragosmg changed the title ARROW-16439: [R] Implement bindings for lubridate's fast_strptimeARROW-16439: [R] Implement bindings for lubridate's fast_strptimeMay 18, 2022
@dragosmgdragosmg changed the title ARROW-16439: [R] Implement bindings for lubridate's fast_strptimeARROW-16439: [R] Implement binding for lubridate::fast_strptimeMay 18, 2022
Comment threadr/R/dplyr-funcs-datetime.R
@dragosmg
dragosmg requested a review from jonkeaneMay 19, 2022 08:41
@dragosmg

dragosmg commented May 19, 2022

Copy link
Copy Markdown
ContributorAuthor

There is a slight problem though - in the way the fast_strptime binding parses a string like "68-12-12 12:34:56" when the format is "%Y..." (incorrect) and not "%y..." (correct).
base R and lubridate behaviour (currently the fast_strptime binding behaves like base R - i.e. doesn't error, but parses a short year xy to 00xy):

b<-"68-10-07 19:04:0"
strptime(b, format="%Y-%m-%d %H:%M:%S")
#> [1] "0068-10-07 19:04:00 LMT"
strptime(b, format="%y-%m-%d %H:%M:%S")
#> [1] "2068-10-07 19:04:00 BST"lubridate::fast_strptime(b, format="%Y-%m-%d %H:%M:%S")
#> [1] NAlubridate::fast_strptime(b, format="%y-%m-%d %H:%M:%S")
#> [1] "2068-10-07 19:04:00 UTC"

Created on 2022-05-19 by the reprex package (v2.0.1)

This could trip users up (mostly when they pass multiple formats):

library(arrow, warn.conflicts=FALSE)
library(lubridate, warn.conflicts=FALSE)
library(dplyr, warn.conflicts=FALSE)
dates_table<- tibble(
string_with_short_year= c("68-05-17", "69-05-17", "55-05-17")
)
dates_table %>% mutate(
date= fast_strptime(
string_with_short_year, format= c("%Y-%m-%d", "%y-%m-%d"),
lt=TRUE
)
)
#> # A tibble: 3 × 2#> string_with_short_year date #> <chr> <dttm> #> 1 68-05-17 2068-05-17 00:00:00#> 2 69-05-17 1969-05-17 00:00:00#> 3 55-05-17 2055-05-17 00:00:00dates_table %>% arrow_table() %>% mutate(
date= fast_strptime(
string_with_short_year, format= c("%Y-%m-%d", "%y-%m-%d"),
lt=FALSE
)
) %>% collect()
#> # A tibble: 3 × 2#> string_with_short_year date #> <chr> <dttm> #> 1 68-05-17 0068-05-17 00:00:00#> 2 69-05-17 0069-05-17 00:00:00#> 3 55-05-17 0055-05-17 00:00:00

Created on 2022-05-19 by the reprex package (v2.0.1)

What happens here 👆🏻 is that lubridate::fast_strptime() fails on the first format and moves on the the second, while the arrow binding doesn't.

I flagged this as part of ARROW-16596. Not quite sure how to go about handling that only on the R side though. arrow's strptime defers to the platform's strptime so we might see different behaviours on different platforms too.

@amol-amol- closed this in 7f31c9dMay 19, 2022

@jonkeanejonkeane left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last cleanup

Comment on lines +1903 to +1905
)#,
# arrow does not preserve the `tzone` attribute
# test ignore_attr = TRUE

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove these commented lines, yeah?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I yes, sorry. Forgot about those. Do I open a minor PR?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should I just do it in one of the other PR I have going?

@dragosmgdragosmgMay 19, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonkeane I removed the tests in dragosmg@bde85ce

@ursabot

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 8394571 and contender = 7f31c9d. 7f31c9d is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.74% ⬆️0.0%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.04% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 7f31c9d2 ec2-t3-xlarge-us-east-2
[Failed] 7f31c9d2 test-mac-arm
[Failed] 7f31c9d2 ursa-i9-9960x
[Finished] 7f31c9d2 ursa-thinkcentre-m75q
[Finished] 83945714 ec2-t3-xlarge-us-east-2
[Failed] 83945714 test-mac-arm
[Failed] 83945714 ursa-i9-9960x
[Finished] 83945714 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

dragosmg added a commit to dragosmg/arrow that referenced this pull request Jun 20, 2022
kou pushed a commit that referenced this pull request Feb 20, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@dragosmg@ursabot@jonkeane@thisisnic