Skip to content

ARROW-18285: [R] Fix for failing test after lubridate 1.9 release - #14615

Merged
paleolimbot merged 1 commit into
apache:masterfrom
paleolimbot:r-fix-updated-lubridage
Nov 10, 2022
Merged

ARROW-18285: [R] Fix for failing test after lubridate 1.9 release#14615
paleolimbot merged 1 commit into
apache:masterfrom
paleolimbot:r-fix-updated-lubridage

Conversation

@paleolimbot

@paleolimbotpaleolimbot commented Nov 9, 2022

Copy link
Copy Markdown
Member

This fixes test failures resulting from an updated lubridate package, whose update removed some functionality that we supported (apparently lubridate::yq(2021.1) was a thing). This works in our source because we cast to string() before doing any further processing.

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@nealrichardson

Copy link
Copy Markdown
Member

On the current test error, it looks like some of the floats do get handled by lubridate, only 2 of the values don't parse--so is removal the right fix?

@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Ah good point, it's the years with the trailing zero:

lubridate::parse_date_time(c(3.2007, 2.1970, 1.2020, 4.2009, 1.1975, NA), "qY")
#> Warning: 2 failed to parse.#> [1] "2007-07-01 UTC" NA NA "2009-10-01 UTC"#> [5] "1975-01-01 UTC" NA

Is it worth the maintenance effort to support numeric inputs here?

@rokrok left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yq/Yq on string still seems to be a thing? https://github.com/tidyverse/lubridate/blob/main/tests/testthat/test-parsers.R#L746-L753
I think the float option is odd but don't have strong feelings on keeping/cutting it.

@rok

rok commented Nov 9, 2022

Copy link
Copy Markdown
Member

Sorry I misread the float/string part.

@nealrichardson

Copy link
Copy Markdown
Member

I agree it's odd but it does seem like lubridate does still support it. An alternative to removing the feature would be to changing the test not to have trailing zeros in the test data, since those are supported the same. I might suggest that as the quick fix for the test failures.

In terms of ongoing maintenance, maybe you could file an issue on lubridate (or check to see if one exists) and see if this is something they intend to support. We know they plan a major refactor soon, so maybe the maintainers can tell us if this is going to stay or go, and we can drop it or not based on that.

@paleolimbot
paleolimbotforce-pushed the r-fix-updated-lubridage branch from 4fe9093 to ab67fbaCompareNovember 9, 2022 19:21
@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Done! We should think about converting the integration tests to actual unit tests, moving them to extra_tests, and running them nightly rather than as part of commit-level CI/tests that run on CRAN. It's good to know when the behaviour diverges but it shouldn't cause every CI check to fail when corner-case behaviour changes elsewhere.

@paleolimbot

Copy link
Copy Markdown
MemberAuthor

I created ARROW-18285 as a follow-up to revisit this test when this has been addressed in lubridate so that we can merge this (since the test is causing a lot of CI/nightlies to fail).

@paleolimbot
paleolimbot merged commit 1e40b0a into apache:masterNov 10, 2022
@paleolimbot
paleolimbot deleted the r-fix-updated-lubridage branch November 10, 2022 17:27
@ursabot

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 6c988db and contender = 1e40b0a. 1e40b0a is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.3% ⬆️0.34%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.18% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 1e40b0ab ec2-t3-xlarge-us-east-2
[Finished] 1e40b0ab test-mac-arm
[Finished] 1e40b0ab ursa-i9-9960x
[Finished] 1e40b0ab ursa-thinkcentre-m75q
[Finished] 6c988db3 ec2-t3-xlarge-us-east-2
[Finished] 6c988db3 test-mac-arm
[Finished] 6c988db3 ursa-i9-9960x
[Finished] 6c988db3 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

kou pushed a commit that referenced this pull request Nov 15, 2022
…4615)
This fixes test failures resulting from an updated lubridate package, whose update removed some functionality that we supported (apparently `lubridate::yq(2021.1)` was a thing). This works in our source because we cast to `string()` before doing any further processing.
Authored-by: Dewey Dunnington <dewey@fishandwhistle.net>
Signed-off-by: Dewey Dunnington <dewey@fishandwhistle.net>
kou pushed a commit that referenced this pull request Feb 20, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@paleolimbot@nealrichardson@rok@ursabot