Uh oh!
There was an error while loading. Please reload this page.
ARROW-12143: [CI] R builds should timeout and fail after some threshold and dump the output. - #9846
ARROW-12143: [CI] R builds should timeout and fail after some threshold and dump the output.#9846westonpace wants to merge 10 commits into
Conversation
ARROW-12143: More changes to dump tests on timeout ARROW-12143: Adding 'Dump test logs' step to non-ubuntu builds
jonkeane
left a comment
There was a problem hiding this comment.
Thanks for this! A few suggestions for improving this.
One thing I've done elsewhere is also upload the artifacts. It's not required, but I find it nice to be able to download these and look at them in a text editor (
arrow/dev/tasks/r/github.macos-linux.local.yml
Lines 82 to 86 in 8e43f23
Could we also add the test log dumps to the templates used in crossbow too: https://github.com/apache/arrow/tree/8e43f23dcc6a9e630516228f110c48b64d13cec6/dev/tasks/r
Most of those should have the same directory structure so it should be "just" a copy paste.
| library(tibble) | ||
| test_check("arrow") | ||
| test_check("arrow", reporter="location") |
There was a problem hiding this comment.
We will probably want to make this change only happen on CI so that the standard reporter is used when we submit to CRAN. We could have this trigger on ARROW_R_DEV which is (almost?) always set on CI and isn't a terrible overloading of the meaning.
We also could use reporter = MultiReporter("check", "location") which should give us decent info in the check log, but also more detailed information for the artifact (this config basically concatenates the two reporter outputs in the output, but in reverse order)
There was a problem hiding this comment.
The environment variable check could be something like:
if (identical(tolower(Sys.getenv("ARROW_R_DEV", FALSE)), "true")) {
arrow_reporter <- MultiReporter("check", "location")
} else {
arrow_reporter <- check_reporter()
}
test_check("arrow", reporter = arrow_reporter)
| rcmdcheck::rcmdcheck(build_args = '--no-build-vignettes', args = c('--no-manual', '--ignore-vignettes', '--run-donttest'), error_on = 'warning', check_dir = 'check') | ||
| }" | ||
| echo "$SCRIPT" | ${R_BIN} --no-save | ||
| echo "$SCRIPT" | timeout -k 3600 3600 ${R_BIN} --no-save |
There was a problem hiding this comment.
I wonder if this is better or if we should add to line 74 the argument timeout = 3600 which will timeout at the check level (and will mark the build as failed if the timeout happens).
There was a problem hiding this comment.
Yes, this is much better. My initial searching on rcmdcheck didn't find any timeout arg otherwise I would have gone this route originally.
| run: cat r/check/arrow.Rcheck/00install.out | ||
| if: always() | ||
| - name: Dump test logs | ||
| run: cat r/check/arrow.Rcheck/tests/testthat.Rout |
There was a problem hiding this comment.
| run: cat r/check/arrow.Rcheck/tests/testthat.Rout | |
| run: cat r/check/arrow.Rcheck/tests/testthat.Rout* |
it turns out that if there's a test failure, the output is written to r/check/arrow.Rcheck/tests/testthat.Rout.fail
westonpace
commented
Mar 30, 2021
@jonkeane I think I've added the save test log step to all of the crossbow tasks but I'm not sure how to run those so haven't verified that at all. |
jonkeane
commented
Mar 30, 2021
@github-actions crossbow submit -g r |
jonkeane
commented
Mar 30, 2021
That should (if I haven't mis-spelled something!) run all of the r crossbow tests. You can also run a single crossbow test with |
Revision: 85a06c8 Submitted crossbow builds: ursacomputing/crossbow @ actions-250
|
jonkeane
commented
Mar 31, 2021
@github-actions crossbow submit -g r |
Thanks for adding those builds I missed. Those changes look good to me. @jonkeane |
Revision: 73641ab Submitted crossbow builds: ursacomputing/crossbow @ actions-251
|
Uh oh!
There was an error while loading. Please reload this page.
Co-authored-by: Ian Cook <ianmcook@gmail.com>
This adds a one hour timeout to the R builds. It changes the R CI test script to use
reporter="location"by default. It adds a dump test logs step to the end of the build that will always dump the test output regardless of success/failure.These three changes combined will make it much easier to debug test failures in R tests.