WIP: [Release] Verify release-10.0.1-rc0 by kou · Pull Request #14654 · apache/arrow

kou · 2022-11-16T00:22:14Z

PR to verify Release Candidate

kou · 2022-11-16T00:22:15Z

@github-actions crossbow submit --group verify-rc-source --param release=10.0.1 --param rc=0

github-actions · 2022-11-16T00:28:52Z

Revision: a6eabc2

Submitted crossbow builds: ursacomputing/crossbow @ actions-36c520c6c6

Task	Status
verify-rc-source-cpp-linux-almalinux-8-amd64
verify-rc-source-cpp-linux-conda-latest-amd64
verify-rc-source-cpp-linux-ubuntu-18.04-amd64
verify-rc-source-cpp-linux-ubuntu-20.04-amd64
verify-rc-source-cpp-linux-ubuntu-22.04-amd64
verify-rc-source-cpp-macos-amd64
verify-rc-source-cpp-macos-arm64
verify-rc-source-cpp-macos-conda-amd64
verify-rc-source-csharp-linux-almalinux-8-amd64
verify-rc-source-csharp-linux-conda-latest-amd64
verify-rc-source-csharp-linux-ubuntu-18.04-amd64
verify-rc-source-csharp-linux-ubuntu-20.04-amd64
verify-rc-source-csharp-linux-ubuntu-22.04-amd64
verify-rc-source-csharp-macos-amd64
verify-rc-source-csharp-macos-arm64
verify-rc-source-go-linux-almalinux-8-amd64
verify-rc-source-go-linux-conda-latest-amd64
verify-rc-source-go-linux-ubuntu-18.04-amd64
verify-rc-source-go-linux-ubuntu-20.04-amd64
verify-rc-source-go-linux-ubuntu-22.04-amd64
verify-rc-source-go-macos-amd64
verify-rc-source-go-macos-arm64
verify-rc-source-integration-linux-almalinux-8-amd64
verify-rc-source-integration-linux-conda-latest-amd64
verify-rc-source-integration-linux-ubuntu-18.04-amd64
verify-rc-source-integration-linux-ubuntu-20.04-amd64
verify-rc-source-integration-linux-ubuntu-22.04-amd64
verify-rc-source-integration-macos-amd64
verify-rc-source-integration-macos-arm64
verify-rc-source-integration-macos-conda-amd64
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-18.04-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64
verify-rc-source-js-linux-almalinux-8-amd64
verify-rc-source-js-linux-conda-latest-amd64
verify-rc-source-js-linux-ubuntu-18.04-amd64
verify-rc-source-js-linux-ubuntu-20.04-amd64
verify-rc-source-js-linux-ubuntu-22.04-amd64
verify-rc-source-js-macos-amd64
verify-rc-source-js-macos-arm64
verify-rc-source-python-linux-almalinux-8-amd64
verify-rc-source-python-linux-conda-latest-amd64
verify-rc-source-python-linux-ubuntu-18.04-amd64
verify-rc-source-python-linux-ubuntu-20.04-amd64
verify-rc-source-python-linux-ubuntu-22.04-amd64
verify-rc-source-python-macos-amd64
verify-rc-source-python-macos-arm64
verify-rc-source-python-macos-conda-amd64
verify-rc-source-ruby-linux-almalinux-8-amd64
verify-rc-source-ruby-linux-conda-latest-amd64
verify-rc-source-ruby-linux-ubuntu-18.04-amd64
verify-rc-source-ruby-linux-ubuntu-20.04-amd64
verify-rc-source-ruby-linux-ubuntu-22.04-amd64
verify-rc-source-ruby-macos-amd64
verify-rc-source-ruby-macos-arm64
verify-rc-source-windows

kou · 2022-11-16T00:35:09Z

Revision: a6eabc2

Submitted crossbow builds: ursacomputing/crossbow @ release-10.0.1-rc0-0

Task	Status
almalinux-8-amd64
almalinux-8-arm64
almalinux-9-amd64
almalinux-9-arm64
amazon-linux-2-amd64
amazon-linux-2-arm64
centos-7-amd64
centos-8-stream-amd64
centos-8-stream-arm64
centos-9-stream-amd64
centos-9-stream-arm64
conan-maximum
conan-minimum
debian-bookworm-amd64
debian-bookworm-arm64
debian-bullseye-amd64
debian-bullseye-arm64
java-jars
nuget
python-sdist
r-binary-packages
test-ubuntu-default-docs
ubuntu-bionic-amd64
ubuntu-bionic-arm64
ubuntu-focal-amd64
ubuntu-focal-arm64
ubuntu-jammy-amd64
ubuntu-jammy-arm64
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-mojave-cp310-amd64
wheel-macos-mojave-cp311-amd64
wheel-macos-mojave-cp37-amd64
wheel-macos-mojave-cp38-amd64
wheel-macos-mojave-cp39-amd64
wheel-manylinux2014-cp310-amd64
wheel-manylinux2014-cp310-arm64
wheel-manylinux2014-cp311-amd64
wheel-manylinux2014-cp311-arm64
wheel-manylinux2014-cp37-amd64
wheel-manylinux2014-cp37-arm64
wheel-manylinux2014-cp38-amd64
wheel-manylinux2014-cp38-arm64
wheel-manylinux2014-cp39-amd64
wheel-manylinux2014-cp39-arm64
wheel-windows-cp310-amd64
wheel-windows-cp311-amd64
wheel-windows-cp37-amd64
wheel-windows-cp38-amd64
wheel-windows-cp39-amd64

kou · 2022-11-16T03:38:05Z

@nealrichardson @paleolimbot @thisisnic r-binary-packages failed. Could you confirm this?

Attaching package: ‘arrow’
The following object is masked from ‘package:utils’:
timestamp
Error in parquet___arrow___ArrowReaderProperties__Make(isTRUE(use_threads)) : Cannot call parquet___arrow___ArrowReaderProperties__Make(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries. Calls: read_parquet ... <Anonymous> -> parquet___arrow___ArrowReaderProperties__Make
Execution halted

Note that the same error is happen on master too: https://github.com/ursacomputing/crossbow/actions/runs/3468776645/jobs/5795178913

thisisnic · 2022-11-16T12:32:20Z

Weird, that error is triggered when the C++ library hasn't been built with enough functionality to support the R package, so I'd expect it to be something odd happening with the CI rather than a code change. I've re-triggered the job to make sure it's not some transient issue.

thisisnic · 2022-11-16T12:34:43Z

Hmm, failed again almost immediately: https://github.com/ursacomputing/crossbow/actions/runs/3475288883/jobs/5817793906

I'm not sure what's going on there. @assignUser - anything jump out at you there?

assignUser · 2022-11-16T13:19:48Z

@thisisnic seems to be an issue with the rocker image overriding the repo settings and forcefull installing the packages via apt and thus installing r-cran-arrow instead of the local package: https://github.com/ursacomputing/crossbow/actions/runs/3475288883/jobs/5817793906#step:13:321

We might need/want to replace that image to stop that. Should not matter for this release though 👍

kou · 2022-11-16T19:55:01Z

We might need/want to replace that image to stop that. Should not matter for this release though +1

If the test is failed, R binaries aren't uploaded: https://github.com/ursacomputing/crossbow/actions/runs/3475288883/jobs/5809989647

Is it OK for R? If it's OK, I'll continue with the failure.

eitsupi · 2022-11-16T23:19:04Z

seems to be an issue with the rocker image overriding the repo settings and forcefull installing the packages via apt and thus installing r-cran-arrow instead of the local package: ursacomputing/crossbow/actions/runs/3475288883/jobs/5817793906#step:13:321

This seems to be due to rocker/r-buntu:22.04 configuring bspm.
bspm replaces the behavior of the install.packages function and sets apt to install the R packages.
I recommend that you simply stop using this image.

@eddelbuettel Did you push the image to DockerHub without updating the Dockerfile on GitHub?
It seems that bspm is indeed installed on rocker/r-ubuntu@sha256:68f72a15f0e972556fb3dfd80b6394c34315722cb9b3e7a3a917a869152f7c40 even though the Dockerfile shows that bspm is not installed.

eddelbuettel · 2022-11-16T23:27:11Z

@eitsupi If so then only by accident. But I did 'batch' update a few (layered) containers following the R 4.2.2 release and may inadvertendly (by doing this manually) crossed 'tags' and Dockerfiles. Building and pushing a new image '22.04' (also as 'jammy') now. And yes, you are correct: bspm should not be in this base layer but one above.

kou · 2022-11-16T23:29:20Z

@eitsupi Thanks for the information!

Do you know Docker images that can be replaced with https://github.com/apache/arrow/blob/master/dev/tasks/r/github.packages.yml#L247-L250 ?

eitsupi · 2022-11-16T23:45:29Z

Do you know Docker images that can be replaced with master/dev/tasks/r/github.packages.yml#L247-L250 ?

I think rocker/r-ubuntu:22.04 can be replaced by rstudio/r-base:jammy.
As @eddelbuettel says, this change in behavior seems to have been an accident, so I think it can be left as is.
(However, I think the build system needs to be reviewed, as it is not good for this type of accident to happen.)

As for rocker/r-ver, it also should be possible to replace it with the image provided by rstudio (Posit).
https://github.com/rstudio/r-docker

However, the update mechanism of rocker/r-ver that I maintain has been fully automated on GitHub for over a year now, so I don't think an accident like this will happen.
https://github.com/rocker-org/rocker-versioned2

eddelbuettel · 2022-11-16T23:46:38Z

@kou "It's complicated." @eitsupi and I are both behind Rocker, and these choices present two different and not exactly interchangeable options with different settings. So in two characters: "no". In more words: "it depends, they both have their (different) use cases". I do all my CI with r-ubuntu / r-bspm / r2u as I want current toolchains and current CRAN easily. YMMV.

eddelbuettel · 2022-11-16T23:48:11Z

in behavior seems to have been an accident,

And if so, has been fixed. Did you try the container I uploaded ~20 mins ago?

assignUser · 2022-11-16T23:56:16Z

And if so, has been fixed. Did you try the container I uploaded ~20 mins ago?

I'll re-run the job

eitsupi · 2022-11-16T23:56:46Z

@eddelbuettel

And if so, has been fixed. Did you try the container I uploaded ~20 mins ago?

I pulled from DockerHub, but the digest does not seem to have changed.

It seems that bspm is indeed installed on rocker/r-ubuntu@sha256:68f72a15f0e972556fb3dfd80b6394c34315722cb9b3e7a3a917a869152f7c40 even though the Dockerfile shows that bspm is not installed.

assignUser · 2022-11-16T23:58:29Z

Indeed: https://github.com/ursacomputing/crossbow/actions/runs/3468776645/jobs/5828306179#step:13:321

eddelbuettel · 2022-11-17T00:11:28Z

My bad, in the rush I did

 2157 docker build --no-cache -t rocker/r-ubuntu:jammy .
2158 #docker build --no-cache -t rocker/r-ubuntu:jammy .
2159 docker push rocker/r-ubuntu:jammy 2160 docker push rocker/r-ubuntu:22.04

which replaced one but not both of these. I skipped the build step for 22,04

Will ship and update 22.04 tag in a minute or two -- now building.

eddelbuettel · 2022-11-17T00:24:54Z

Rebuilt and repushed. I used --no-cache on both hence different sha1 (which confused me for a second so I did a rebuild or two). They should both work are identical, based on the same Dockerfile.

edd@rob:~/git/rocker-ubuntu-lts/jammy(master)$ docker images |head -3
REPOSITORY TAG IMAGE ID CREATED SIZE
rocker/r-ubuntu jammy 7214e2c7ed54 7 minutes ago 743MB
rocker/r-ubuntu 22.04 091427e0bcf7 11 minutes ago 743MB
edd@rob:~/git/rocker-ubuntu-lts/jammy(master)$

kou · 2022-11-17T00:37:37Z

@eitsupi Sorry! I misunderstood.

@eddelbuettel Thanks! It worked: https://github.com/ursacomputing/crossbow/actions/runs/3468776645/jobs/5828688028

kou · 2022-11-17T03:35:24Z

@github-actions crossbow submit --group verify-rc-binaries --group verify-rc-wheels --param release=10.0.1 --param rc=0

github-actions · 2022-11-17T03:39:16Z

Revision: a6eabc2

Submitted crossbow builds: ursacomputing/crossbow @ actions-96463f290f

Task	Status
verify-rc-binaries-apt-linux-amd64
verify-rc-binaries-binary-linux-amd64
verify-rc-binaries-jars-linux-almalinux-8-amd64
verify-rc-binaries-jars-linux-conda-latest-amd64
verify-rc-binaries-jars-linux-ubuntu-18.04-amd64
verify-rc-binaries-jars-linux-ubuntu-20.04-amd64
verify-rc-binaries-jars-linux-ubuntu-22.04-amd64
verify-rc-binaries-wheels-linux-almalinux-8-amd64
verify-rc-binaries-wheels-linux-conda-latest-amd64
verify-rc-binaries-wheels-linux-ubuntu-18.04-amd64
verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64
verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64
verify-rc-binaries-wheels-macos-11-amd64
verify-rc-binaries-wheels-macos-11-arm64
verify-rc-binaries-wheels-macos-12-amd64
verify-rc-binaries-wheels-windows
verify-rc-binaries-yum-linux-amd64

kou added 3 commits November 16, 2022 08:56

MINOR: [Release] Update CHANGELOG.md for 10.0.1
f786b8d

MINOR: [Release] Update .deb/.rpm changelogs for 10.0.1
3ec21e0

MINOR: [Release] Update versions for 10.0.1
a6eabc2

github-actionsBot added Component: Documentation Component: Gandiva Component: C++ Component: GLib Component: C# Component: Go Component: Java Component: JavaScript Component: MATLAB Component: Python Component: R Component: Ruby Component: Parquet labels Nov 16, 2022

thisisnic mentioned this pull request Nov 16, 2022
MINOR: [R][CI] Use docker.io/library/r-base instead of docker.io/rocker/r-base#14607
Merged

kou merged commit a6eabc2 into maint-10.0.1Nov 22, 2022

thisisnic mentioned this pull request Nov 29, 2022
ARROW-18345: [R] Create a CRAN-specific packaging checklist that lives in the R package directory #14678
Merged

raulcd deleted the release-10.0.1-rc0 branch July 13, 2023 08:37

Uh oh!

Conversation

kou commented Nov 16, 2022

Uh oh!

kou commented Nov 16, 2022

Uh oh!

github-actionsBot commented Nov 16, 2022

Uh oh!

kou commented Nov 16, 2022

Uh oh!

kou commented Nov 16, 2022

Uh oh!

thisisnic commented Nov 16, 2022

Uh oh!

thisisnic commented Nov 16, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

assignUser commented Nov 16, 2022

Uh oh!

kou commented Nov 16, 2022

Uh oh!

eitsupi commented Nov 16, 2022

Uh oh!

eddelbuettel commented Nov 16, 2022

Uh oh!

kou commented Nov 16, 2022

Uh oh!

eitsupi commented Nov 16, 2022

Uh oh!

eddelbuettel commented Nov 16, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eddelbuettel commented Nov 16, 2022

Uh oh!

assignUser commented Nov 16, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eitsupi commented Nov 16, 2022

Uh oh!

assignUser commented Nov 16, 2022

Uh oh!

eddelbuettel commented Nov 17, 2022

Uh oh!

eddelbuettel commented Nov 17, 2022

Uh oh!

kou commented Nov 17, 2022

Uh oh!

kou commented Nov 17, 2022

Uh oh!

github-actionsBot commented Nov 17, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

thisisnic commented Nov 16, 2022•
edited
Loading

eddelbuettel commented Nov 16, 2022•
edited
Loading

assignUser commented Nov 16, 2022•
edited
Loading