Skip to content

GH-38079: [R] Allow override of version used to download libarrow static library - #38080

Closed
paleolimbot wants to merge 2 commits into
apache:mainfrom
paleolimbot:version-override
Closed

GH-38079: [R] Allow override of version used to download libarrow static library#38080
paleolimbot wants to merge 2 commits into
apache:mainfrom
paleolimbot:version-override

Conversation

@paleolimbot

@paleolimbotpaleolimbot commented Oct 6, 2023

Copy link
Copy Markdown
Member

Rationale for this change

Doing a bundled build of libarrow is time-consuming and makes casual contribution difficult. Downloading a recent nightly static library is sufficient in most cases; however, there is no route to override the version used to do this download.

What changes are included in this PR?

Adds an environment variable LIBARROW_BINARY_VERSION that can be used to override the VERSION for the purposes of downloading the binary.

Are these changes tested?

Not yet! Working on the best way to do this.

Are there any user-facing changes?

No (this is currently just a developer-facing tool).

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #35049has been automatically assigned in GitHub to PR creator.

@paleolimbotpaleolimbot changed the title GH-35049: [R] Allow override of version used to download libarrow static libraryGH-38079: [R] Allow override of version used to download libarrow static libraryOct 6, 2023
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #38079has been automatically assigned in GitHub to PR creator.

@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Tested on MacOS 10.13 using

LIBARROW_BINARY_VERSION=13.0.0.100000258 LIBARROW_BINARY=true ARROW_R_DEV=true R CMD INSTALL .

Build log:

Details
* installing to library ‘/Library/Frameworks/R.framework/Versions/4.2/Resources/library’
* installing *source* package ‘arrow’ ...
** using staged installation
*** Generating code with data-raw/codegen.R
*** > 542 functions decorated with [[arrow|acero|dataset|substrait|parquet|s3|gcs|json::export]]
*** > `src/arrowExports.cpp` not modified
*** > `R/arrowExports.R` not modified
*** Found libcurl and OpenSSL >= 3.0.0
trying URL 'https://nightlies.apache.org/arrow/r/libarrow/bin/darwin-x86_64-openssl-3.0/arrow-13.0.0.100000258.zip'
Content type 'application/zip' length 23952616 bytes (22.8 MB)
==================================================
downloaded 22.8 MB
*** Successfully retrieved C++ binaries (darwin-x86_64-openssl-3.0)
Warning message:
In dir.create(dst_dir, showWarnings = !quietly, recursive = TRUE) :
'libarrow/arrow-13.0.0.9000' already exists
5
PKG_CFLAGS=-I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY
PKG_LIBS=-L/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/lib -larrow_acero -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lcurl -lssl -lcrypto -lcurl -lssl -lcrypto
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c RTasks.cpp -o RTasks.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c altrep.cpp -o altrep.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c array.cpp -o array.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c array_to_vector.cpp -o array_to_vector.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c arraydata.cpp -o arraydata.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c arrowExports.cpp -o arrowExports.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c bridge.cpp -o bridge.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c buffer.cpp -o buffer.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c chunkedarray.cpp -o chunkedarray.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c compression.cpp -o compression.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c compute-exec.cpp -o compute-exec.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c compute.cpp -o compute.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c config.cpp -o config.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c csv.cpp -o csv.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c dataset.cpp -o dataset.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c datatype.cpp -o datatype.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c expression.cpp -o expression.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c extension-impl.cpp -o extension-impl.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c feather.cpp -o feather.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c field.cpp -o field.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c filesystem.cpp -o filesystem.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c io.cpp -o io.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c json.cpp -o json.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c memorypool.cpp -o memorypool.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c message.cpp -o message.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c parquet.cpp -o parquet.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c r_to_arrow.cpp -o r_to_arrow.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatch.cpp -o recordbatch.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatchreader.cpp -o recordbatchreader.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c recordbatchwriter.cpp -o recordbatchwriter.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c safe-call-into-r-impl.cpp -o safe-call-into-r-impl.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c scalar.cpp -o scalar.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c schema.cpp -o schema.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c symbols.cpp -o symbols.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c table.cpp -o table.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c threadpool.cpp -o threadpool.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/include -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -D_LIBCPP_DISABLE_AVAILABILITY -I'/Library/Frameworks/R.framework/Versions/4.2/Resources/library/cpp11/include' -I/usr/local/include -fPIC -D_LIBCPP_DISABLE_AVAILABILITY -c type_infer.cpp -o type_infer.o
clang++ -mmacosx-version-min=10.13 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o arrow.so RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o expression.o extension-impl.o feather.o field.o filesystem.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/Users/dewey/Desktop/rscratch/arrow/r/libarrow/arrow-13.0.0.9000/lib -larrow_acero -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lcurl -lssl -lcrypto -lcurl -lssl -lcrypto -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation are out of sync. Falling back to library file for linking.
installing to /Library/Frameworks/R.framework/Versions/4.2/Resources/library/00LOCK-r/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (arrow)

...and we get the package built with S3:

> arrow::arrow_info()
Arrow package version: 13.0.0.9000
Capabilities:
acero TRUE
dataset TRUE
substrait FALSE
parquet TRUE
json TRUE
s3 TRUE
gcs TRUE
utf8proc TRUE
re2 TRUE
snappy TRUE
gzip TRUE
brotli TRUE
zstd TRUE
lz4 TRUE
lz4_frame TRUE
lzo FALSE
bz2 TRUE
jemalloc TRUE
mimalloc TRUE
Memory:
Allocator mimalloc
Current 0 bytes
Max 0 bytes
Runtime:
SIMD Level avx2
Detected SIMD Level avx2
Build:
C++ Library Version 14.0.0-SNAPSHOT
C++ Compiler AppleClang
C++ Compiler Version 10.0.0.10001044
Git ID e1d7459bce3f10c941cec1665b15d3388cbe5adb

@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Tested on Windows, too. The easiest way to get it there is to usethis::edit_r_environ() and add LIBARROW_BINARY_VERSION=13.0.0.100000258.

@paleolimbot
paleolimbot marked this pull request as ready for review October 9, 2023 01:40
@paleolimbot

Copy link
Copy Markdown
MemberAuthor

@assignUser is this fix worth merging? I do need this or something like it to debug the Windows issues we've been seeing. An alternative would be to go straight for LIBARROW_BINARY_URI (which could be file:// as well), which would also work for my personal degugging needs.

@assignUser

Copy link
Copy Markdown
Member

I like the idea but this will cause conflicts with #38115 but as that is done (imo) I will start on the 'get latest nightly binary'-feature now were I can integrate this.

For your local test we do have ARROW_DOWNLOADED_BINARIES and RWINLIB_LOCAL which can be used to build with a downloaded libarrow zip file, not a 1:1 match as you have to download manually once but pretty close. Would that work for you for now?

assignUser pushed a commit that referenced this pull request Oct 9, 2023
### Rationale for this change
We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.
### What changes are included in this PR?
This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., #38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.
### Are these changes tested?
These changes remove tests and infrastructure that are no longer relevant.
### Are there any user-facing changes?
No.
* Closes: #38088
Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Got it! I will rework this once there's a pre-calculatable URL for "the last one". I like URL better than a local file (because URL can be either) but the right way to do that would be to replace the existing "local file" option and unify it under "libarrow zip url".

@paleolimbot
paleolimbot marked this pull request as draft October 9, 2023 13:15
@paleolimbot

Copy link
Copy Markdown
MemberAuthor

Closing as #38236 is close to completion!

loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
…pache#38089)
### Rationale for this change
We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.
### What changes are included in this PR?
This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., apache#38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.
### Are these changes tested?
These changes remove tests and infrastructure that are no longer relevant.
### Are there any user-facing changes?
No.
* Closes: apache#38088
Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
…pache#38089)
### Rationale for this change
We no longer use autobrew to install on MacOS, so we no longer need CI jobs or helper scripts that had previously been referenced from this install path.
### What changes are included in this PR?
This PR removes references to autobrew that are no longer needed. It also removes the R homebrew CI job, which has been failing for several months. We had supported homebrew because `brew install apache-arrow --HEAD` was one method of avoiding a full binary install for potential MacOS contributors; however, downloading nightly static libraries (e.g., apache#38080) is an even faster way to get a compatible libarrow built for the R package and works on MacOS, Windows, and Linux.
### Are these changes tested?
These changes remove tests and infrastructure that are no longer relevant.
### Are there any user-facing changes?
No.
* Closes: apache#38088
Authored-by: Dewey Dunnington <dewey@voltrondata.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting committer reviewAwaiting committer reviewComponent: R

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[R] Allow overriding the libarrow static lib version

2 participants

@paleolimbot@assignUser