GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989

Merged
kou merged 43 commits into
apache:mainfrom
raulcd:GH-44950
Feb 13, 2025
Merged

GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom
raulcd:GH-44950

Conversation

@raulcd

@raulcdraulcd commented Dec 10, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Rationale for this change

We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk

What changes are included in this PR?

  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere

Are these changes tested?

Yes, via CI.

Are there any user-facing changes?

Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.

@github-actions

github-actionsBot commented Dec 10, 2024

Copy link
Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.

@raulcd
raulcdforce-pushed the GH-44950 branch 2 times, most recently from a2f952c to 81b112aCompareDecember 10, 2024 11:31
@raulcd

raulcd commented Dec 10, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

I am unsure on how to fix the remaining failures for R.
For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39

Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).

@h-vetinari

h-vetinari commented Dec 11, 2024

Copy link
Copy Markdown
Contributor

For the gcc 12 job I am unsure why it is failing

Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)

@raulcd

raulcd commented Dec 11, 2024

Copy link
Copy Markdown
MemberAuthor

ok, it seems R forces the builds to use the CMake provided on the images:

**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?

root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser

assignUser commented Dec 11, 2024•
edited
Loading

Copy link
Copy Markdown
Member

I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26

So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.

@nealrichardson

nealrichardson commented Dec 11, 2024

Copy link
Copy Markdown
Member

It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743

cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")

@nealrichardson

nealrichardson commented Dec 11, 2024

Copy link
Copy Markdown
Member

Two other places in the R nixlibs.R script worth updating:

@pitrou

pitrou commented Dec 11, 2024•
edited
Loading

Copy link
Copy Markdown
Member

The error case should probably return("0")

It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?

@nealrichardson

nealrichardson commented Dec 11, 2024•
edited
Loading

Copy link
Copy Markdown
Member

The error case should probably return("0")

It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?

I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.

This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718

 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.

(To be clear, we need to fix both this and the return("0") above.)

jonkeane
jonkeane reviewed Dec 12, 2024
View reviewed changes
Comment thread.env Outdated
@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Dec 12, 2024
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 13, 2024
@jonkeane

jonkeane commented Dec 14, 2024

Copy link
Copy Markdown
Member

I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)

@jonkeane

jonkeane commented Dec 14, 2024

Copy link
Copy Markdown
Member

I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).

https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.

@nealrichardson

nealrichardson commented Dec 15, 2024

Copy link
Copy Markdown
Member

I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).

https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.

Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.

For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?

@raulcd

raulcd commented Dec 17, 2024•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake
And we seem to be updating to pull the newest packages:

+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:
Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/

Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson

@nealrichardson

nealrichardson commented Dec 17, 2024

Copy link
Copy Markdown
Member

Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson

Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.

Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.

IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.

@pitrou

pitrou commented Dec 18, 2024

Copy link
Copy Markdown
Member

We support more versions of R than CRAN actively checks on

Do we still want to do that? How is our R support policy decided?

@nealrichardson

nealrichardson commented Dec 18, 2024

Copy link
Copy Markdown
Member

We support more versions of R than CRAN actively checks on

Do we still want to do that? How is our R support policy decided?

We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.

If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.

raulcdand others added 18 commits February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou

kou commented Feb 11, 2025

Copy link
Copy Markdown
Member

Rebased on main.

@kou

kou commented Feb 11, 2025

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*

@github-actions

github-actionsBot commented Feb 11, 2025

Copy link
Copy Markdown

Revision: 7d64cb7

Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd

TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions

@kou

kou commented Feb 13, 2025

Copy link
Copy Markdown
Member

I'll merge this.

@koukou mentioned this pull request Feb 13, 2025
Closed
@kou

kou commented Feb 13, 2025

Copy link
Copy Markdown
Member

Oh... This broke lint CI...:

https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733

> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings

I don't know why the last push didn't execute this job...

@koukou mentioned this pull request Feb 13, 2025
Closed
@conbench-apache-arrow

conbench-apache-arrowBot commented Feb 13, 2025

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.

@nealrichardsonnealrichardson mentioned this pull request Mar 19, 2025
Closed
This was referenced Jun 17, 2025
Merged
Closed
@thisisnicthisisnic mentioned this pull request Jun 15, 2026
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees

No one assigned

Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
GH-44950: [C++] Bump minimum CMake version to 3.25 - #44989
Merged
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Feb 13, 2025
Merged
GH-44950: [C++] Bump minimum CMake version to 3.25#44989
kou merged 43 commits into
apache:mainfrom 
raulcd:GH-44950
Conversation
@raulcd
@raulcdraulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

Member
Rationale for this change
We want to upgrade our CMake version to 3.25 as discussed on the ML:
https://lists.apache.org/thread/h8jp16ktrj11fmjmjhlg6xvkvv9wzvjk
What changes are included in this PR?
  • Bump minimal CMake version to 3.25
  • Manually install CMake on distributions where CMake < 3.25 was installed via package repositories
  • Minor fixes to CI in order to have passing builds everywhere
Are these changes tested?
Yes, via CI.
Are there any user-facing changes?
Yes, the minimum CMake version to be used to build Arrow is bumped to 3.25.
This PR includes breaking changes to build systems.
@github-actions
github-actionsBot
commented
Dec 10, 2024

Copy link


Copy Markdown

⚠️ GitHub issue #44950has been automatically assigned in GitHub to PR creator.
@raulcd
raulcdforce-pushed
the
GH-44950
branch
2 times, most recently
from
a2f952c to
81b112aCompareDecember 10, 2024 11:31
@raulcd
raulcd
commented
Dec 10, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I am unsure on how to fix the remaining failures for R.

For the gcc 12 job I am unsure why it is failing and for the Windows C++ RTools 40 ucrt64 it seems we install CMake from MINGW here but I am not sure if this is necessary or can be updated.
https://github.com/apache/arrow/blob/main/ci/scripts/PKGBUILD#L39
Of course this is the initial CI (we also have to update all the extended CI jobs for crossbow).
@h-vetinari
h-vetinari
commented
Dec 11, 2024

Copy link


Copy Markdown

Contributor
For the gcc 12 job I am unsure why it is failing
Updating the CMake lower bound will flip the default of several policies from legacy to new; sounds like you might be relying on legacy behaviour there in some way (once you figure out which policy is at fault, there's usually a migration path to keep the old behaviour)
@raulcd
raulcd
commented
Dec 11, 2024

Copy link


Copy Markdown

MemberAuthor
ok, it seems R forces the builds to use the CMake provided on the images:
**** Not using cmake found at /bin/cmake
Error in .make_numeric_version(x, strict, .standard_regexps()$valid_numeric_version) : invalid non-character version specification 'x' (type: double)
Calls: build_libarrow ... as.numeric_version -> numeric_version -> .make_numeric_version
Execution halted

@jonkeane@assignUser@amoeba will this be an issue for CRAN? Are we somehow forced to the CMake version on those images?
root@03abb5b759ba:/# /bin/cmake --version
cmake version 3.22.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

@assignUser
assignUser
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
I went through the logs of our recent checks on cran and only one is using a version < 3.25 and that seems more incidental then purposely as it's the r-odrel arm64 but the intel version has 3.26
So I don't think we should be forced to use that cmake version, additionally we have a function that fetches current cmake if an unsuited version is found but apparently there is an issue with it as seen above. IIRC there was a change to numeric version in one of the las R Versions that is causing this? I'll have a look.
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
It's possible the version comparison error is from this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL731-L743
cmake_version <- function(cmd = "cmake") {
tryCatch(
{
raw_version <- system(paste(cmd, "--version"), intern = TRUE, ignore.stderr = TRUE)
pat <- ".* ([0-9\\.]+).*?"
which_line <- grep(pat, raw_version)
package_version(sub(pat, "\\1", raw_version[which_line]))
},
error = function(e) {
return(0)
}
)
}

The error case should probably return("0")
@nealrichardson
nealrichardson
commented
Dec 11, 2024

Copy link


Copy Markdown

Member
Two other places in the R nixlibs.R script worth updating:
@pitrou
pitrou
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
@nealrichardson
nealrichardson
commented
Dec 11, 2024•

edited

Loading

Copy link


Copy Markdown

Member
The error case should probably return("0")
It would probably be more forward-looking to avoid the error entirely. Why does the function fail parsing the CMake version? Can we add cmake --version somewhere in the GH workflow?
I could be remembering wrong, but I believe the function is used to check for cmake of a certain version, and this is to be robust to where cmake may not be installed or not found at the path provided. It does not emit an error, it traps it.
This might not be where the error is coming from that was observed in CI, I was just browsing the source to see where you might get a numeric version error. Looking again, and reading the output it produced, I think we're hitting this: https://github.com/apache/arrow/pull/44989/files#diff-935746c34b16289a07b0d9bf7642dbd268b18059b6187f7cdec7c464be47a3deL718
 } else {
# Keep trying
lg("Not using cmake found at %s", path, .indent = "****")
if (found_version > 0) {
lg("Version >= %s required; found %s", version_required, found_version, .indent = "*****")
} else {

should be found_version > "0". We must not have been running into this before because the "found_version" was always sufficient if found, and if it wasn't found, we were returning a numeric 0 which works in this comparison.
(To be clear, we need to fix both this and the return("0") above.)
jonkeane
jonkeane
reviewed
Dec 12, 2024
View reviewed changes
Comment thread.env
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Dec 12, 2024
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Dec 13, 2024
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I've pushed the changes Neal suggested which should fix the ubuntu failure 🤞 (hope you don't mind, @raulcd !)
@jonkeane
jonkeane
commented
Dec 14, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
@nealrichardson
nealrichardson
commented
Dec 15, 2024

Copy link


Copy Markdown

Member
I did a bit of digging on the windows front, and I suspect what's going on is the the MSYS2/mingw cmake is being preferred (I'm not familiar enough with that ecosystem to know if we can override it with something on the system, but that also seems fragile itself).
https://github.com/raulcd/arrow/blob/e5d521134db4bed8507572fece3b56eb4a1b9158/ci/scripts/r_windows_build.sh#L26-L30 has some info about how to test newer dependencies, I wonder if we (conditionally) use those pacman commands to insall a newer cmake than what's in the CRAN repo (https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/ has the version I'm seeing installed listed, so if we override that one in our builds that might be sufficient.
Unfortunately, that won't help us. https://github.com/r-windows/rtools-packages/ is archived; CRAN has moved away from the toolchain that Jeroen was maintaining. We should delete that comment.
For newer cmake with older rtools, maybe we can get it from https://github.com/Kitware/CMake/releases/download/v3.31.2/cmake-3.31.2-windows-x86_64.zip or something? Or maybe we can install newer rtools for cmake but use the older rtools for compilers etc.?
@raulcd
raulcd
commented
Dec 17, 2024•

edited

Loading

Copy link


Copy Markdown

MemberAuthor
I don't understand why the job is installing mingw-w64-ucrt-x86_64-cmake-3.21.3-1-any.pkg.tar.x when the remote package seems to be updated to a newer version: https://packages.msys2.org/packages/mingw-w64-ucrt-x86_64-cmake

And we seem to be updating to pull the newest packages:
+ pacman --noconfirm -Syy
:: Synchronizing package databases...
downloading mingw32.db...
downloading mingw64.db...
downloading ucrt64.db...
downloading mirrors.db...

edit:

Ok, I see this is using CRAN mirrors here: https://cloud.r-project.org/bin/windows/Rtools/4.0/ucrt64/
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
@nealrichardson
nealrichardson
commented
Dec 17, 2024

Copy link


Copy Markdown

Member
Probably naive question here, I suppose we can't update the mirrors here to point to msys2 and download the newer cmake from there before building Arrow as we would have the same issue when we try to publish to CRAN, right? @jonkeane@nealrichardson
Honestly I don't know. But it's not relevant for CRAN because (a) CRAN currently only builds on R 4.3 and 4.4, both of which have new enough cmake, and (b) we don't require cmake on CRAN anyway because we build the libarrow C++ library in our CI and download it in the CRAN build--CRAN only compiles the R bindings, which do not require cmake.
Solving the cmake issue for R < 4.3 is only for our own purposes of building C++ libraries that are compatible with older versions of R on Windows. We support more versions of R than CRAN actively checks on, though it appears that in CI, we only check on the Windows current release version, and when we test old R versions, we do it on linux.
IIUC we're building libarrow with the rtools4.0 toolchain for maximum compatibility. If we want to continue doing that, we could try installing newer cmake somewhere before https://github.com/apache/arrow/blob/main/.github/workflows/r.yml#L293 and making sure it's on the PATH. Or, to bump up, we would change the rtools version in that job from 40 to 43 to use the rtools43 toolchain with newer cmake.
@pitrou
pitrou
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
@nealrichardson
nealrichardson
commented
Dec 18, 2024

Copy link


Copy Markdown

Member
We support more versions of R than CRAN actively checks on
Do we still want to do that? How is our R support policy decided?
We generally follow the tidyverse version support policy, which is the most recent 5 versions. Many enterprise users of R don't upgrade versions eagerly so they can be stuck on older versions longer than CRAN's testing window.
If there is an easy way to get new enough cmake into the R windows-cpp job so that we can keep building with the R 4.0 toolchain, that would be ideal. I don't see why we couldn't do that.
raulcdand others
added 18 commits
February 11, 2025 13:34
@raulcd@kou
Removing CMP0126 OLD policy behavior and force unset ZLIB_LIBRARY
7020fb5
@raulcd@kou
Remove system wide pip upgrade
3b99995
@amoeba@kou
Add readme to ci/rtools dir
5e25f01
@amoeba@kou
Fix bug in patch
ca25b71
@amoeba@kou
Move patch hunks into ThirdpartyToolchain.cmake
d93a55b
@amoeba@kou
Remove unneeded #define
abf5b84
@kou
Fix wrong command
4f20c6a
@raulcd@kou
Add License header to new README
74d5606
@raulcd@kou
Comment Build-Depends on cmake for debian packages until we drop Ubun…
17f502c
…tu 22.04 support
@raulcd@kou
Try using /usr/local/bin/cmake for focal and jammy
fcd35fa
@raulcd@kou
New attempt to fix debian rules
eacab29
@amoeba@kou
Remove remainder of _WIN32_WINNT defines
6e72a16
@amoeba@kou
Remove _WIN32_WINNT define in awssdk_ep.patch
2e98564
@raulcd@kou
Create a link to make debhelper build system happy
0395a6a
@raulcd@kou
Fix extremely weird alignment on debian rules
147ecc0
@amoeba@kou
Update awssdk_ep.patch.
6f19834
See conversation
https://github.com/apache/arrow/pull/44989/files#r1934292489.
@raulcd@kou
Add back cmake and link to Dockerfile to avoid having to do the link …
62c8933
…on debian rules
@raulcd@kou
Some final review comments, sort lists, add comment and add empty line
7d64cb7
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
Rebased on main.
@kou
kou
commented
Feb 11, 2025

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp -g r -g python -g linux verify-rc-source-*
@github-actions
github-actionsBot
commented
Feb 11, 2025

Copy link


Copy Markdown

Revision: 7d64cb7
Submitted crossbow builds: ursacomputing/crossbow @ actions-becbe304cd
TaskStatus
almalinux-8-amd64GitHub Actions
almalinux-8-arm64GitHub Actions
almalinux-9-amd64GitHub Actions
almalinux-9-arm64GitHub Actions
amazon-linux-2023-amd64GitHub Actions
amazon-linux-2023-arm64GitHub Actions
centos-7-amd64GitHub Actions
centos-8-stream-amd64GitHub Actions
centos-8-stream-arm64GitHub Actions
centos-9-stream-amd64GitHub Actions
centos-9-stream-arm64GitHub Actions
debian-bookworm-amd64GitHub Actions
debian-bookworm-arm64GitHub Actions
debian-trixie-amd64GitHub Actions
debian-trixie-arm64GitHub Actions
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
example-python-minimal-build-fedora-condaGitHub Actions
example-python-minimal-build-ubuntu-venvGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-conda-python-3.10GitHub Actions
test-conda-python-3.10-hdfs-2.9.2GitHub Actions
test-conda-python-3.10-hdfs-3.2.1GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.10-substraitGitHub Actions
test-conda-python-3.11GitHub Actions
test-conda-python-3.11-dask-latestGitHub Actions
test-conda-python-3.11-dask-upstream_develGitHub Actions
test-conda-python-3.11-hypothesisGitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latestGitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightlyGitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightlyGitHub Actions
test-conda-python-3.11-spark-masterGitHub Actions
test-conda-python-3.12GitHub Actions
test-conda-python-3.12-cpython-debugGitHub Actions
test-conda-python-3.13GitHub Actions
test-conda-python-3.9GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5GitHub Actions
test-conda-python-emscriptenGitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-debian-12-python-3-amd64GitHub Actions
test-debian-12-python-3-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-fedora-39-python-3GitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-clang-sanitizerGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-debian-gcc-release-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-opensuse155Azure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-22.04-python-3GitHub Actions
test-ubuntu-22.04-python-313-freethreadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-24.04-python-3GitHub Actions
test-ubuntu-r-sanitizerGitHub Actions
ubuntu-jammy-amd64GitHub Actions
ubuntu-jammy-arm64GitHub Actions
ubuntu-noble-amd64GitHub Actions
ubuntu-noble-arm64GitHub Actions
verify-rc-source-cpp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-cpp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-cpp-macos-amd64GitHub Actions
verify-rc-source-cpp-macos-arm64GitHub Actions
verify-rc-source-cpp-macos-conda-amd64GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-csharp-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-csharp-macos-amd64GitHub Actions
verify-rc-source-csharp-macos-arm64GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-integration-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-integration-macos-amd64GitHub Actions
verify-rc-source-integration-macos-arm64GitHub Actions
verify-rc-source-integration-macos-conda-amd64GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-js-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-js-macos-amd64GitHub Actions
verify-rc-source-js-macos-arm64GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-python-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-python-macos-amd64GitHub Actions
verify-rc-source-python-macos-arm64GitHub Actions
verify-rc-source-python-macos-conda-amd64GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64GitHub Actions
verify-rc-source-ruby-linux-ubuntu-24.04-amd64GitHub Actions
verify-rc-source-ruby-macos-amd64GitHub Actions
verify-rc-source-ruby-macos-arm64GitHub Actions
verify-rc-source-windowsGitHub Actions
@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
I'll merge this.
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@kou
kou
commented
Feb 13, 2025

Copy link


Copy Markdown

Member
Oh... This broke lint CI...:
https://github.com/apache/arrow/actions/runs/13299364571/job/37137873622#step:8:6733
> lintr::lint_package('/arrow/r')
Error in `lint()`:
! Linter `linter()` failed in '/arrow/r/data-raw/docgen.R':
Caused by error in `linter_fun()`:
! Cyclocomp complexity is computed using `cyclocomp::cyclocomp()`.
ℹ Please install the needed cyclocomp package.
Backtrace:
▆
1. └─lintr::lint_package("/arrow/r")
2. └─lintr::lint_dir(...)
3. └─base::lapply(...)
4. └─lintr (local) FUN(X[[i]], ...)
5. └─lintr::lint(file, ..., parse_settings = FALSE, exclusions = exclusions)
6. ├─base::withCallingHandlers(...)
7. └─lintr:::get_lints(...)
8. ├─lintr:::flatten_lints(linter_fun(expr))
9. │ └─lintr:::flatten_list(x, class = "lint")
10. │ └─lintr (local) assign_item(x)
11. └─lintr (local) linter_fun(expr)
12. └─cli::cli_abort(...)
13. └─rlang::abort(...)
Warning message:
Found unused settings in config file (.lintr): unused_settings 
I don't know why the last push didn't execute this job...
@koukou
mentioned this pull request
Feb 13, 2025

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Feb 13, 2025

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdd3e15.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
@nealrichardsonnealrichardson
mentioned this pull request
Mar 19, 2025
 Closed


This was referenced Jun 17, 2025
 Merged


Closed

@thisisnicthisisnic
mentioned this pull request
Jun 15, 2026

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@amoebaamoebaamoeba left review comments
@jonkeanejonkeanejonkeane left review comments
@pitroupitroupitrou left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments
@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@kevingurneykevingurneyAwaiting requested review from kevingurneykevingurney is a code owner
@sgilmore10sgilmore10Awaiting requested review from sgilmore10sgilmore10 is a code owner

Assignees


No one assigned


Labels

Component: C++Component: DocumentationComponent: MATLABComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

8 participants

@raulcd@h-vetinari@assignUser@nealrichardson@pitrou@jonkeane@amoeba@kou