Skip to content

GH-48091: [C++] Use FetchContent for bundled c-ares - #48092

Merged
raulcd merged 4 commits into
apache:mainfrom
raulcd:GH-48091
Nov 19, 2025
Merged

GH-48091: [C++] Use FetchContent for bundled c-ares#48092
raulcd merged 4 commits into
apache:mainfrom
raulcd:GH-48091

Conversation

@raulcd

@raulcdraulcd commented Nov 10, 2025

Copy link
Copy Markdown
Member

Rationale for this change

As a follow up of requiring a minimum CMake version >= 3.25 we discussed moving our dependencies from ExternalProject to FetchContent. This can heavily simplify our third party dependency management. Moving c-ares is the next step before moving grpc.

What changes are included in this PR?

The general change is moving from ExternalProject to FetchContent.

It also add some required integration due to other dependencies, like grpc, using ExternalProject. We not only have to build but also install in order for those other dependencies to find c-ares. This causes some timing issues between config, build, install that requires us to create a custom target to depend on so the other dependencies find abseil.

Are these changes tested?

Yes, the changes are tested locally and on CI.

Are there any user-facing changes?

No

@raulcd

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit bundled

@github-actions

Copy link
Copy Markdown

Revision: 7e7f084

Submitted crossbow builds: ursacomputing/crossbow @ actions-d7ad44f8f1

TaskStatus
test-r-depsource-bundledAzure
test-ubuntu-22.04-cpp-bundledGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions

@raulcd

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit bundled

@github-actions

Copy link
Copy Markdown

Revision: 91f6b54

Submitted crossbow builds: ursacomputing/crossbow @ actions-490508cbda

TaskStatus
test-r-depsource-bundledAzure
test-ubuntu-22.04-cpp-bundledGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions

@raulcd
raulcd marked this pull request as ready for review November 17, 2025 12:04

@raulcdraulcd left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kou this follows the same structure of what we did for Abseil, the idea is to move RE2, Protobuf and then try to move GRPC and get rid of the need for installation.

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Nov 17, 2025
@kou

kou commented Nov 18, 2025

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g r -g cpp

@koukou added the CI: Extra Run extra CI label Nov 18, 2025
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actions

Copy link
Copy Markdown

Revision: 91f6b54

Submitted crossbow builds: ursacomputing/crossbow @ actions-d22c15fa3f

TaskStatus
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-42-cppGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-ubuntu-gcc12-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-focalAzure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions

@raulcd

Copy link
Copy Markdown
MemberAuthor

Some of the failures on the R job failures above (test-r-macos-as-cran) seem related to the previous abseil change, I'll investigate:

 CMake Error at cmake_modules/ThirdpartyToolchain.cmake:3171 (set_property):
set_property can not be used on an ALIAS target.
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:173 (build_absl)
cmake_modules/ThirdpartyToolchain.cmake:304 (build_dependency)
cmake_modules/ThirdpartyToolchain.cmake:3490 (resolve_dependency)
CMakeLists.txt:523 (include)

@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Nov 18, 2025
@raulcd

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit r-binary-packages

@github-actions

Copy link
Copy Markdown

Revision: 0b0a3d1

Submitted crossbow builds: ursacomputing/crossbow @ actions-f99573dc0b

TaskStatus
r-binary-packagesGitHub Actions

@raulcd

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g r -g cpp

@github-actions

Copy link
Copy Markdown

Revision: 0b0a3d1

Submitted crossbow builds: ursacomputing/crossbow @ actions-c43d12b696

TaskStatus
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
r-binary-packagesGitHub Actions
r-recheck-mostGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1GitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-42-cppGitHub Actions
test-r-arrow-backwards-compatibilityGitHub Actions
test-r-depsource-bundledAzure
test-r-depsource-systemGitHub Actions
test-r-dev-duckdbGitHub Actions
test-r-devdocsGitHub Actions
test-r-extra-packagesGitHub Actions
test-r-gcc-11GitHub Actions
test-r-gcc-12GitHub Actions
test-r-install-localGitHub Actions
test-r-install-local-minsizerelGitHub Actions
test-r-linux-as-cranGitHub Actions
test-r-linux-rchkGitHub Actions
test-r-linux-sanitizersGitHub Actions
test-r-linux-valgrindGitHub Actions
test-r-m1-sanGitHub Actions
test-r-macos-as-cranGitHub Actions
test-r-minimal-buildAzure
test-r-offline-maximalGitHub Actions
test-r-offline-minimalAzure
test-r-rhub-debian-gcc-devel-lto-latestAzure
test-r-rhub-ubuntu-gcc12-custom-ccacheAzure
test-r-rhub-ubuntu-release-latestAzure
test-r-rocker-r-ver-latestAzure
test-r-rstudio-r-base-4.1-focalAzure
test-r-rstudio-r-base-4.2-focalAzure
test-r-ubuntu-22.04GitHub Actions
test-r-versionsGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions

Comment on lines +3174 to +3176
# When ABSL_ENABLE_INSTALL is ON, the real target is "time" not "absl_time"
# Cannot use set_property on alias targets (absl::time is an alias)
set_property(TARGET time

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not required for c-ares but a bug fix required for macOS abseil as seen on some R failures

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Nov 18, 2025
@raulcd
raulcd requested a review from kouNovember 18, 2025 18:15
@raulcd

Copy link
Copy Markdown
MemberAuthor

I have a working PoC on this branch where I have migrated Abseil, c-ares, protobuf, re2 and gRPC and I can reduce back c-ares and re2 to something like:

function(build_re2)
list(APPEND CMAKE_MESSAGE_INDENT "RE2: ")
message(STATUS"Building RE2 from source using FetchContent")
set(RE2_VENDORED
TRUEPARENT_SCOPE)
fetchcontent_declare(re2
URL${RE2_SOURCE_URL}URL_HASH"SHA256=${ARROW_RE2_BUILD_SHA256_CHECKSUM}")
prepare_fetchcontent()
fetchcontent_makeavailable(re2)
set(ARROW_BUNDLED_STATIC_LIBS
${ARROW_BUNDLED_STATIC_LIBS} re2::re2
PARENT_SCOPE)
list(POP_BACK CMAKE_MESSAGE_INDENT)
endfunction()

and

function(build_cares)
list(APPEND CMAKE_MESSAGE_INDENT "c-ares: ")
message(STATUS"Building c-ares from source using FetchContent")
set(CARES_VENDORED
TRUEPARENT_SCOPE)
fetchcontent_declare(cares
URL${CARES_SOURCE_URL}URL_HASH"SHA256=${ARROW_CARES_BUILD_SHA256_CHECKSUM}")
prepare_fetchcontent()
set(CARES_SHARED OFF)
set(CARES_STATIC ON)
fetchcontent_makeavailable(cares)
if(APPLE)
# libresolv must be linked from c-ares version 1.16.1find_library(LIBRESOLV_LIBRARYNAMESresolvlibresolvREQUIRED)
set_target_properties(c-ares::cares PROPERTIES INTERFACE_LINK_LIBRARIES"${LIBRESOLV_LIBRARY}")
 endif()
set(ARROW_BUNDLED_STATIC_LIBS
${ARROW_BUNDLED_STATIC_LIBS} c-ares::cares
PARENT_SCOPE)
list(POP_BACK CMAKE_MESSAGE_INDENT)
endfunction()

For Abseil and Protobuf as those are dependencies for other libraries too, it will take longer but we should be able to reduce them once everything is migrated to FetchContent. That branch is just for investigation purposes at the moment.

kou
kou approved these changes Nov 18, 2025

@koukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Is this ready to merge? Do we need more changes in this PR?

@github-actionsgithub-actionsBot added awaiting merge Awaiting merge and removed awaiting changes Awaiting changes labels Nov 18, 2025

@raulcdraulcd left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is ready to be merged. I just wanted to point out that we will be able to simplify it once the rest of the work is done :)
I'll merge it.

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting merge Awaiting merge labels Nov 19, 2025
@raulcd
raulcd merged commit bc51a76 into apache:mainNov 19, 2025
63 of 66 checks passed
@raulcdraulcd removed the awaiting changes Awaiting changes label Nov 19, 2025
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit bc51a76.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.

mkillik-wayspring pushed a commit to mkillik-wayspring/arrow that referenced this pull request Nov 24, 2025
### Rationale for this change
As a follow up of requiring a minimum CMake version >= 3.25 we discussed moving our dependencies from ExternalProject to FetchContent. This can heavily simplify our third party dependency management. Moving c-ares is the next step before moving grpc.
### What changes are included in this PR?
The general change is moving from `ExternalProject` to `FetchContent`. It also add some required integration due to other dependencies, like grpc, using `ExternalProject`. We not only have to build but also install in order for those other dependencies to find c-ares. This causes some timing issues between config, build, install that requires us to create a custom target to depend on so the other dependencies find abseil.
### Are these changes tested?
Yes, the changes are tested locally and on CI.
### Are there any user-facing changes?
No
* GitHub Issue: apache#48091
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
Mottl pushed a commit to Mottl/arrow that referenced this pull request May 26, 2026
### Rationale for this change
As a follow up of requiring a minimum CMake version >= 3.25 we discussed moving our dependencies from ExternalProject to FetchContent. This can heavily simplify our third party dependency management. Moving c-ares is the next step before moving grpc.
### What changes are included in this PR?
The general change is moving from `ExternalProject` to `FetchContent`. It also add some required integration due to other dependencies, like grpc, using `ExternalProject`. We not only have to build but also install in order for those other dependencies to find c-ares. This causes some timing issues between config, build, install that requires us to create a custom target to depend on so the other dependencies find abseil.
### Are these changes tested?
Yes, the changes are tested locally and on CI.
### Are there any user-facing changes?
No
* GitHub Issue: apache#48091
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@raulcd@kou