GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219

Merged
kou merged 6 commits into
apache:mainfrom
felipecrv:resolve_absl
Jul 12, 2024
Merged

GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom
felipecrv:resolve_absl

Conversation

@felipecrv

@felipecrvfelipecrv commented Jul 11, 2024•
edited by kou
Loading

Copy link
Copy Markdown
Contributor

Rationale for this change

The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.

What changes are included in this PR?

Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).

Are these changes tested?

Yes, by me trying different build configurations on my macOS and existing builds in CI.

@felipecrv
felipecrv requested a review from kouJuly 11, 2024 01:39
felipecrv
felipecrv commented Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Jul 11, 2024
kou
kou reviewed Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake Outdated
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake Outdated
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser

assignUser commented Jul 11, 2024

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g cpp

@github-actions

github-actionsBot commented Jul 11, 2024

Copy link
Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510

Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16

TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions

@felipecrv
felipecrv requested a review from kouJuly 11, 2024 19:21
kou
kou reviewed Jul 11, 2024
View reviewed changes

@koukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?

diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \

We may also need this to avoid an unexpected combination:

diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake Outdated
@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Jul 11, 2024
@felipecrv
felipecrv requested a review from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jul 11, 2024
@kou

kou commented Jul 12, 2024

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g cpp

@github-actions

github-actionsBot commented Jul 12, 2024

Copy link
Copy Markdown

Revision: 164898f

Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4

TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions

kou
kou approved these changes Jul 12, 2024
View reviewed changes

@koukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@koukou changed the title GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Jul 12, 2024
@kou
kou merged commit e65c1e2 into apache:mainJul 12, 2024
@koukou removed the awaiting merge Awaiting merge label Jul 12, 2024
@koukou mentioned this pull request Jul 12, 2024
Closed
@conbench-apache-arrow

conbench-apache-arrowBot commented Jul 12, 2024

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

@felipecrv
felipecrv deleted the resolve_absl branch July 12, 2024 13:35
@koukou mentioned this pull request Jul 13, 2024
Closed
@felipecrvfelipecrv mentioned this pull request Jul 15, 2024
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees

No one assigned

Labels

Component: C++

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
GH-43218: [C++] Resolve Abseil like any other dependency in the build system - #43219
Merged
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Jul 12, 2024
Merged
GH-43218: [C++] Resolve Abseil like any other dependency in the build system#43219
kou merged 6 commits into
apache:mainfrom 
felipecrv:resolve_absl
Conversation
@felipecrv
@felipecrvfelipecrv
commented
Jul 11, 2024•

edited by kou

Loading

Copy link


Copy Markdown

Contributor
Rationale for this change
The workarounds around Abseil resolution don't seem necessary anymore and they don't work on all possible configurations of the build.
What changes are included in this PR?
Removal of the ensure_absl macro and adding a call to resolve_dependency when depending on the Google Cloud SDK (a GCS filesystem dependency) or gRPC (a flight dependency).
Are these changes tested?
Yes, by me trying different build configurations on my macOS and existing builds in CI.
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 01:39
felipecrv
felipecrv
commented
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting committer review
Awaiting committer review
labels
Jul 11, 2024
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@felipecrv@kou
cmake: Pass correct package name to resolve_dependency(absl...)
9c07dfa
And don't pass a pkg-config parameter for absl as there isn't a single package but many.
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
@assignUser
assignUser
commented
Jul 11, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 11, 2024

Copy link


Copy Markdown

Revision: c43af58b6c5c61ef0493572b12186666c3c07510
Submitted crossbow builds: ursacomputing/crossbow @ actions-402e522d16
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 19:21
kou
kou
reviewed
Jul 11, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove absl_SOURCE=BUNDLED from Fedora 39's Dockerfile?
diff --git a/ci/docker/fedora-39-cpp.dockerfile b/ci/docker/fedora-39-cpp.dockerfile
index 8ecaa6c3ca..33d1182309 100644
--- a/ci/docker/fedora-39-cpp.dockerfile+++ b/ci/docker/fedora-39-cpp.dockerfile@@ -77,8 +77,7 @@ RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
# PYARROW_TEST_GANDIVA=OFF: GH-39695: We need to make LLVM symbols visible in
# Python process explicitly if we use LLVM 17 or later.
-ENV absl_SOURCE=BUNDLED \- ARROW_ACERO=ON \+ENV ARROW_ACERO=ON \
ARROW_AZURE=OFF \
ARROW_BUILD_TESTS=ON \
ARROW_DEPENDENCY_SOURCE=SYSTEM \
We may also need this to avoid an unexpected combination:
diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake
index a973c47848..1d382c967f 100644
--- a/cpp/cmake_modules/ThirdpartyToolchain.cmake+++ b/cpp/cmake_modules/ThirdpartyToolchain.cmake@@ -4154,6 +4154,11 @@ if(ARROW_WITH_GRPC)
endif()
set(ARROW_GRPC_REQUIRED_VERSION "1.30.0")
+ if(absl_SOURCE STREQUAL "BUNDLED" AND NOT gRPC_SOURCE STREQUAL "BUNDLED")+ # System gRPC can't be used with bundled Abseil+ message(STATUS "Forcing gRPC_SOURCE to BUNDLED because absl_SOURCE is BUNDLED")+ set(gRPC_SOURCE "BUNDLED")+ endif()
if(NOT Protobuf_SOURCE STREQUAL gRPC_SOURCE)
# ARROW-15495: Protobuf/gRPC must come from the same source
message(STATUS "Forcing gRPC_SOURCE to Protobuf_SOURCE (${Protobuf_SOURCE})")
Comment threadcpp/cmake_modules/ThirdpartyToolchain.cmake
Outdated

@github-actionsgithub-actionsBot
added

awaiting changes
Awaiting changes
and removed

awaiting change review
Awaiting change review
labels
Jul 11, 2024
@felipecrv
felipecrv
requested a review
from kouJuly 11, 2024 21:56
@github-actionsgithub-actionsBot
added

awaiting change review
Awaiting change review
and removed

awaiting changes
Awaiting changes
labels
Jul 11, 2024
@kou
kou
commented
Jul 12, 2024

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g cpp
@github-actions
github-actionsBot
commented
Jul 12, 2024

Copy link


Copy Markdown

Revision: 164898f
Submitted crossbow builds: ursacomputing/crossbow @ actions-0dab8bdfd4
TaskStatus
test-alpine-linux-cppGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-cuda-cppGitHub Actions
test-debian-12-cpp-amd64GitHub Actions
test-debian-12-cpp-i386GitHub Actions
test-fedora-39-cppGitHub Actions
test-ubuntu-20.04-cppGitHub Actions
test-ubuntu-20.04-cpp-bundledGitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizerGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-20GitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
kou
kou
approved these changes
Jul 12, 2024
View reviewed changes
@koukou
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
@koukou
changed the title
GH-43218: [C++] Resolve abseil like any other dependency in the build systemGH-43218: [C++] Resolve Abseil like any other dependency in the build systemJul 12, 2024
@github-actionsgithub-actionsBot
added

awaiting merge
Awaiting merge
and removed

awaiting change review
Awaiting change review
labels
Jul 12, 2024
@kou
kou
merged commit e65c1e2
into
apache:mainJul 12, 2024
@koukou
removed
the 
awaiting merge
Awaiting merge
label
Jul 12, 2024
@koukou
mentioned this pull request
Jul 12, 2024

Closed

@conbench-apache-arrow
conbench-apache-arrowBot
commented
Jul 12, 2024

Copy link


Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit e65c1e2.
There were no benchmark performance regressions. 🎉
The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
@felipecrv
felipecrv
deleted the

resolve_absl

branch
July 12, 2024 13:35
@koukou
mentioned this pull request
Jul 13, 2024

Closed

@felipecrvfelipecrv
mentioned this pull request
Jul 15, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@koukoukou approved these changes
@assignUserassignUserAwaiting requested review from assignUserassignUser is a code owner
@raulcdraulcdAwaiting requested review from raulcdraulcd is a code owner

Assignees


No one assigned


Labels

Component: C++

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@felipecrv@assignUser@kou