GH-45331: [C++] Use xsimd for CPU feature detection by AntoinePrv · Pull Request #49940 · apache/arrow

AntoinePrv · 2026-05-06T14:29:11Z

Rationale for this change

Use xsimd cpu feature instead of maitaining them here.
Stacked on GH-49922.
Should also handle issue from GH-30368.

What changes are included in this PR?

Use xsimd for CPu feature detection, cache sizes are still using the same implementation
We are loosing CPU model name (which was unused)
Simplify CpuInfo (remove Pimpl pattern)

Are these changes tested?

in CI

Are there any user-facing changes?

No

GitHub Issue: [C++] Use existing library for CPU detection #45331

github-actions · 2026-05-06T14:32:51Z

⚠️ GitHub issue #45331has been automatically assigned in GitHub to PR creator.

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR refactors Arrow’s CpuInfo implementation by removing the PIMPL indirection and migrating runtime CPU feature detection to xsimd.

Changes:

Replace OS/arch-specific CPUID and /proc/cpuinfo parsing with xsimd-based feature detection.
Inline CpuInfo accessors in the header and store state directly on the class (no Impl).
Generalize OS cache-size retrieval helpers to be std::array-size agnostic via templates.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
cpp/src/arrow/util/cpu_info.h	Inlines getters, removes PIMPL plumbing, and changes the `model_name()` API/behavior.
cpp/src/arrow/util/cpu_info.cc	Implements xsimd-backed CPU feature detection, removes legacy per-OS detection, and stores state directly in `CpuInfo`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

AntoinePrv · 2026-06-03T15:07:46Z

@pitrou this is in pretty good shape !

pitrou

Neat, thanks a lot for doing this @AntoinePrv

pitrou · 2026-06-04T17:08:45Z

@github-actions crossbow submit -g cpp

pitrou · 2026-06-04T17:08:53Z

@github-actions crossbow submit wheelcp314*

github-actions · 2026-06-04T17:11:24Z

Revision: 557d65b

Submitted crossbow builds: ursacomputing/crossbow @ actions-ee545fa2be

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

github-actions · 2026-06-04T17:11:29Z

Revision: 557d65b

Submitted crossbow builds: ursacomputing/crossbow @ actions-055eb82a87

Task	Status
wheel-macos-monterey-cp314-cp314-amd64
wheel-macos-monterey-cp314-cp314-arm64
wheel-macos-monterey-cp314-cp314t-amd64
wheel-macos-monterey-cp314-cp314t-arm64
wheel-manylinux-2-28-cp314-cp314-amd64
wheel-manylinux-2-28-cp314-cp314-arm64
wheel-manylinux-2-28-cp314-cp314t-amd64
wheel-manylinux-2-28-cp314-cp314t-arm64
wheel-musllinux-1-2-cp314-cp314-amd64
wheel-musllinux-1-2-cp314-cp314-arm64
wheel-musllinux-1-2-cp314-cp314t-amd64
wheel-musllinux-1-2-cp314-cp314t-arm64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

AntoinePrv · 2026-06-05T07:22:59Z

We have two

/arrow/cpp/src/arrow/util/cpu_info.cc:29:10: fatal error: 'xsimd/xsimd.hpp' file not found

I wonder if it is becuase previous includes were gated by
#if defined(ARROW_HAVE_NEON) || defined(ARROW_HAVE_SSE4_2)
and we are now catching cases were xsimd is in fact not available.

pitrou · 2026-06-05T08:15:57Z

Ahah. Well, let's make it mandatory then? See

arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake

Lines 2847 to 2852 in e16067a

if((NOT ARROW_SIMD_LEVEL STREQUAL"NONE") OR (NOT ARROW_RUNTIME_SIMD_LEVEL STREQUAL"NONE"
 )) 
set(ARROW_USE_XSIMD TRUE) 
else() 
set(ARROW_USE_XSIMD FALSE) 
endif() 

AntoinePrv · 2026-06-05T11:12:56Z

@github-actions crossbow submit -g cpp

github-actions · 2026-06-05T11:15:34Z

Revision: f5cab60

Submitted crossbow builds: ursacomputing/crossbow @ actions-f9e8dca2bd

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

AntoinePrv · 2026-06-05T14:39:38Z

I pushed a fix for emscripten (and other ?), though I think the issue must have been there before.
I think the rest must be unreleated.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

AntoinePrv · 2026-06-05T15:21:22Z

@github-actions crossbow submit -g cpp

github-actions · 2026-06-05T15:24:13Z

Revision: feda2a5

Submitted crossbow builds: ursacomputing/crossbow @ actions-eced990723

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

pitrou

+1, LGTM. Just one question.

pitrou · 2026-06-09T13:33:10Z

+ list(APPEND ARROW_COMPUTE_STATIC_LINK_LIBS ${ARROW_XSIMD})
+ list(APPEND ARROW_COMPUTE_SHARED_PRIVATE_LINK_LIBS ${ARROW_XSIMD})


I don't see xsimd in use anywhere in arrow/compute, do you remember why this was required @raulcd ?
Edit: perhaps because of src/arrow/util/utf8_internal.h? It's a pity this indirect dependency isn't picked up automatically.

pitrou · 2026-06-09T13:37:50Z

@github-actions crossbow submit wheelcp314*

github-actions · 2026-06-09T13:40:25Z

Revision: feda2a5

Submitted crossbow builds: ursacomputing/crossbow @ actions-f1dbe8e148

Task	Status
wheel-macos-monterey-cp314-cp314-amd64
wheel-macos-monterey-cp314-cp314-arm64
wheel-macos-monterey-cp314-cp314t-amd64
wheel-macos-monterey-cp314-cp314t-arm64
wheel-manylinux-2-28-cp314-cp314-amd64
wheel-manylinux-2-28-cp314-cp314-arm64
wheel-manylinux-2-28-cp314-cp314t-amd64
wheel-manylinux-2-28-cp314-cp314t-arm64
wheel-musllinux-1-2-cp314-cp314-amd64
wheel-musllinux-1-2-cp314-cp314-arm64
wheel-musllinux-1-2-cp314-cp314t-amd64
wheel-musllinux-1-2-cp314-cp314t-arm64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

conbench-apache-arrow · 2026-06-10T15:42:43Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit ca47cd1.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 6 possible false positives for unstable benchmarks that are known to sometimes produce them.

github-actionsBot added Component: C++ awaiting review Awaiting review labels May 6, 2026

AntoinePrv force-pushed the cpu-feature branch 2 times, most recently from 2958760 to c132835Compare May 7, 2026 09:12

AntoinePrv added 3 commits June 3, 2026 14:23

Use xsimf in CpuInfo and remove Pimpl
5454426

Fix headers
6ca9c33

Fix convertion
bdc923a

CopilotAI review requested due to automatic review settings June 3, 2026 12:23

AntoinePrv force-pushed the cpu-feature branch from c3f4c26 to bdc923aCompare June 3, 2026 12:23

CopilotAI reviewed Jun 3, 2026
View reviewed changes

Comment threadcpp/src/arrow/util/cpu_info.cc
Comment threadcpp/src/arrow/util/cpu_info.cc
Comment threadcpp/src/arrow/util/cpu_info.h
Comment threadcpp/src/arrow/util/cpu_info.cc
Comment threadcpp/src/arrow/util/cpu_info.cc Outdated

Fix comments
557d65b

CopilotAI review requested due to automatic review settings June 3, 2026 13:50

AntoinePrv force-pushed the cpu-feature branch from e8a35fe to 557d65bCompare June 3, 2026 13:50

CopilotAI reviewed Jun 3, 2026
View reviewed changes

Comment threadcpp/src/arrow/util/cpu_info.h
Comment threadcpp/src/arrow/util/cpu_info.cc
Comment threadcpp/src/arrow/util/cpu_info.cc

AntoinePrv marked this pull request as ready for review June 3, 2026 15:07

pitrou added the CI: Extra: C++ Run extra C++ CI label Jun 4, 2026

pitrou approved these changes Jun 4, 2026
View reviewed changes

github-actionsBot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jun 4, 2026

Make xsimd mandatory
f5cab60

AntoinePrv requested a review from wgtmac as a code owner June 5, 2026 09:32

github-actionsBot removed the CI: Extra: C++ Run extra C++ CI label Jun 5, 2026

github-actionsBot added Component: Parquet Component: Gandiva labels Jun 5, 2026

Handle emscripten
feda2a5

CopilotAI review requested due to automatic review settings June 5, 2026 14:38

CopilotAI reviewed Jun 5, 2026
View reviewed changes

Comment threadcpp/src/arrow/util/cpu_info.cc
Comment threadcpp/src/arrow/util/cpu_info.h
Comment threadcpp/src/arrow/util/cpu_info.cc

pitrou added the CI: Extra: C++ Run extra C++ CI label Jun 9, 2026

pitrou approved these changes Jun 9, 2026
View reviewed changes

pitrou changed the title ~~GH-45331: Use xsimd for CPU feature detection~~GH-45331: [C++≠ Use xsimd for CPU feature detectionJun 9, 2026

pitrou changed the title ~~GH-45331: [C++≠ Use xsimd for CPU feature detection~~GH-45331: [C++] Use xsimd for CPU feature detectionJun 9, 2026

pitrou merged commit ca47cd1 into apache:mainJun 10, 2026
86 of 87 checks passed

pitrou removed the awaiting committer review Awaiting committer review label Jun 10, 2026

pitrou mentioned this pull request Jun 10, 2026
[C++] Use existing library for CPU detection #45331
Closed

AntoinePrv deleted the cpu-feature branch June 10, 2026 15:47

jholownia mentioned this pull request Jul 22, 2026
[Python][Parquet] Silent wrong values decoding a double column on aarch64 SVE (pyarrow 25.0.0) #50605
Closed

		list(APPEND ARROW_COMPUTE_STATIC_LINK_LIBS ${ARROW_XSIMD})
		list(APPEND ARROW_COMPUTE_SHARED_PRIVATE_LINK_LIBS ${ARROW_XSIMD})

Uh oh!

Conversation

AntoinePrv commented May 6, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented May 6, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AntoinePrv commented Jun 3, 2026

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

pitrou commented Jun 4, 2026

Uh oh!

pitrou commented Jun 4, 2026

Uh oh!

github-actionsBot commented Jun 4, 2026

Uh oh!

github-actionsBot commented Jun 4, 2026

Uh oh!

AntoinePrv commented Jun 5, 2026

Uh oh!

pitrou commented Jun 5, 2026

Uh oh!

AntoinePrv commented Jun 5, 2026

Uh oh!

github-actionsBot commented Jun 5, 2026

Uh oh!

AntoinePrv commented Jun 5, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AntoinePrv commented Jun 5, 2026

Uh oh!

github-actionsBot commented Jun 5, 2026

Uh oh!

pitrou left a comment

Choose a reason for hiding this comment

Uh oh!

pitrouJun 9, 2026

Choose a reason for hiding this comment

Uh oh!

pitrou commented Jun 9, 2026

Uh oh!

github-actionsBot commented Jun 9, 2026

Uh oh!

Uh oh!

conbench-apache-arrowBot commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

AntoinePrv commented May 6, 2026•
edited
Loading