ARROW-12175: [C++] Fix CMake packages - #13892

Merged
kou merged 40 commits into
apache:masterfrom
kou:cpp-cmake-package
Aug 30, 2022
Merged

ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom
kou:cpp-cmake-package

Conversation

@kou

@koukou commented Aug 16, 2022•
edited
Loading

Copy link
Copy Markdown
Member

ARROW-9171 and ARROW-17231 are also fixed.

Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.

find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".

This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.

This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.

This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no
namespace targets such as arrow_shared and parquet_static are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because add_library(ALIAS)
for non-global IMPORTED library is available since CMake 3.18.

(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)

FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install
our .cmake files to ${PREFIX}/arrow/. But this approach breaks
backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake
approach.

@koukou mentioned this pull request Aug 16, 2022
Closed
@github-actions

github-actionsBot commented Aug 16, 2022

Copy link
Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175

@github-actions

github-actionsBot commented Aug 16, 2022

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@kou
kouforce-pushed the cpp-cmake-package branch 2 times, most recently from 38ee685 to 40e2e9dCompareAugust 17, 2022 01:06
@kou

kou commented Aug 22, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release

@github-actions

This comment was marked as outdated.

Sign in to view
@kou

kou commented Aug 22, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release

@github-actions

This comment was marked as outdated.

Sign in to view
@kou

kou commented Aug 23, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64

@github-actions

This comment was marked as outdated.

Sign in to view
@kou

kou commented Aug 24, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64

@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed the cpp-cmake-package branch from f2fd6ee to 119c8e1CompareAugust 24, 2022 05:18
@kou

kou commented Aug 24, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release

@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed the cpp-cmake-package branch from 119c8e1 to c0d38cbCompareAugust 25, 2022 07:21
@kou

kou commented Aug 25, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release

@github-actions

This comment was marked as outdated.

Sign in to view
@kou

kou commented Aug 26, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit conda-linux-gcc-py310-cpu

@github-actions

This comment was marked as outdated.

Sign in to view
@kou

kou commented Aug 27, 2022

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit conda-linux-gcc-py310-cpu

@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou deleted the cpp-cmake-package branch August 30, 2022 04:57
kou added a commit to kou/arrow that referenced this pull request Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou mentioned this pull request Aug 30, 2022
Merged
@ursabot

ursabot commented Aug 30, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 93b63e8f ec2-t3-xlarge-us-east-2
[Failed] 93b63e8f test-mac-arm
[Failed] 93b63e8f ursa-i9-9960x
[Finished] 93b63e8f ursa-thinkcentre-m75q
[Finished] b43c6f6b ec2-t3-xlarge-us-east-2
[Failed] b43c6f6b test-mac-arm
[Failed] b43c6f6b ursa-i9-9960x
[Finished] b43c6f6b ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot

ursabot commented Aug 30, 2022

Copy link
Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

kou added a commit that referenced this pull request Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche reviewed Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake
Comment on lines -460 to -465
message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")

@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).
But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)

@koukouSep 6, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.

But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:

if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()

Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).

See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables

@raulcdraulcdSep 6, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything

@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!

jorisvandenbossche
jorisvandenbossche reviewed Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)

@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))

@koukouSep 6, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.

@dhruv9vatsdhruv9vats mentioned this pull request Sep 7, 2022
Merged
kou pushed a commit that referenced this pull request Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd mentioned this pull request Sep 12, 2022
Merged
kou pushed a commit that referenced this pull request Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari mentioned this pull request Sep 14, 2022
Merged
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit to kou/arrow that referenced this pull request Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou mentioned this pull request Oct 11, 2022
Merged
kou added a commit that referenced this pull request Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier mentioned this pull request Dec 1, 2022
Closed
This was referenced Dec 1, 2022
Closed
Closed
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees

No one assigned

Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
ARROW-12175: [C++] Fix CMake packages - #13892
Merged
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Aug 30, 2022
Merged
ARROW-12175: [C++] Fix CMake packages#13892
kou merged 40 commits into
apache:masterfrom 
kou:cpp-cmake-package
Conversation
@kou
@koukou
commented
Aug 16, 2022•

edited

Loading

Copy link


Copy Markdown

Member
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, find_package(Parquet)

doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are

always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't

"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}

by CMake, pkg-config or manual .so/.h search. But we don't need to

support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this

removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and
parquet_static is exported as Parquet::parquet_static. But no

namespace targets such as arrow_shared and parquet_static are

still also exported for keeping backward compatibility. But this

requires CMake 3.18 or later for users because add_library(ALIAS)

for non-global IMPORTED library is available since CMake 3.18.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is used instead of find_package(Parquet). With
COMPONENTS, ${PACKAGE} is always "Arrow". So we can still install

our .cmake files to ${PREFIX}/arrow/. But this approach breaks

backward compatibility. So I choose ${PREFIX}/${PACKAGE}/*.cmake

approach.
@koukou
mentioned this pull request
Aug 16, 2022
 Closed

@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-12175
@github-actions
github-actionsBot
commented
Aug 16, 2022

Copy link


Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.
@kou
kouforce-pushed
the
cpp-cmake-package
branch
2 times, most recently
from
38ee685 to
40e2e9dCompareAugust 17, 2022 01:06
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 22, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 23, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit example-python-minimal-build-ubuntu-venv homebrew-cpp verify-rc-source-python-linux-conda-latest-amd64 wheel-macos-big-sur-cp310-arm64 wheel-manylinux2014-cp310-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit verify-rc-source-python-linux-conda-latest-amd64 wheel-windows-cp310-amd64
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
f2fd6ee to
119c8e1CompareAugust 24, 2022 05:18
@kou
kou
commented
Aug 24, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kouforce-pushed
the
cpp-cmake-package
branch
from
119c8e1 to
c0d38cbCompareAugust 25, 2022 07:21
@kou
kou
commented
Aug 25, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 26, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
kou
commented
Aug 27, 2022

Copy link


Copy Markdown

MemberAuthor
@github-actions crossbow submit conda-linux-gcc-py310-cpu
@github-actions

This comment was marked as outdated.

Sign in to view
@kou
ARROW-12175: [C++] Fix CMake packages
d4f0062
ARROW-9171 is also fixed.
Our CMake packages are broken. For example, find_package(Parquet)
doesn't work without specifying CMAKE_MODULE_PATH.
find_package(${PACKAGE}) searches
${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake or
${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake. But our .cmake files are
always installed ${PREFIX}/arrow/. So find_package(Parquet) can't find
${PREFIX}/arrow/FindParquet.cmake because "/arrow/" isn't
"/${PACKAGE}".
This change fixes this by installing ${PACKAGE}Config.cmake to
${PREFIX}/${PACKAGE}/ instead of ${PREFIX}/arrow/.
This also removes all Find${PACKAGE}.cmake. We only provides
${PACKAGE}Config.cmake. Our Find${PACKAGE}.cmake can find ${PACKAGE}
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
${PACKAGE}Config.cmake to support CMake package search. So this
removes all Find${PACKAGE}.cmake.
This also introduces namespace to our CMake targets. For example,
arrow_shared is exported as Arrow::arrow_shared and parquet_static is
exported as Parquet::parquet_static. But no namespace targets such as
arrow_shared and parquet_static are still also exported for keeping
backward compatibility. But this requires CMake 3.11 or later for
users because we can't use add_library(IMPORTED) is available since
CMake 3.11.
(Plasma::plasma-store-server target is also added for
plasma-store-server executable.)
FYI: We can resolve this problem by using COMPONENTS feature of
find_package(). For example, find_package(Arrow COMPONENTS Parquet) is
used instead of find_package(Parquet). With COMPONENTS, ${PACKAGE} is
always "Arrow". So we can still install our .cmake files to
${PREFIX}/arrow/. But this approach breaks backward compatibility. So
I choose ${PREFIX}/${PACKAGE}/*.cmake approach.
@kou
kou
deleted the

cpp-cmake-package

branch
August 30, 2022 04:57
kou added a commit
to kou/arrow
that referenced
this pull request
Aug 30, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
2800c5f
… isn't provided
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static` but we also
provides no namespaced CMake targets such as `arrow_share` and
`arrow_static` as aliases of namespaced CMake targets. But the
backward compatibility feature isn't worked for `_shared`.
@koukou
mentioned this pull request
Aug 30, 2022
 Merged

@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = b43c6f6 and contender = 93b63e8. 93b63e8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Failed] test-mac-arm

[Failed ⬇️0.55% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.32% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 93b63e8f ec2-t3-xlarge-us-east-2

[Failed] 93b63e8f test-mac-arm

[Failed] 93b63e8f ursa-i9-9960x

[Finished] 93b63e8f ursa-thinkcentre-m75q

[Finished] b43c6f6b ec2-t3-xlarge-us-east-2

[Failed] b43c6f6b test-mac-arm

[Failed] b43c6f6b ursa-i9-9960x

[Finished] b43c6f6b ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
@ursabot
ursabot
commented
Aug 30, 2022

Copy link


Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x
kou added a commit
that referenced
this pull request
Aug 31, 2022
@kou
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
5be95c7
… isn't provided (#14003)
This is a follow-up of ARROW-12175 / #13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadcpp/cmake_modules/FindArrow.cmake

Comment on lines

-460
 to 
-465

 message(STATUS "Arrow version: ${ARROW_VERSION} (${ARROW_FIND_APPROACH})")
 message(STATUS "Arrow SO and ABI version: ${ARROW_SO_VERSION}")
 message(STATUS "Arrow full SO version: ${ARROW_FULL_SO_VERSION}")
 message(STATUS "Found the Arrow core shared library: ${ARROW_SHARED_LIB}")
 message(STATUS "Found the Arrow core import library: ${ARROW_IMPORT_LIB}")
 message(STATUS "Found the Arrow core static library: ${ARROW_STATIC_LIB}")
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this file is now replaced by cmake's find_package(Arrow) now using the ArrowConfig.cmake file that gets installed in the cmake directory (when installing Arrow C++).

But so before, the above printed some details about Arrow being found (and which version, the path, ..). Is there a way to let find_package(Arrow) still print this? (would adding that to ArrowConfig.cmake.in do this?)
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We need to add message(STATUS ...) to ArrowConfig.cmake.in for it. cmake --debug-print prints more details but it will not show these information.
But *Config.cmake doesn't use message(STATUS ...) in general. If we want to add message(STATUS ...), it's better that we check Arrow_FIND_QUIETLY like the following:
if(NOT Arrow_FIND_QUIETLY)
message(STATUS ...)
endif()
Arrow_FIND_QUIETLY is TRUE only when an user specify QUIET explicitly like find_package(Arrow QUIET).
See also; https://cmake.org/cmake/help/latest/command/find_package.html#package-file-interface-variables
@raulcdraulcdSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created https://issues.apache.org/jira/browse/ARROW-17632 to follow this up, feel free to update the ticket if I missed anything
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kou for the explanation and @raulcd for opening the JIRA!
jorisvandenbossche
jorisvandenbossche
reviewed
Sep 6, 2022
View reviewed changes
Comment threadpython/pyarrow/src/CMakeLists.txt
#

find_package(Arrow REQUIRED)
include(ArrowOptions)
@jorisvandenbosschejorisvandenbosscheSep 6, 2022

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own understanding, the reason this line is no longer needed, is because this will also be included by find_package(Arrow) already? (since ArrowConfig.cmake has a line include("${CMAKE_CURRENT_LIST_DIR}/ArrowOptions.cmake"))
@koukouSep 6, 2022

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.
@dhruv9vatsdhruv9vats
mentioned this pull request
Sep 7, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 12, 2022
@dhruv9vats
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
3aee08b
…nd during build (#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of #13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@raulcdraulcd
mentioned this pull request
Sep 12, 2022
 Merged

kou pushed a commit
that referenced
this pull request
Sep 13, 2022
@raulcd
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
6fc33e7
…ckage (#14097)
This is a follow up to update our build documentation from the changes on #13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Sep 14, 2022
 Merged

zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-12175: [C++] Fix CMake packages (apache#13892)
4fa6702
ARROW-9171 and ARROW-17231 are also fixed.
Our CMake packages are broken. For example, `find_package(Parquet)`
doesn't work without specifying `CMAKE_MODULE_PATH`.
`find_package(${PACKAGE})` searches
`${PREFIX}/${PACKAGE}/${PACKAGE}Config.cmake` or
`${PREFIX}/${PACKAGE}/Find${PACKAGE}.cmake`. But our .cmake files are
always installed `${PREFIX}/arrow/`. So `find_package(Parquet)` can't find
`${PREFIX}/arrow/FindParquet.cmake` because "`/arrow/`" isn't
"`/${PACKAGE}`".
This change fixes this by installing `${PACKAGE}Config.cmake` to
`${PREFIX}/${PACKAGE}/` instead of `${PREFIX}/arrow/`.
This also removes all `Find${PACKAGE}.cmake`. We only provides
`${PACKAGE}Config.cmake`. Our `Find${PACKAGE}.cmake` can find `${PACKAGE}`
by CMake, pkg-config or manual .so/.h search. But we don't need to
support pkg-config nor manual .so/.h search. We can use
`${PACKAGE}Config.cmake` to support CMake package search. So this
removes all `Find${PACKAGE}.cmake`.
This also introduces namespace to our CMake targets. For example,
`arrow_shared` is exported as `Arrow::arrow_shared` and
`parquet_static` is exported as `Parquet::parquet_static`. But no
namespace targets such as `arrow_shared` and `parquet_static` are
still also exported for keeping backward compatibility. But this
requires CMake 3.18 or later for users because `add_library(ALIAS)`
for non-global `IMPORTED` library is available since CMake 3.18.
(`Plasma::plasma-store-server` target is also added for
`plasma-store-server` executable.)
FYI: We can resolve this problem by using `COMPONENTS` feature of
`find_package()`. For example, `find_package(Arrow COMPONENTS
Parquet)` is used instead of `find_package(Parquet)`. With
`COMPONENTS`, `${PACKAGE}` is always "Arrow". So we can still install
our .cmake files to `${PREFIX}/arrow/`. But this approach breaks
backward compatibility. So I choose `${PREFIX}/${PACKAGE}/*.cmake`
approach.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@kou@zagto
ARROW-17565: [C++] Backward compatible ${PACKAGE}_shared CMake target…
bd7a588
… isn't provided (apache#14003)
This is a follow-up of ARROW-12175 / apache#13892 .
We introduced `${PACKAGE}::` namespace to all exported CMake targets
such as `Arrow::arrow_shared` and `Arrow::arrow_static`, but we also
provided no-namespaced CMake targets such as `arrow_shared` and
`arrow_static` as aliases of namespaced CMake targets.
However, the logic to provide `arrow_shared` was buggy.
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@dhruv9vats@zagto
ARROW-17632: [Python][C++] Add details of where libarrow is being fou…
e122efe
…nd during build (apache#14059)
This PR aims to add back `message(STATUS ...)` statements that printed some details about Arrow being found, its version, and the paths. These were refactored away as part of apache#13892.
Authored-by: Dhruv Vats <dhruv25vats@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zagto pushed a commit
to zagto/arrow
that referenced
this pull request
Oct 7, 2022
@raulcd@zagto
ARROW-17575: [Docs][C++] Update build document to follow new CMake pa…
b1ab2b7
…ckage (apache#14097)
This is a follow up to update our build documentation from the changes on apache#13892
Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
kou added a commit
to kou/arrow
that referenced
this pull request
Oct 11, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnings
9718fc8
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in apache#13892 but I can't remember why I pinned to
10...
@koukou
mentioned this pull request
Oct 11, 2022
 Merged

kou added a commit
that referenced
this pull request
Oct 12, 2022
@kou
ARROW-17992: [CI][C++][Conda] Remove needless clangdev/llvmdev pinnin…
2724150
…gs (#14376)
We need clang 11 or later for Gandiva.
https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=37098&view=logs&j=4c86bc1b-1091-5192-4404-c74dfaad23e7&t=41795ef0-6501-5db4-3ad4-33c0cf085626&l=1001
[6/94] Generating decimal_ops.bc
FAILED: src/gandiva/precompiled/decimal_ops.bc
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
cmd.exe /C "cd /D
D:\bld\arrow-cpp-ext_1665469353844\work\cpp\build\src\gandiva\precompiled
&& D:\bld\arrow-cpp-ext_1665469353844\_h_env\Library\bin\clang.exe
-std=c++17 -fms-compatibility -fms-compatibility-version=19.20
-DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC
-fno-use-cxa-atexit -emit-llvm -O3 -c
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc
-o
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src/gandiva/precompiled/decimal_ops.bc
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/src
-ID:/bld/arrow-cpp-ext_1665469353844/work/cpp/build/src
-ID:/bld/arrow-cpp-ext_1665469353844/_h_env/Library/include"
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src/gandiva/precompiled/decimal_ops.cc:20:
In file included from
D:/bld/arrow-cpp-ext_1665469353844/work/cpp/src\gandiva/precompiled/decimal_ops.h:20:
In file included from C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\cstdint:9:
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include\yvals_core.h:571:2:
error: STL1000: Unexpected compiler version, expected Clang 11.0.0
or newer.
#error STL1000: Unexpected compiler version, expected Clang 11.0.0 or newer.
^
This was added by me in #13892 but I can't remember why I pinned to 10...
Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@piotrwendykierpiotrwendykier
mentioned this pull request
Dec 1, 2022

Closed


This was referenced Dec 1, 2022

Closed


Closed


Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@raulcdraulcdraulcd left review comments
@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels

Component: C++Component: FlightRPCComponent: GandivaComponent: ParquetComponent: PythonComponent: R

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@kou@ursabot@raulcd@jorisvandenbossche