GH-55: [Gandiva] Re-enable tests

[lriggs]

FixesGH-55.

Rationale for this change

I fixed the linking error here: apache/arrow#45114

What changes are included in this PR?

Reenabling some disabled tests.

Are these changes tested?

Yes, they are tests.

Are there any user-facing changes?

No.

[lriggs]

How do you specify the arrow version used to build/test with arrow-java (so that I know it has my fix)? Does it always use the latest?

[kou]

We use the latest release for push/pull request:

arrow-java/.github/workflows/rc.yml

Lines 110 to 113 in a5b8604

	- name: Download the latest Apache Arrow C++
	if: github.event_name != 'schedule'
	run: |
	ci/scripts/download_cpp.sh

We use the main branch for schedule:

arrow-java/.github/workflows/rc.yml

Lines 114 to 119 in a5b8604

	- name: Checkout Apache Arrow C++
	if: github.event_name == 'schedule'
	uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
	with:
	repository: apache/arrow
	path: arrow

We need to update

arrow-java/.env

Line 57 in a5b8604

VCPKG="943c5ef1c8f6b5e6ced092b242c8299caae2ff01"# 2024.04.26 Release

to use vcpkg configuration introduced by apache/arrow#45114 .

[lriggs]

Thanks. I updated vcpkg as well now.

[lidavidm]

Looks like one more minor CI nit:

 Error: /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterProjectTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

[lidavidm]

There's others - are you able to run the build locally at all to test?

 Error: /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/FilterTest.java:37:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]
Error: /home/runner/work/arrow-java/arrow-java/gandiva/src/test/java/org/apache/arrow/gandiva/evaluator/ProjectorDecimalTest.java:45:8: Unused import: org.junit.jupiter.api.Disabled. [UnusedImports]

[lriggs]

Now its failing with "Gandiva Failed to make LLVM module due to Could not create LLJIT instance: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ]" which is what should be fixed in my other change in apache/arrow.

Seems to be using the 18.2 jars: /home/runner/.m2/repository/org/apache/arrow/arrow-avro/18.2.0/

[lidavidm]

Ah, the CI here uses the latest released version of the C++ libraries, not the development version...

[lidavidm]

We could either make the skips conditional on the version of the C++ code somehow, or otherwise just always test against the dev release?

[kou]

C++ version must not be related.
apache/arrow#45114 just updated LLVM version.

[lidavidm]

But we're building 19.0.0, not main: https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595

[kou]

Ah, I thought that updating VCPKG in .env is enough to use LLVM 18 but we refer ci/vcpkg in apache/arrow here...:

arrow-java/ci/docker/vcpkg-jni.dockerfile

Line 26 in 34e2b08

--x-manifest-root=/arrow/ci/vcpkg \

The log shows that we use LLVM 17:

https://github.com/apache/arrow-java/actions/runs/13298859368/job/37137015507?pr=595#step:10:1663

 -- Found LLVMAlt: 17.0.2

OK. Let's use main for now:

diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
index faad5983..f2666a31 100644
--- a/.github/workflows/rc.yml+++ b/.github/workflows/rc.yml@@ -299,13 +299,13 @@ jobs:
shell: bash
run: |
tar -xf apache-arrow-java-*.tar.gz --strip-components=1
- - name: Download the latest Apache Arrow C++- if: github.event_name != 'schedule'- shell: bash- run: |- ci/scripts/download_cpp.sh+ # - name: Download the latest Apache Arrow C+++ # if: github.event_name != 'schedule'+ # shell: bash+ # run: |+ # ci/scripts/download_cpp.sh
- name: Checkout Apache Arrow C++
- if: github.event_name == 'schedule'+ # if: github.event_name == 'schedule'
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: apache/arrow

[lidavidm]

Ah, I see what you meant by the original comment now...

[lriggs]

I'll try your suggested change @kou

[lriggs]

Latest failures were because I only updated the windows build in rc.yml. Just fixed that for mac and linux.

[kou]

Oh, sorry. It's my fault...
We just need to use main only for Linux because Linux build only uses vcpkg to install LLVM.

[lriggs]

Strange, it still seems to be using the old vcpkg.

Does arrow-java need something like this? https://github.com/apache/arrow/blob/main/ci/scripts/install_vcpkg.sh

2025-02-14T00:07:03.2003955Z CMake Warning at /opt/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package):
2025-02-14T00:07:03.2004967Z Could not find a configuration file for package "LLVM" that is compatible
2025-02-14T00:07:03.2005618Z with requested version "19.1".

....

2025-02-14T00:07:03.2600773Z -- Found LLVMAlt: 17.0.2
2025-02-14T00:07:03.2601530Z -- Using LLVMConfig.cmake in: /opt/vcpkg/installed/amd64-linux-static-release/share/llvm

[kou]

Ah, we need to expire a cache when we change vcpkg version. Could you try this?

diff --git a/compose.yaml b/compose.yaml
index 58a19676..b125c3c9 100644
--- a/compose.yaml+++ b/compose.yaml@@ -92,12 +92,12 @@ services:
# Usage:
# docker compose build vcpkg-jni
# docker compose run vcpkg-jni
- image: ${REPO}:${ARCH}-vcpkg-jni+ image: ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
build:
context: .
dockerfile: ci/docker/vcpkg-jni.dockerfile
cache_from:
- - ${REPO}:${ARCH}-vcpkg-jni+ - ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
args:
base: ${ARROW_REPO}:${ARCH}-python-${PYTHON}-wheel-manylinux-2014-vcpkg-${VCPKG}
volumes:

[lriggs]

Ok, just pushed a change to expire the cache. Hopefully it works.

[lidavidm]

Looks like that worked!