GH-43769: [Java] Pin Java JNI CI build to llvm 16

[danepitkin]

Rationale for this change

ORC does not support LLVM 17+ yet.

What changes are included in this PR?

• Pin llvm to v16 for the java-jni-manylinux_build job

Are these changes tested?

CI

Are there any user-facing changes?

No

• GitHub Issue: [Java][Gandiva] Could not create LLJIT instance: Symbols not found: [ llvm_orc_registerEHFrameSectionWrapper ] arrow-java#61
• GitHub Issue: ARROW-100: [C++] Computing RowBatch size #61

[laurentgo]

I'm a bit confused: is it ORC or Gandiva?

[github-actions]

⚠️ GitHub issue apache/arrow-java#61has been automatically assigned in GitHub to PR creator.

[danepitkin]

From my limited understanding, its ORC that's broken with LLVM 17, but this ORC feature is only used by the Arrow Java Gandiva module. Here's the LLVM issue I was reviewing: llvm/llvm-project#74671

[kou]

In this context, "ORC" is not "Apache ORC".
"ORC" is a JIT API in LLVM: https://llvm.org/docs/ORCv2.html

ORC JIT can work with LLVM 17+ but it requires that the main process (java in this case) exports some ORC JIT symbols such as llvm_orc_registerEHFrameSectionWrapper globally.

To do this, we have some approaches:

• Use LD_PRELOAD=/.../libLLVM-XX.so java ... (or something)
• Use RTLD_GLOBAL not RTLD_LOCAL (this is the default) for dlopen('libgandiva_jni.so')
• Remove the restriction in LLVM

See also: #39695 (comment)

If we can't do any of them, we need to pin LLVM 16-. But it's just a workaround because LLVM 16- will reach EOL eventually.

[kou]

I think that we need to create a port for LLVM 16 by ourselves to pin LLVM to 16.
Because vcpkg can provide only one LLVM version at a time. We can't reuse the official LLVM port that is for LLVM 18 now.

Or we can use old vcpkg that uses LLVM 16-. But it will have other problems because other dependencies such as aws-sdk-cpp are also downgraded...

[danepitkin]

Thank you @kou. I'll close this in favor of fixing properly. I'll repost your comment on the issue so the context is not lost.

[github-actions]

⚠️ GitHub issue #43769 has no components, please add labels for components.