GH-41571: [Java] Revert GH-41307 (#41309) by lidavidm · Pull Request #41628 · apache/arrow

lidavidm · 2024-05-12T23:16:25Z

Rationale for this change

The commit in question caused a lot of CI issues

Are these changes tested?

N/A

Are there any user-facing changes?

N/A

GitHub Issue: [CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf #41571

github-actions · 2024-05-12T23:16:48Z

⚠️ GitHub issue #41571has been automatically assigned in GitHub to PR creator.

lidavidm · 2024-05-12T23:17:20Z

@github-actions crossbow submit test--spark-java

github-actions · 2024-05-12T23:19:46Z

Revision: 8285a2ba8b8c271690766226ca527da78b67457a

Submitted crossbow builds: ursacomputing/crossbow @ actions-a804e6c132

Task	Status
java-jars
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.11-spark-master
test-conda-python-3.8-spark-v3.5.0
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

lidavidm · 2024-05-12T23:21:07Z

@danepitkin @vibhatha if the java-jars and spark jobs are this finicky, they really should be part of the regular CI and not Crossbow

lidavidm · 2024-05-12T23:59:26Z

It also appears the C++ build broke again

vibhatha · 2024-05-13T00:53:55Z

@lidavidm +1 to including java-jars with regular CIs. It has been failing for many cases very recently.

vibhatha · 2024-05-13T00:57:37Z

@felipecrv have you come across this issue: https://github.com/ursacomputing/crossbow/actions/runs/9054922130/job/24875318571#step:8:2503

FAILED: src/gandiva/precompiled/hash.bc /build/cpp/src/gandiva/precompiled/hash.bc cd /build/cpp/src/gandiva/precompiled && /opt/vcpkg/installed/arm64-linux-static-release/tools/llvm/clang-17 -std=c++17 -DGANDIVA_IR -DNDEBUG -DARROW_STATIC -DGANDIVA_STATIC -fno-use-cxa-atexit -emit-llvm -O3 -c /arrow/cpp/src/gandiva/precompiled/hash.cc -o /build/cpp/src/gandiva/precompiled/hash.bc -isystem -isystem /x86_64-redhat-linux -isystem -lpthread -I/build/cpp/src -I/arrow/cpp/src
clang-17: error: no such file or directory: '/x86_64-redhat-linux'

lidavidm · 2024-05-13T02:59:47Z

CC @kiszk @jbonofre

felipecrv · 2024-05-13T21:25:12Z

@vibhatha I never seen it because I never build Gandiva.

lidavidm · 2024-05-20T22:55:26Z

No review? I'm going to close this PR soon if it looks like we don't want it @vibhatha @danepitkin

vibhatha · 2024-05-21T00:14:40Z

@lidavidm I will review this today. Shall we rebase since there were some changes in Java and CIs.

vibhatha · 2024-05-21T00:45:49Z

@lidavidm just curious, I am comparing this PR agains: #41309.
And We the following dependency in the main
<plugin> <groupId>org.cyclonedx</groupId> <artifactId>cyclonedx-maven-plugin</artifactId> <version>2.8.0</version> </plugin>
And we had the following in the said PR.
<plugin> <groupId>org.cyclonedx</groupId> <artifactId>cyclonedx-maven-plugin</artifactId> <version>2.7.11</version> </plugin>
And in here we have 2.7.11, any specific reason we picked 2.7.11? not 2.8.0?

I'm not particularly bothered so long as CI passes. We can let dependabot upgrade it again.

vibhatha · 2024-05-21T00:55:11Z

Here in the previous PR the following component has been removed, but we haven't reverted it back here.
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration combine.self="override"> <compilerVersion>${javac.target}</compilerVersion> <source>${javac.target}</source> <target>${javac.target}</target> </configuration> </plugin>
And in the main we have
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <annotationProcessorPaths combine.children="append"> <path> <groupId>org.openjdk.jmh</groupId> <artifactId>jmh-generator-annprocess</artifactId> <version>${jmh.version}</version> </path> </annotationProcessorPaths> </configuration> </plugin>
So I hope this is okay?

I'm not particularly bothered so long as CI passes.

vibhatha

@lidavidm I added a few comments. Shall we rebase the PR and run the CIs once more?

vibhatha · 2024-05-21T01:19:15Z

@github-actions crossbow submit -g java

github-actions · 2024-05-21T01:21:40Z

Revision: 8285a2ba8b8c271690766226ca527da78b67457a

Submitted crossbow builds: ursacomputing/crossbow @ actions-4bdc9eb706

Task	Status
java-jars
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.11-spark-master
test-conda-python-3.8-spark-v3.5.0
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

vibhatha · 2024-05-21T02:10:00Z

@lidavidm the java-jars CI failures are from a previously resolved issued.

This reverts commit 9090e67.

lidavidm · 2024-05-21T03:32:36Z

@github-actions crossbow submit -g java

github-actions · 2024-05-21T03:35:06Z

Revision: 3c561a2

Submitted crossbow builds: ursacomputing/crossbow @ actions-057b79da1a

Task	Status
java-jars
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.11-spark-master
test-conda-python-3.8-spark-v3.5.0
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

lidavidm · 2024-05-21T06:22:39Z

Looks like Spark is fixed but Dataset is now broken.

lidavidm · 2024-05-21T06:23:57Z

@github-actions crossbow submit -g java

github-actions · 2024-05-21T06:26:23Z

Revision: dd15529

Submitted crossbow builds: ursacomputing/crossbow @ actions-a5a11e35cd

Task	Status
java-jars
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.11-spark-master
test-conda-python-3.8-spark-v3.5.0
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

vibhatha · 2024-05-21T07:19:14Z

Looks like Spark is fixed but Dataset is now broken.

Yes, a test is failing. Let me check if it is failing in other PRs.

lidavidm · 2024-05-21T07:22:52Z

@github-actions crossbow submit java-jars

github-actions · 2024-05-21T07:25:08Z

Revision: 290cdba

Submitted crossbow builds: ursacomputing/crossbow @ actions-d1d168f413

Task	Status
java-jars

lidavidm · 2024-05-21T08:23:38Z

@github-actions crossbow submit -g java

github-actions · 2024-05-21T08:26:05Z

Revision: 290cdba

Submitted crossbow builds: ursacomputing/crossbow @ actions-d90628c846

Task	Status
java-jars
test-conda-python-3.10-spark-v3.5.0
test-conda-python-3.11-spark-master
test-conda-python-3.8-spark-v3.5.0
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

vibhatha · 2024-05-21T09:32:02Z

@lidavidm seems like java-jars is passing 🎉

vibhatha

CI's are passing. And PR LGTM!

conbench-apache-arrow · 2024-05-21T13:45:25Z

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit e3cd0ae.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.

…1628) ### Rationale for this change The commit in question caused a lot of CI issues ### Are these changes tested? N/A ### Are there any user-facing changes? N/A * GitHub Issue: apache#41571 Authored-by: David Li <li.davidm96@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>

github-actionsBot added Component: Java awaiting committer review Awaiting committer review labels May 12, 2024

vibhatha mentioned this pull request May 13, 2024
GH-41287: [Java] ListViewVector Implementation #41285
Merged
15 tasks

lidavidm marked this pull request as ready for review May 13, 2024 01:17

lidavidm mentioned this pull request May 20, 2024
GH-41717: [Java][Vector] fix issue with ByteBuffer rewind in MessageSerializer #41718
Merged

vibhatha requested a review from jbonofre May 21, 2024 00:27

vibhatha reviewed May 21, 2024
View reviewed changes

github-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels May 21, 2024

apacheGH-41571: [Java] RevertapacheGH-41307(apache#41309)
3c561a2
This reverts commit 9090e67.

lidavidm force-pushed the revert branch from 8285a2b to 3c561a2Compare May 21, 2024 03:32

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 21, 2024

Test
dd15529

Test
290cdba

vibhatha approved these changes May 21, 2024
View reviewed changes

lidavidm merged commit e3cd0ae into apache:mainMay 21, 2024

lidavidm removed the awaiting change review Awaiting change review label May 21, 2024

lidavidm deleted the revert branch May 21, 2024 10:14

lidavidm mentioned this pull request May 21, 2024
[CI][Integration] Spark jobs are failing with problem on org.apache.arrow.flatbuf #41571
Closed

Uh oh!

Conversation

lidavidm commented May 12, 2024• edited by github-actions BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented May 12, 2024

Uh oh!

lidavidm commented May 12, 2024

Uh oh!

github-actionsBot commented May 12, 2024

Uh oh!

lidavidm commented May 12, 2024

Uh oh!

lidavidm commented May 12, 2024

Uh oh!

vibhatha commented May 13, 2024

Uh oh!

vibhatha commented May 13, 2024

Uh oh!

lidavidm commented May 13, 2024

Uh oh!

felipecrv commented May 13, 2024

Uh oh!

lidavidm commented May 20, 2024

Uh oh!

vibhatha commented May 21, 2024

Uh oh!

vibhathaMay 21, 2024

Choose a reason for hiding this comment

Uh oh!

lidavidmMay 21, 2024

Choose a reason for hiding this comment

Uh oh!

vibhathaMay 21, 2024

Choose a reason for hiding this comment

Uh oh!

lidavidmMay 21, 2024

Choose a reason for hiding this comment

Uh oh!

vibhatha left a comment

Choose a reason for hiding this comment

Uh oh!

vibhatha commented May 21, 2024

Uh oh!

github-actionsBot commented May 21, 2024

Uh oh!

vibhatha commented May 21, 2024

Uh oh!

lidavidm commented May 21, 2024

Uh oh!

github-actionsBot commented May 21, 2024

Uh oh!

lidavidm commented May 21, 2024

Uh oh!

lidavidm commented May 21, 2024

Uh oh!

github-actionsBot commented May 21, 2024

Uh oh!

vibhatha commented May 21, 2024

Uh oh!

lidavidm commented May 21, 2024

Uh oh!

github-actionsBot commented May 21, 2024

Uh oh!

lidavidm commented May 21, 2024

Uh oh!

github-actionsBot commented May 21, 2024

Uh oh!

vibhatha commented May 21, 2024

Uh oh!

vibhatha left a comment

Choose a reason for hiding this comment

Uh oh!

conbench-apache-arrowBot commented May 21, 2024

Uh oh!

Reviewers

Assignees

lidavidm commented May 12, 2024•
edited by github-actions Bot
Loading