Skip to content

TEZ-4669: Spotbugs fails in precommit - #457

Merged
abstractdog merged 1 commit into
apache:masterfrom
abstractdog:TEZ-4669
Jan 29, 2026
Merged

TEZ-4669: Spotbugs fails in precommit#457
abstractdog merged 1 commit into
apache:masterfrom
abstractdog:TEZ-4669

Conversation

@abstractdog

@abstractdogabstractdog commented Jan 26, 2026

Copy link
Copy Markdown
Contributor

Upgraded sisu-inject-plexus that spotbugs plugin depends on to make it work (it was mainly a guesswork).

Regarding the process:

  1. first took a look at: https://issues.apache.org/jira/secure/attachment/13080464/maven_build_spotbugs.log
  2. found some related dependencies (to the exception reported, see in Jira):
org.sonatype.sisu:sisu-inject-plexus:jar:1.4.2:compile
  1. upgrading those locally for spotbugs plugin worked

solution double-checked locally too:

find . -name "spotbugs.html"
./hadoop-shim/target/site/spotbugs.html
./tez-mapreduce/target/site/spotbugs.html
./tez-common/target/site/spotbugs.html
./tez-runtime-internals/target/site/spotbugs.html
./tez-examples/target/site/spotbugs.html
./tez-dag/target/site/spotbugs.html
./tez-runtime-library/target/site/spotbugs.html
...

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

I was also debugging this 😅. I think based on the stacktrace the issue is because of missing VelocityComponentConfigurator class used by DefaultVelocityComponent constructor

but the 1st parameter of org.codehaus.plexus.velocity.internal.DefaultVelocityComponent.<init>(Unknown Source) is not @Nullable
at org.eclipse.sisu.wire.LocatorWiring
while locating org.codehaus.plexus.velocity.VelocityComponentConfigurator

So, if we somehow add that class, it should work. I was going to add

jar tvf org/codehaus/plexus/plexus-velocity/2.2.1/plexus-velocity-2.2.1.jar | rg VelocityComponentConfigurator
216 Sat Apr 05 14:56:42 IST 2025 org/codehaus/plexus/velocity/VelocityComponentConfigurator.class

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec16m 22sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall11m 31smaster passed
+1 💚compile2m 34smaster passed
+1 💚javadoc1m 41smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 1sthe patch passed
+1 💚codespell1m 53sNo new issues.
+1 💚compile2m 33sthe patch passed
+1 💚javac2m 33sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚javadoc1m 30sthe patch passed
_ Other Tests _
+1 💚unit74m 17sroot in the patch passed.
+1 💚asflicense0m 35sThe patch does not generate ASF License warnings.
119m 46s
SubsystemReport/Notes
DockerClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/1/artifact/out/Dockerfile
GITHUB PR#457
Optional Testsdupname asflicense javac javadoc unit codespell detsecrets xmllint compile
unameLinux 5ee911e4fd5e 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-457/src/.yetus/personality.sh
git revisionmaster / e841c92
Default JavaUbuntu-21.0.9+10-Ubuntu-124.04
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/1/testReport/
Max. process+thread count1348 (vs. ulimit of 5500)
modulesC: . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/1/console
versionsgit=2.43.0 maven=3.8.7 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 44sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 1sdetect-secrets was not available.
+0 🆗xmllint0m 1sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep3m 42sMaven dependency ordering for branch
+1 💚mvninstall13m 25smaster passed
+1 💚compile3m 4smaster passed
+1 💚javadoc2m 58smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 18sMaven dependency ordering for patch
+1 💚mvninstall6m 31sthe patch passed
+1 💚codespell2m 4sNo new issues.
+1 💚compile3m 52sthe patch passed
+1 💚javac3m 52sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚javadoc2m 32sthe patch passed
_ Other Tests _
-1 ❌unit16m 59s/patch-unit-tez-api.txttez-api in the patch passed.
-1 ❌unit15m 56s/patch-unit-root.txtroot in the patch failed.
+0 🆗asflicense1m 3sASF License check generated no output?
75m 12s
ReasonTests
Failed junit teststez.common.TestTezCommonUtils
tez.common.TestTezCommonUtils
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/4/artifact/out/Dockerfile
GITHUB PR#457
Optional Testsdupname asflicense javac javadoc unit codespell detsecrets xmllint compile
unameLinux b50d1ab5bc65 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-457/src/.yetus/personality.sh
git revisionmaster / 69a7cb2
Default JavaUbuntu-21.0.9+10-Ubuntu-124.04
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/4/testReport/
Max. process+thread count412 (vs. ulimit of 5500)
modulesC: tez-api . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/4/console
versionsgit=2.43.0 maven=3.8.7 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@ayushtkn

Copy link
Copy Markdown
Member

I think the CI didn't kick in the Spotbugs?

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

I think the CI didn't kick in the Spotbugs?

ack, apparently fake pom.xml change is not enough, let me do a fake .java change

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 46sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep3m 33sMaven dependency ordering for branch
-1 ❌mvninstall8m 55s/branch-mvninstall-root.txtroot in master failed.
+1 💚compile4m 0smaster passed
+1 💚checkstyle1m 21smaster passed
+1 💚javadoc2m 11smaster passed
-1 ❌spotbugs1m 15s/branch-spotbugs-tez-api.txttez-api in master failed.
-1 ❌spotbugs0m 22s/branch-spotbugs-root.txtroot in master failed.
_ Patch Compile Tests _
+0 🆗mvndep0m 13sMaven dependency ordering for patch
+1 💚mvninstall5m 51sthe patch passed
+1 💚codespell1m 2sNo new issues.
+1 💚compile3m 1sthe patch passed
+1 💚javac3m 1sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle1m 6sthe patch passed
+1 💚javadoc1m 58sthe patch passed
+1 💚spotbugs9m 25sthe patch passed
_ Other Tests _
+1 💚unit2m 36stez-api in the patch passed.
+1 💚unit73m 8sroot in the patch passed.
+1 💚asflicense0m 59sThe patch does not generate ASF License warnings.
124m 37s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/6/artifact/out/Dockerfile
GITHUB PR#457
Optional Testsdupname asflicense javac javadoc unit codespell detsecrets xmllint compile spotbugs checkstyle
unameLinux 315a2b9f4b21 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-457/src/.yetus/personality.sh
git revisionmaster / a39157c
Default JavaUbuntu-21.0.9+10-Ubuntu-124.04
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/6/testReport/
Max. process+thread count1323 (vs. ulimit of 5500)
modulesC: tez-api . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/6/console
versionsgit=2.43.0 maven=3.8.7 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

yikes, the same error, even though it worked for me on ubuntu, need to double-check this

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

I was also debugging this 😅. I think based on the stacktrace the issue is because of missing VelocityComponentConfigurator class used by DefaultVelocityComponent constructor

but the 1st parameter of org.codehaus.plexus.velocity.internal.DefaultVelocityComponent.<init>(Unknown Source) is not @Nullable
at org.eclipse.sisu.wire.LocatorWiring
while locating org.codehaus.plexus.velocity.VelocityComponentConfigurator

So, if we somehow add that class, it should work. I was going to add

jar tvf org/codehaus/plexus/plexus-velocity/2.2.1/plexus-velocity-2.2.1.jar | rg VelocityComponentConfigurator
216 Sat Apr 05 14:56:42 IST 2025 org/codehaus/plexus/velocity/VelocityComponentConfigurator.class

@abstractdog , can you please give 1 try with the plexus-velocity dependency? As the sisu-inject-plexus didn't have the VelocityComponentConfigurator class (checked using jar tvf). Also the jar is old (2015) and had CVE's

<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-velocity</artifactId>
<version>2.3.0</version>
</dependency>

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec16m 33sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep3m 30sMaven dependency ordering for branch
+1 💚mvninstall12m 16smaster passed
+1 💚compile3m 3smaster passed
+1 💚checkstyle1m 29smaster passed
+1 💚javadoc2m 11smaster passed
-1 ❌spotbugs1m 38s/branch-spotbugs-tez-api.txttez-api in master failed.
-1 ❌spotbugs0m 21s/branch-spotbugs-root.txtroot in master failed.
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall5m 13sthe patch passed
+1 💚codespell1m 48sNo new issues.
+1 💚compile3m 0sthe patch passed
+1 💚javac3m 0sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle1m 6sthe patch passed
+1 💚javadoc1m 56sthe patch passed
+1 💚spotbugs9m 27sthe patch passed
_ Other Tests _
+1 💚unit2m 39stez-api in the patch passed.
+1 💚unit68m 57sroot in the patch passed.
+1 💚asflicense0m 58sThe patch does not generate ASF License warnings.
139m 18s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/artifact/out/Dockerfile
GITHUB PR#457
Optional Testsdupname asflicense javac javadoc unit codespell detsecrets xmllint compile spotbugs checkstyle
unameLinux 2ff10113178c 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-home/workspace/tez-multibranch_PR-457/src/.yetus/personality.sh
git revisionmaster / 62fd11e
Default JavaUbuntu-21.0.9+10-Ubuntu-124.04
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/testReport/
Max. process+thread count2097 (vs. ulimit of 5500)
modulesC: tez-api . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/console
versionsgit=2.43.0 maven=3.8.7 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

plexus-velocity solves the problem on my testing ubuntu server, but not in precommit, unfortunately, as a next step, I'm going for the yetus image and doing it inside

@abstractdog

abstractdog commented Jan 28, 2026

Copy link
Copy Markdown
ContributorAuthor

plexus-velocity solves the problem on my testing ubuntu server, but not in precommit, unfortunately, as a next step, I'm going for the yetus image and doing it inside

UPDATE: the pom.xml change solves the problem in the yetus docker container too for me locally...this is getting more and more interesting

in the failing job I can see:

[INFO] Downloading from maven2-repository.atlassian: https://maven.atlassian.com/repository/public/org/codehaus/plexus/plexus-velocity/2.2.0/plexus-velocity-2.2.0.pom
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-velocity/2.2.0/plexus-velocity-2.2.0.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-velocity/2.2.0/plexus-velocity-2.2.0.pom (4.1 kB at 50 kB/s)

whereas for me locally:

Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-velocity/2.3.0/plexus-velocity-2.3.0.jar (6.2 kB at 66 kB/s)

apparently spotbugs in precommit doesn't depend on the fix

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

apparently spotbugs in precommit doesn't depend on the fix

Yes, looks like it:

tez/Jenkinsfile

Lines 119 to 120 in 03b5f1d

# -1 spotbugs issues that show up prior to the patch being applied
# YETUS_ARGS+=("--spotbugs-strict-precheck")

So, if we merge this then for new PR's this change should be ok I believe!

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

apparently spotbugs in precommit doesn't depend on the fix

Yes, looks like it:

tez/Jenkinsfile

Lines 119 to 120 in 03b5f1d

# -1 spotbugs issues that show up prior to the patch being applied
# YETUS_ARGS+=("--spotbugs-strict-precheck")

So, if we merge this then for new PR's this change should be ok I believe!

I believe so, btw, not sure how "spotbugs-strict-precheck" is related to this behavior, how did you mean?

@Aggarwal-Raghav

Copy link
Copy Markdown
Contributor

I believe so, btw, not sure how "spotbugs-strict-precheck" is related to this behavior, how did you mean?

My mistake, I got confused.

@ayushtkn

Copy link
Copy Markdown
Member

I think SpotBugs passed:
image
It passes in the patch compile test, It failed in the master build, which is built without the patch. The way Yetus works is it builds first without the patch, that results in _master_compile_tests, then applies the patch and then again those tests. Since master is broken it will always be -1 there.

The patch results:
root:

https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/artifact/out/patch-spotbugs-root.txt
tez-api:
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/artifact/out/patch-spotbugs-tez-api.txt

Am I missing something?

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

I think SpotBugs passed: image It passes in the patch compile test, It failed in the master build, which is built without the patch. The way Yetus works is it builds first without the patch, that results in _master_compile_tests, then applies the patch and then again those tests. Since master is broken it will always be -1 there.

The patch results: root:

https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/artifact/out/patch-spotbugs-root.txttez-api:https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/7/artifact/out/patch-spotbugs-tez-api.txt

Am I missing something?

LOL yes! :) the master branch doesn't contain the patch, that's the point
good catch, let me cleanup the patch

@abstractdog

Copy link
Copy Markdown
ContributorAuthor

so with the patch:

[INFO] --- spotbugs-maven-plugin:4.9.3.0:spotbugs (default-cli) @ tez-api ---
[INFO] Downloading from maven2-repository.atlassian: https://maven.atlassian.com/repository/public/org/codehaus/plexus/plexus-velocity/2.3.0/plexus-velocity-2.3.0.pom
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-velocity/2.3.0/plexus-velocity-2.3.0.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-velocity/2.3.0/plexus-velocity-2.3.0.pom (4.3 kB at 13 kB/s)

@ayushtkn : could this be merged now?

@ayushtknayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

One comment, feel free to merge once you change

Comment threadpom.xml Outdated
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-velocity</artifactId>
<version>2.3.0</version>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you define it as a variable and use, all other dependencies have defined variable, I think we did that explicitly in some patch in past to define variable

@abstractdogabstractdogJan 28, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, I was hesitating to do that: it's just a workaround that ideally should go away with a spotbugs upgrade, does it "deserve" a top-level property?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say lets keep the code in sync, once we upgrade and it is no longer required we can drop it like a normal dependency, it is just one more line to rollback and keep the code in sync

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack, it won't hurt :)
please see bef9055
assuming your approval holds with this change, I'm waiting for precommit tests

@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 46sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+0 🆗xmllint0m 0sxmllint was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall14m 48smaster passed
+1 💚compile2m 26smaster passed
+1 💚javadoc1m 50smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 43sthe patch passed
+1 💚codespell1m 47sNo new issues.
+1 💚compile2m 24sthe patch passed
+1 💚javac2m 24sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚javadoc1m 17sthe patch passed
_ Other Tests _
+1 💚unit71m 39sroot in the patch passed.
+1 💚asflicense0m 40sThe patch does not generate ASF License warnings.
104m 6s
SubsystemReport/Notes
DockerClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/11/artifact/out/Dockerfile
GITHUB PR#457
Optional Testsdupname asflicense javac javadoc unit codespell detsecrets xmllint compile
unameLinux dbf2fd162841 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-agent/workspace/tez-multibranch_PR-457/src/.yetus/personality.sh
git revisionmaster / bef9055
Default JavaUbuntu-21.0.9+10-Ubuntu-124.04
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/11/testReport/
Max. process+thread count1350 (vs. ulimit of 5500)
modulesC: . U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-457/11/console
versionsgit=2.43.0 maven=3.8.7 codespell=2.4.1
Powered byApache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@abstractdog
abstractdog merged commit f09ba7f into apache:masterJan 29, 2026
4 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@abstractdog@Aggarwal-Raghav@tez-yetus@ayushtkn