TEZ-4471: Precommit fails while building yetus image by abstractdog · Pull Request #265 · apache/tez

abstractdog · 2023-02-10T10:14:04Z

No description provided.

tez-yetus · 2023-02-10T10:14:40Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/1/console in case of problems.

abstractdog · 2023-02-10T10:27:35Z

I haven't been able to reproduce the issue locally, so simply upgraded to the next LTS ubuntu version, seems to work

I'm keeping this open for a couple of hours for reviews, then merge it anyway to unblock precommit

tez-yetus · 2023-02-10T10:42:57Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/2/console in case of problems.

tez-yetus · 2023-02-10T12:07:34Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/3/console in case of problems.

ayushtkn

LGTM.
Anyway good to upgrade to the latest LTS release

tez-yetus · 2023-02-12T07:58:45Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/4/console in case of problems.

tez-yetus · 2023-02-12T08:36:26Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/5/console in case of problems.

abstractdog · 2023-02-12T08:55:15Z

after some hacking I realized the root cause of installation failures in tez-ui is the absence of the bzip2 package, handling it with the latest commit
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/2/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_352-8u352-ga-1~22.04-b08.txt

[INFO] tar (child): bzip2: Cannot exec: No such file or directory

ayushtkn · 2023-02-12T09:01:15Z

So, is there any commit which added the dependency on bzip2? Just curious then why didn't it fail in that PR jenkins report

abstractdog · 2023-02-12T09:06:56Z

it's because tez precommit flow depends on which files have been modified, e.g. when you only change the Dockerfile, there is no full precommit run, only the docker image is built
that's why I added a fake commit to change pom.xml to run full precommit:
e87a0de

actually, the bzip2 change is here:
b0fce54

once the precommit passes, I'll simply remove the pom.xml hack

tez-yetus · 2023-02-12T12:41:27Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/6/console in case of problems.

tez-yetus · 2023-02-12T12:58:51Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/7/console in case of problems.

tez-yetus · 2023-02-12T15:00:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	22m 47s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+0 🆗	mvndep	5m 50s	Maven dependency ordering for branch
+1 💚	mvninstall	10m 43s	master passed
+1 💚	compile	2m 57s	master passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	compile	2m 46s	master passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
+1 💚	javadoc	2m 38s	master passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javadoc	2m 6s	master passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 39s	the patch passed
+1 💚	compile	2m 59s	the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javac	2m 59s	the patch passed
+1 💚	compile	2m 45s	the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
+1 💚	javac	2m 45s	the patch passed
+1 💚	hadolint	0m 2s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	2m 26s	the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javadoc	2m 8s	the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
		_ Other Tests _
+1 💚	unit	1m 35s	tez-ui in the patch passed.
+1 💚	unit	70m 6s	root in the patch passed.
+1 💚	asflicense	1m 6s	The patch does not generate ASF License warnings.
		138m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/6/artifact/out/Dockerfile
GITHUB PR	#265
JIRA Issue	TEZ-4471
Optional Tests	dupname asflicense hadolint shellcheck shelldocs javac javadoc unit xml compile
uname	Linux a3ccd766fff2 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `39e5a8e`
Default Java	Private Build-1.8.0_352-8u352-ga-1~22.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~22.04-b08
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/6/testReport/
Max. process+thread count	2100 (vs. ulimit of 5500)
modules	C: tez-ui . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/6/console
versions	git=2.34.1 maven=3.6.3 shellcheck=0.7.1 hadolint=1.18.0-0-g76eee5c
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

tez-yetus · 2023-02-12T15:00:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+0 🆗	mvndep	5m 57s	Maven dependency ordering for branch
+1 💚	mvninstall	10m 41s	master passed
+1 💚	compile	2m 57s	master passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	compile	2m 48s	master passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
+1 💚	javadoc	2m 39s	master passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javadoc	2m 7s	master passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 38s	the patch passed
+1 💚	compile	2m 55s	the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javac	2m 55s	the patch passed
+1 💚	compile	2m 48s	the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
+1 💚	javac	2m 48s	the patch passed
+1 💚	hadolint	0m 3s	There were no new hadolint issues.
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	javadoc	2m 27s	the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04
+1 💚	javadoc	2m 5s	the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~22.04-b08
		_ Other Tests _
+1 💚	unit	1m 37s	tez-ui in the patch passed.
+1 💚	unit	75m 16s	root in the patch passed.
+1 💚	asflicense	1m 5s	The patch does not generate ASF License warnings.
		122m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/7/artifact/out/Dockerfile
GITHUB PR	#265
JIRA Issue	TEZ-4471
Optional Tests	dupname asflicense hadolint shellcheck shelldocs javac javadoc unit xml compile
uname	Linux 3eb61d42993a 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `39e5a8e`
Default Java	Private Build-1.8.0_352-8u352-ga-1~22.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu222.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~22.04-b08
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/7/testReport/
Max. process+thread count	2100 (vs. ulimit of 5500)
modules	C: tez-ui . U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-265/7/console
versions	git=2.34.1 maven=3.6.3 shellcheck=0.7.1 hadolint=1.18.0-0-g76eee5c
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

abstractdog · 2023-02-12T20:12:12Z

okay, latest precommit passed, I'm about to proceed with it tomorrow

rbalamohan · 2023-02-13T07:58:09Z

LGTM +1

TEZ-4471: Precommit fails while building yetus image
26cb090

abstractdog requested a review from rbalamohan February 10, 2023 10:27

This comment was marked as outdated.
Sign in to view

abstractdog force-pushed the TEZ-4471 branch from e87a0de to 26cb090Compare February 10, 2023 12:06

This comment was marked as outdated.
Sign in to view

ayushtkn approved these changes Feb 11, 2023
View reviewed changes

abstractdog force-pushed the TEZ-4471 branch from 665dfb4 to b0fce54Compare February 12, 2023 08:35

This comment was marked as outdated.
Sign in to view

install bzip2
cd43ba8

abstractdog force-pushed the TEZ-4471 branch from b0fce54 to 86f867bCompare February 12, 2023 12:40

OPENSSL_CONF workaround
130324f

abstractdog force-pushed the TEZ-4471 branch from 86f867b to 130324fCompare February 12, 2023 12:58

rbalamohan approved these changes Feb 13, 2023
View reviewed changes

abstractdog merged commit e236f51 into apache:masterFeb 13, 2023

Uh oh!

Conversation

abstractdog commented Feb 10, 2023

Uh oh!

tez-yetus commented Feb 10, 2023

Uh oh!

abstractdog commented Feb 10, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

tez-yetus commented Feb 10, 2023

Uh oh!

tez-yetus commented Feb 10, 2023

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

tez-yetus commented Feb 12, 2023

Uh oh!

tez-yetus commented Feb 12, 2023

Uh oh!

abstractdog commented Feb 12, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ayushtkn commented Feb 12, 2023

Uh oh!

abstractdog commented Feb 12, 2023

Uh oh!

This comment was marked as outdated.

tez-yetus commented Feb 12, 2023

Uh oh!

tez-yetus commented Feb 12, 2023

Uh oh!

tez-yetus commented Feb 12, 2023

Uh oh!

tez-yetus commented Feb 12, 2023

Uh oh!

abstractdog commented Feb 12, 2023

Uh oh!

rbalamohan commented Feb 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

abstractdog commented Feb 10, 2023•
edited
Loading

abstractdog commented Feb 12, 2023•
edited
Loading