HBASE-23779 Up the default fork count to make builds complete faster;… by saintstack · Pull Request #1461 · apache/hbase

saintstack · 2020-04-08T21:45:22Z

… make count relative to CPU count

Halve the build speed by doubling the -C count from 0.25C to 0.5C.
Pass mvn a -T of 0.5C too.

Apache-HBase · 2020-04-08T21:51:07Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console in case of problems.

Apache-HBase · 2020-04-08T21:51:07Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console in case of problems.

Apache-HBase · 2020-04-08T21:51:07Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console in case of problems.

Apache-HBase · 2020-04-08T22:10:52Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
		_ Patch Compile Tests _
+1 💚	shellcheck	0m 0s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	12m 36s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		20m 38s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	dupname asflicense shellcheck shelldocs hadoopcheck xml
uname	Linux 126a8e104277 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Max. process+thread count	62 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2020-04-08T23:25:35Z

+ # See below for more on -T:
+ # https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3
+ forkcount="0.5C"
+ extra="-T${forkcount} -Dsurefire.firstPartForkCount=${forkcount} -Dsurefire.secondPartForkCount=${forkcount} -DHBasePatchProcess"


How does maven --threads interact with surefire forks? Isn't it the case that we get forkCount maven process thread each building a module simultaneously, and each of those modules is launching up to forkCount concurrent test jvms? Means we're looking at 64 concurrent test processes? Seems high to me.
If it is indeed launching like this, I would want -T{forkCount} when running compile/install, but i want just a single core when running test.
I took a read through the surefire docs but I don't have a better handle on this than when I started.
If Maven is instead managing an upper-bound on concurrent test jvms at ${forkcount}, I say +1.

Thanks for the review.
bq. How does maven --threads interact with surefire forks?
There is no correlation. There may be cases where it acts as a multiplier where modules are independent of each other but generally, there is little opportunity for -T benefit in hbase builds.
I went back and checked a few machines. Couldn't find instance where fork count rose above forkcount limit. Doesn't mean it doesn't happen.
Let me untangle the two.

I simplified the patch after the above. Thanks.

Hmm... on big box, with 1.0C and no T, build takes 1hr 15mins. With 0.5C and T==2, takes 53minutes. There are probably spurts of lots of processes but interesting that overall time taken goes down significantly. Let me come back and set the T in another follow-on issue.

Opened HBASE-24150 Might be more speed up there than here.

Good one, I left you a comment over there too.
I wonder if there's a docker param that limits the number of processes a container can launch... could set that to the expected max parallel jvms and see if it breaches.

Will poke around....

Apache-HBase · 2020-04-09T00:01:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 22s	branch-2 passed
+1 💚	compile	1m 57s	branch-2 passed
+1 💚	shadedjars	4m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 22s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 4s	the patch passed
+1 💚	compile	1m 58s	the patch passed
+1 💚	javac	1m 58s	the patch passed
+1 💚	shadedjars	4m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 14s	the patch passed
		_ Other Tests _
-1 ❌	unit	104m 6s	root in the patch failed.
		130m 48s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a4b422329b3e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	1.8.0_232
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/testReport/
Max. process+thread count	4309 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T00:16:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 56s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 49s	branch-2 passed
+1 💚	compile	2m 30s	branch-2 passed
+1 💚	shadedjars	5m 41s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 18s	root in branch-2 failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 33s	the patch passed
+1 💚	compile	2m 29s	the patch passed
+1 💚	javac	2m 29s	the patch passed
+1 💚	shadedjars	5m 29s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 16s	root in the patch failed.
		_ Other Tests _
-1 ❌	unit	116m 20s	root in the patch failed.
		146m 9s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9b5f2f07aad6 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/testReport/
Max. process+thread count	6625 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/1/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T00:21:09Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console in case of problems.

Apache-HBase · 2020-04-09T00:21:12Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console in case of problems.

Apache-HBase · 2020-04-09T00:21:27Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console in case of problems.

Apache-HBase · 2020-04-09T00:38:28Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
		_ Patch Compile Tests _
+1 💚	shellcheck	0m 2s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	10m 44s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
		_ Other Tests _
+1 💚	asflicense	0m 19s	The patch does not generate ASF License warnings.
		18m 14s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	dupname asflicense shellcheck shelldocs hadoopcheck xml
uname	Linux 6796ca787704 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Max. process+thread count	76 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T02:34:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	5m 36s	branch-2 passed
+1 💚	compile	2m 59s	branch-2 passed
+1 💚	shadedjars	4m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 46s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 5s	the patch passed
+1 💚	compile	2m 59s	the patch passed
+1 💚	javac	3m 0s	the patch passed
+1 💚	shadedjars	4m 20s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 43s	the patch passed
		_ Other Tests _
-1 ❌	unit	99m 11s	root in the patch failed.
		133m 57s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux dca13b9f23b7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	1.8.0_232
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/testReport/
Max. process+thread count	7529 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T03:47:28Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 58s	Docker mode activated.
-0 ⚠️	yetus	0m 8s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	8m 17s	branch-2 passed
+1 💚	compile	4m 25s	branch-2 passed
+1 💚	shadedjars	5m 54s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 23s	root in branch-2 failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	7m 15s	the patch passed
+1 💚	compile	4m 25s	the patch passed
+1 💚	javac	4m 25s	the patch passed
+1 💚	shadedjars	5m 58s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 19s	root in the patch failed.
		_ Other Tests _
+1 💚	unit	165m 35s	root in the patch passed.
		206m 53s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux bd34ff2524ef 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/testReport/
Max. process+thread count	6467 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/2/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T03:59:20Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console in case of problems.

Apache-HBase · 2020-04-09T03:59:23Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console in case of problems.

Apache-HBase · 2020-04-09T03:59:25Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console in case of problems.

Apache-HBase · 2020-04-09T04:17:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 46s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	shelldocs	0m 0s	Shelldocs was not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
		_ Patch Compile Tests _
+1 💚	shellcheck	0m 2s	There were no new shellcheck issues.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	10m 55s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
		_ Other Tests _
+1 💚	asflicense	0m 17s	The patch does not generate ASF License warnings.
		18m 50s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	dupname asflicense shellcheck shelldocs hadoopcheck xml
uname	Linux 277bf39c0c66 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Max. process+thread count	76 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

saintstack · 2020-04-09T04:24:21Z

Rerunning tests.

Apache-HBase · 2020-04-09T05:45:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 46s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	5m 38s	branch-2 passed
+1 💚	compile	3m 7s	branch-2 passed
+1 💚	shadedjars	4m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 49s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 9s	the patch passed
+1 💚	compile	2m 57s	the patch passed
+1 💚	javac	2m 56s	the patch passed
+1 💚	shadedjars	4m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 43s	the patch passed
		_ Other Tests _
-1 ❌	unit	73m 9s	root in the patch failed.
		107m 32s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 153ad2e0026d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	1.8.0_232
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/testReport/
Max. process+thread count	5159 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T06:45:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 21s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	7m 17s	branch-2 passed
+1 💚	compile	4m 30s	branch-2 passed
+1 💚	shadedjars	5m 56s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 20s	root in branch-2 failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	7m 12s	the patch passed
+1 💚	compile	4m 22s	the patch passed
+1 💚	javac	4m 22s	the patch passed
+1 💚	shadedjars	5m 55s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 20s	root in the patch failed.
		_ Other Tests _
-1 ❌	unit	127m 22s	root in the patch failed.
		166m 30s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6a524b914f05 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `611c62f`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/testReport/
Max. process+thread count	3855 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/3/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

ndimiduk · 2020-04-09T16:49:27Z

 clear_personality_queue

- extra="-DHBasePatchProcess"
+ extra="-DHBasePatchProcess"


… make count relative to CPU count Halve the build speed by doubling the -C count from 0.25C to 0.5C.

saintstack · 2020-04-09T17:29:45Z

Fix the white space. Re-run. I don't like the test failures I'm seeing. jenkins doesn't like this 0.5C for forkcount. HBASE-24150 might have higher yield and be more stable. Lets see. Will see how this build goes and then will push a non-change for compare.

Apache-HBase · 2020-04-09T17:48:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
		_ Patch Compile Tests _
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	10m 33s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.
		_ Other Tests _
+1 💚	asflicense	0m 17s	The patch does not generate ASF License warnings.
		17m 59s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	dupname asflicense hadoopcheck xml
uname	Linux f5f1787e4e42 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5012241`
Max. process+thread count	76 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T19:53:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 46s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	6m 26s	branch-2 passed
+1 💚	compile	3m 45s	branch-2 passed
+1 💚	shadedjars	5m 13s	branch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 21s	root in branch-2 failed.
		_ Patch Compile Tests _
+1 💚	mvninstall	6m 20s	the patch passed
+1 💚	compile	3m 48s	the patch passed
+1 💚	javac	3m 48s	the patch passed
+1 💚	shadedjars	6m 28s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 19s	root in the patch failed.
		_ Other Tests _
-1 ❌	unit	105m 53s	root in the patch failed.
		142m 55s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 90407d09b37d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5012241`
Default Java	2020-01-14
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt
javadoc	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/testReport/
Max. process+thread count	7709 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-04-09T20:39:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	6m 22s	branch-2 passed
+1 💚	compile	3m 22s	branch-2 passed
+1 💚	shadedjars	4m 46s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 51s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 37s	the patch passed
+1 💚	compile	3m 15s	the patch passed
+1 💚	javac	3m 15s	the patch passed
+1 💚	shadedjars	4m 38s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 49s	the patch passed
		_ Other Tests _
+1 💚	unit	150m 45s	root in the patch passed.
		188m 52s

Subsystem	Report/Notes
Docker	Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#1461
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d33fc8e05fd1 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `5012241`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/testReport/
Max. process+thread count	5915 (vs. ulimit of 12500)
modules	C: . U: .
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1461/4/console
versions	git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

saintstack · 2020-04-10T05:21:10Z

Closing for now. Too destabilizing. Closing in favor of the approach over in HBASE-24150. Upping the forkcount to 0.5C threatens to destabilize builds, especially after HBASE-24150 (currently merged to branch-2 for survey)

saintstack · 2020-04-10T05:21:20Z

Closing for now. Too destabilizing. Closing in favor of the approach over in HBASE-24150. Upping the forkcount to 0.5C threatens to destabilize builds, especially after HBASE-24150 (currently merged to branch-2 for survey)

ndimiduk reviewed Apr 8, 2020
View reviewed changes

saintstack force-pushed the HBASE-23779 branch from 5cfd1d5 to cabd6e9Compare April 9, 2020 00:15

saintstack force-pushed the HBASE-23779 branch from cabd6e9 to 1474fe4Compare April 9, 2020 00:17

ndimiduk reviewed Apr 9, 2020
View reviewed changes

HBASE-23779 Up the default fork count to make builds complete faster;…
bbc04d0
… make count relative to CPU count Halve the build speed by doubling the -C count from 0.25C to 0.5C.

saintstack force-pushed the HBASE-23779 branch from 1474fe4 to bbc04d0Compare April 9, 2020 17:27

saintstack closed this Apr 10, 2020

Uh oh!

Conversation

saintstack commented Apr 8, 2020

Uh oh!

Apache-HBase commented Apr 8, 2020

Uh oh!

Apache-HBase commented Apr 8, 2020

Uh oh!

Apache-HBase commented Apr 8, 2020

Uh oh!

Apache-HBase commented Apr 8, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

saintstack commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

saintstack commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

Apache-HBase commented Apr 9, 2020

Uh oh!

saintstack commented Apr 10, 2020

Uh oh!

saintstack commented Apr 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants