Skip to content

HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev… - #3011

Closed
clarax wants to merge 3 commits into
apache:masterfrom
clarax:HBASE-25625
Closed

HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev…#3011
clarax wants to merge 3 commits into
apache:masterfrom
clarax:HBASE-25625

Conversation

@clarax

Copy link
Copy Markdown
Contributor

…aluate resource distribution

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 12smaster passed
+1 💚compile3m 20smaster passed
+1 💚checkstyle1m 14smaster passed
+1 💚spotbugs2m 9smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 50sthe patch passed
-1 ❌compile3m 14shbase-server in the patch failed.
-0 ⚠️javac3m 14shbase-server in the patch failed.
-0 ⚠️checkstyle1m 12shbase-server: The patch generated 2 new + 100 unchanged - 1 fixed = 102 total (was 101)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 10sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 18sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
49m 50s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3011
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux a74798b76a14 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5d9a6ed
Default JavaAdoptOpenJDK-1.8.0_282-b08
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 10smaster passed
+1 💚compile1m 0smaster passed
+1 💚shadedjars7m 10sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 52sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚shadedjars7m 14spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
-1 ❌unit211m 57shbase-server in the patch failed.
240m 38s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 3f7895912066 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5d9a6ed
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/testReport/
Max. process+thread count3481 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 19sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 50smaster passed
+1 💚compile1m 13smaster passed
+1 💚shadedjars7m 28sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 42sthe patch passed
+1 💚compile1m 13sthe patch passed
+1 💚javac1m 13sthe patch passed
+1 💚shadedjars7m 25spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41sthe patch passed
_ Other Tests _
-1 ❌unit212m 29shbase-server in the patch failed.
243m 55s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 897dd517b4a7 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5d9a6ed
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/testReport/
Max. process+thread count3121 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@virajjasani

virajjasani commented Mar 4, 2021

Copy link
Copy Markdown
Contributor

@clarax could you please check compilation issue here

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 12smaster passed
+1 💚compile3m 22smaster passed
+1 💚checkstyle1m 13smaster passed
+1 💚spotbugs2m 10smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 49sthe patch passed
+1 💚compile3m 19sthe patch passed
+1 💚javac3m 19sthe patch passed
-0 ⚠️checkstyle1m 12shbase-server: The patch generated 2 new + 100 unchanged - 1 fixed = 102 total (was 101)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 5sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 17sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
55m 58s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3011
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux e77e0872c24b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 42sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 55smaster passed
+1 💚compile1m 17smaster passed
+1 💚shadedjars8m 27sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 38sthe patch passed
+1 💚compile1m 19sthe patch passed
+1 💚javac1m 19sthe patch passed
+1 💚shadedjars7m 36spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43sthe patch passed
_ Other Tests _
-1 ❌unit209m 42shbase-server in the patch failed.
248m 58s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 3dabc5f1454a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/testReport/
Max. process+thread count2685 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 19sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 53smaster passed
+1 💚compile0m 58smaster passed
+1 💚shadedjars7m 13sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 52sthe patch passed
+1 💚compile0m 57sthe patch passed
+1 💚javac0m 57sthe patch passed
+1 💚shadedjars7m 13spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
-1 ❌unit216m 43shbase-server in the patch failed.
251m 15s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux be50297a1d42 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/testReport/
Max. process+thread count3472 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 5sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 48smaster passed
+1 💚compile3m 22smaster passed
+1 💚checkstyle1m 12smaster passed
+1 💚spotbugs2m 8smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 51sthe patch passed
+1 💚compile3m 21sthe patch passed
+1 💚javac3m 21sthe patch passed
-0 ⚠️checkstyle1m 12shbase-server: The patch generated 2 new + 134 unchanged - 1 fixed = 136 total (was 135)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck18m 56sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 17sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
49m 20s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3011
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 2a1ab759d5f3 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 41smaster passed
+1 💚compile1m 0smaster passed
+1 💚shadedjars7m 11sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 40smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 50sthe patch passed
+1 💚compile1m 0sthe patch passed
+1 💚javac1m 0sthe patch passed
+1 💚shadedjars7m 13spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
-1 ❌unit212m 5shbase-server in the patch failed.
241m 19s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux c34eb45f8c70 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/testReport/
Max. process+thread count3400 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 39smaster passed
+1 💚compile1m 14smaster passed
+1 💚shadedjars7m 26sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 37sthe patch passed
+1 💚compile1m 13sthe patch passed
+1 💚javac1m 13sthe patch passed
+1 💚shadedjars7m 50spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 47sthe patch passed
_ Other Tests _
-1 ❌unit215m 21shbase-server in the patch failed.
246m 53s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux f22b7fab953b 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/testReport/
Max. process+thread count2840 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

test updates
@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 51smaster passed
+1 💚compile0m 58smaster passed
+1 💚shadedjars7m 14sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35smaster passed
_ Patch Compile Tests _
-1 ❌mvninstall2m 11sroot in the patch failed.
-1 ❌compile0m 55shbase-server in the patch failed.
-0 ⚠️javac0m 55shbase-server in the patch failed.
-1 ❌shadedjars5m 38spatch has 14 errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
-1 ❌unit0m 54shbase-server in the patch failed.
25m 5s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux aa563d95dd2d 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
mvninstallhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt
shadedjarshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/testReport/
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 28sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 18smaster passed
+1 💚compile1m 5smaster passed
+1 💚shadedjars6m 44sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42smaster passed
_ Patch Compile Tests _
-1 ❌mvninstall2m 29sroot in the patch failed.
-1 ❌compile1m 5shbase-server in the patch failed.
-0 ⚠️javac1m 5shbase-server in the patch failed.
-1 ❌shadedjars5m 21spatch has 14 errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39sthe patch passed
_ Other Tests _
-1 ❌unit1m 6shbase-server in the patch failed.
25m 12s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux d7d5ef3a6c42 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-11.0.10+9
mvninstallhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt
shadedjarshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/testReport/
Max. process+thread count103 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 58sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 53smaster passed
+1 💚compile3m 13smaster passed
+1 💚checkstyle1m 9smaster passed
+1 💚spotbugs2m 2smaster passed
_ Patch Compile Tests _
-1 ❌mvninstall2m 3sroot in the patch failed.
-1 ❌compile1m 52shbase-server in the patch failed.
-0 ⚠️javac1m 52shbase-server in the patch failed.
-0 ⚠️checkstyle1m 5shbase-server: The patch generated 3 new + 134 unchanged - 1 fixed = 137 total (was 135)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
-1 ❌hadoopcheck2m 9sThe patch causes 14 errors with Hadoop v3.1.2.
-1 ❌hadoopcheck4m 21sThe patch causes 14 errors with Hadoop v3.2.1.
-1 ❌hadoopcheck6m 43sThe patch causes 14 errors with Hadoop v3.3.0.
-1 ❌spotbugs0m 32shbase-server in the patch failed.
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
28m 23s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3011
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux b84641460c04 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
mvninstallhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt
compilehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javachttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
hadoopcheckhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.1.2.txt
hadoopcheckhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.2.1.txt
hadoopcheckhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-javac-3.3.0.txt
spotbugshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/4/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@d-c-manningd-c-manning left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm excited for working towards a balancer that works better for large clusters! Thanks for proposing changes in that direction.

}
}
// recalculate stdev
reComputeRegionStDevPerTable();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like here we should only be recomputing the stdev for the affected tableIndex. I don't want to over-optimize if it makes the code less readable... but in clusters with large numbers of tables, we would be doing unnecessary work.

I'm thinking in the context of HBASE-22300 where large clusters no longer run through enough iterations of the balancer. In that case, the issue is number of regions, which will always be larger than number of tables... and likely much larger. So it may not be a major issue here, but I'm bringing it up just in case.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for bringing up this. reComputeRegionStDevPerTable() can be optimized to only compute the impacted table index to reduce computation cost.

testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, true, true);

// we need to capture the outlier and generate a move
testWithCluster(numNodes, numRegions, numRegionsPerServer, replication, numTables, false, false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change of code suggests that the balancer will no longer fully balance the cluster in one run... why is that? Are we making the balancer less effective, even in this test with large cluster with one outlier node? I thought this is the type of case that the fix is supposed to improve?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the side impact. the computation gets more complicated. I will work on it.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out to be related to https://issues.apache.org/jira/browse/HBASE-25767. Will merge the fix.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 9sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 56smaster passed
+1 💚compile3m 21smaster passed
+1 💚checkstyle1m 13smaster passed
+1 💚spotbugs2m 10smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 49sthe patch passed
+1 💚compile3m 16sthe patch passed
+1 💚javac3m 16sthe patch passed
-0 ⚠️checkstyle1m 12shbase-server: The patch generated 4 new + 134 unchanged - 1 fixed = 138 total (was 135)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 2sPatch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚spotbugs2m 23sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
49m 27s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3011
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 43ab0b3af714 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 13sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 0smaster passed
+1 💚compile1m 1smaster passed
+1 💚shadedjars7m 20sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 52sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚shadedjars7m 15spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
-1 ❌unit211m 4shbase-server in the patch failed.
239m 36s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5ccd0d41708f 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/testReport/
Max. process+thread count3657 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 44sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall5m 42smaster passed
+1 💚compile1m 23smaster passed
+1 💚shadedjars8m 46sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 49smaster passed
_ Patch Compile Tests _
+1 💚mvninstall5m 51sthe patch passed
+1 💚compile1m 40sthe patch passed
+1 💚javac1m 40sthe patch passed
+1 💚shadedjars8m 58spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 52sthe patch passed
_ Other Tests _
-1 ❌unit226m 58shbase-server in the patch failed.
264m 33s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3011
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 05654c226f00 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 830d289
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/testReport/
Max. process+thread count3147 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3011/5/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@saintstack

Copy link
Copy Markdown
Contributor

Is this PR live still @clarax ? Or superceded by #3067 ? Thanks.

@clarax

Copy link
Copy Markdown
ContributorAuthor

Is this PR live still @clarax ? Or superceded by #3067 ? Thanks.

The new heuristic needs to be evaluated by simulation at different scales. so this pr is currently pending the validation. 3067 is a scaled back fix for table skew only as a sub-task https://issues.apache.org/jira/browse/HBASE-25739?filter=-2

@saintstack

Copy link
Copy Markdown
Contributor

Should this or the other PR have their own JIRA then @clarax Thanks.

@clarax

Copy link
Copy Markdown
ContributorAuthor

Close the pr for now since the other pr #3067 landed and relieved most of the problem. Will revisit when the problem arises again.

@claraxclarax closed this Aug 20, 2021
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@clarax@Apache-HBase@virajjasani@saintstack@sunhelly@d-c-manning