Skip to content

Introduce a Snapshot Procedure - #4115

Merged
Apache9 merged 6 commits into
apache:masterfrom
frostruan:HBASE-26323
Mar 12, 2022
Merged

Introduce a Snapshot Procedure#4115
Apache9 merged 6 commits into
apache:masterfrom
frostruan:HBASE-26323

Conversation

@frostruan

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 34smaster passed
+1 💚compile3m 21smaster passed
+1 💚checkstyle1m 1smaster passed
+1 💚spotbugs4m 25smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 23sthe patch passed
+1 💚compile3m 23sthe patch passed
+1 💚cc3m 23sthe patch passed
+1 💚javac3m 23sthe patch passed
-0 ⚠️checkstyle0m 38shbase-server: The patch generated 6 new + 105 unchanged - 3 fixed = 111 total (was 108)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 23sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚hbaseprotoc1m 17sthe patch passed
+1 💚spotbugs4m 47sthe patch passed
_ Other Tests _
+1 💚asflicense0m 28sThe patch does not generate ASF License warnings.
42m 39s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4115
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux 394a2a3bc318 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 7316ba9
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count60 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 54sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for branch
+1 💚mvninstall2m 59smaster passed
+1 💚compile1m 27smaster passed
+1 💚shadedjars5m 1sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 50smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 46sthe patch passed
+1 💚compile1m 29sthe patch passed
+1 💚javac1m 29sthe patch passed
+1 💚shadedjars5m 0spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 49sthe patch passed
_ Other Tests _
+1 💚unit0m 34shbase-protocol-shaded in the patch passed.
+1 💚unit1m 48shbase-client in the patch passed.
+1 💚unit316m 32shbase-server in the patch passed.
345m 26s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux dfab7b34cd42 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 7316ba9
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/testReport/
Max. process+thread count1656 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 40sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for branch
+1 💚mvninstall2m 37smaster passed
+1 💚compile1m 13smaster passed
+1 💚shadedjars5m 2sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 24sthe patch passed
+1 💚compile1m 13sthe patch passed
+1 💚javac1m 13sthe patch passed
+1 💚shadedjars5m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42sthe patch passed
_ Other Tests _
+1 💚unit0m 25shbase-protocol-shaded in the patch passed.
+1 💚unit1m 37shbase-client in the patch passed.
+1 💚unit325m 14shbase-server in the patch passed.
348m 44s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6801a7633bae 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 7316ba9
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/testReport/
Max. process+thread count1609 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 5sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 22smaster passed
+1 💚compile3m 22smaster passed
+1 💚checkstyle1m 0smaster passed
+1 💚spotbugs4m 31smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall2m 23sthe patch passed
-1 ❌compile0m 39shbase-server in the patch failed.
-0 ⚠️cc0m 39shbase-server in the patch failed.
-0 ⚠️javac0m 39shbase-server in the patch failed.
-0 ⚠️checkstyle0m 36shbase-server: The patch generated 6 new + 105 unchanged - 3 fixed = 111 total (was 108)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 16sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚hbaseprotoc1m 18sthe patch passed
+1 💚spotbugs5m 0sthe patch passed
_ Other Tests _
+1 💚asflicense0m 27sThe patch does not generate ASF License warnings.
41m 15s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4115
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux a7432f536865 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8ddd1d0
Default JavaAdoptOpenJDK-1.8.0_282-b08
compilehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
cchttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javachttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count60 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 41sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 28sMaven dependency ordering for branch
+1 💚mvninstall3m 3smaster passed
+1 💚compile1m 29smaster passed
+1 💚shadedjars5m 3sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 47smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 44sthe patch passed
+1 💚compile1m 28sthe patch passed
+1 💚javac1m 28sthe patch passed
+1 💚shadedjars5m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45sthe patch passed
_ Other Tests _
+1 💚unit0m 32shbase-protocol-shaded in the patch passed.
+1 💚unit1m 48shbase-client in the patch passed.
+1 💚unit317m 40shbase-server in the patch passed.
343m 46s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a8221a61516e 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8ddd1d0
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/testReport/
Max. process+thread count1581 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 40sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for branch
+1 💚mvninstall3m 30smaster passed
+1 💚compile1m 16smaster passed
+1 💚shadedjars5m 10sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 46smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 29sthe patch passed
+1 💚compile1m 16sthe patch passed
+1 💚javac1m 16sthe patch passed
+1 💚shadedjars5m 8spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 43sthe patch passed
_ Other Tests _
+1 💚unit0m 25shbase-protocol-shaded in the patch passed.
+1 💚unit1m 41shbase-client in the patch passed.
+1 💚unit330m 23shbase-server in the patch passed.
355m 18s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 427345492f8e 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8ddd1d0
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/testReport/
Max. process+thread count1585 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 7sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 23smaster passed
+1 💚compile3m 20smaster passed
+1 💚checkstyle1m 0smaster passed
+1 💚spotbugs4m 25smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 24sthe patch passed
+1 💚compile3m 20sthe patch passed
+1 💚cc3m 20sthe patch passed
+1 💚javac3m 20sthe patch passed
-0 ⚠️checkstyle0m 38shbase-server: The patch generated 1 new + 105 unchanged - 3 fixed = 106 total (was 108)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 20sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚hbaseprotoc1m 19sthe patch passed
+1 💚spotbugs5m 3sthe patch passed
_ Other Tests _
+1 💚asflicense0m 27sThe patch does not generate ASF License warnings.
42m 56s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4115
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux bb3190079a18 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count60 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 40sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 33sMaven dependency ordering for branch
+1 💚mvninstall2m 55smaster passed
+1 💚compile1m 29smaster passed
+1 💚shadedjars5m 1sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 46sthe patch passed
+1 💚compile1m 30sthe patch passed
+1 💚javac1m 30sthe patch passed
+1 💚shadedjars4m 58spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 44sthe patch passed
_ Other Tests _
+1 💚unit0m 32shbase-protocol-shaded in the patch passed.
+1 💚unit1m 47shbase-client in the patch passed.
-1 ❌unit317m 46shbase-server in the patch failed.
343m 21s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9eb9de6f0562 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/testReport/
Max. process+thread count1609 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 38sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for branch
+1 💚mvninstall2m 19smaster passed
+1 💚compile1m 12smaster passed
+1 💚shadedjars4m 59sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 20sthe patch passed
+1 💚compile1m 18sthe patch passed
+1 💚javac1m 18sthe patch passed
+1 💚shadedjars4m 58spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41sthe patch passed
_ Other Tests _
+1 💚unit0m 23shbase-protocol-shaded in the patch passed.
+1 💚unit1m 36shbase-client in the patch passed.
+1 💚unit329m 21shbase-server in the patch passed.
352m 22s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux fa8198078bcb 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/testReport/
Max. process+thread count1548 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 6sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall3m 4smaster passed
+1 💚compile4m 25smaster passed
+1 💚checkstyle1m 12smaster passed
+1 💚spotbugs6m 16smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall3m 1sthe patch passed
+1 💚compile4m 7sthe patch passed
+1 💚cc4m 7sthe patch passed
+1 💚javac4m 7sthe patch passed
-0 ⚠️checkstyle0m 40shbase-server: The patch generated 1 new + 105 unchanged - 3 fixed = 106 total (was 108)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 55sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚hbaseprotoc1m 24sthe patch passed
+1 💚spotbugs5m 50sthe patch passed
_ Other Tests _
+1 💚asflicense0m 24sThe patch does not generate ASF License warnings.
51m 10s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4115
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux 606f16d522a0 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count60 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 45sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for branch
+1 💚mvninstall2m 55smaster passed
+1 💚compile1m 27smaster passed
+1 💚shadedjars5m 2sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 46smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 45sthe patch passed
+1 💚compile1m 31sthe patch passed
+1 💚javac1m 31sthe patch passed
+1 💚shadedjars4m 59spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45sthe patch passed
_ Other Tests _
+1 💚unit0m 32shbase-protocol-shaded in the patch passed.
+1 💚unit1m 46shbase-client in the patch passed.
+1 💚unit315m 45shbase-server in the patch passed.
340m 48s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux e8da03a02f8b 5.4.0-1054-aws #57~18.04.1-Ubuntu SMP Thu Jul 15 03:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/testReport/
Max. process+thread count1612 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 22sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 59smaster passed
+1 💚compile1m 35smaster passed
+1 💚shadedjars5m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 53smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile1m 44sthe patch passed
+1 💚javac1m 44sthe patch passed
+1 💚shadedjars6m 7spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 53sthe patch passed
_ Other Tests _
+1 💚unit0m 33shbase-protocol-shaded in the patch passed.
+1 💚unit1m 48shbase-client in the patch passed.
+1 💚unit353m 12shbase-server in the patch passed.
381m 45s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 290212966ceb 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 8dec499
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/testReport/
Max. process+thread count1629 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/4/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@frostruan

Copy link
Copy Markdown
ContributorAuthor

hi duo, I've made some progress recently.

  1. using WorkerAssigner to distribute the task of snapshot verification.
    a. move WorkerAssigner out of SplitWALManager for reuse.
    b. new a WorkerAssigner instance in the SnapshotManger and distribute the task of snapshot verification with it.
  2. introduce a RSSnapshotVerifier
    a. since now we verify only one region via a verification procedure, it could be slow if we load the SnapshotManaifest from hdfs every time. so I add a cache for it.
    b. the RSSnapshotVerifier is used to verify regioninfo and store files.
  3. if the snapshot is corrupted, we touch a hdfs file to notify the upper layer procedures.
  4. using nonceGroup and nonce field in the snapshot request to identify whether the client can handle snapshot proc id.
  5. introduce a switch in SnapshotManager hbase.snapshot.procedure.enabled to enable snapshot procedure so that we can fallback to the zk-coordinated snapshot.

If you have time, would you mind taking a look on this at your convenience and see if there is any problems ? Thanks. @Apache9

@Apache9Apache9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

Only need to confirm that the fencing method is correct.

Thanks for the hard work @frostruan !

}

@RestrictedApi(explanation = "Should only be called in tests", link = "",
allowedOnPath = ".*(/src/test/.*|TestSnapshotProcedure).java")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'src/test/.* ' contains TestSnapshotProcedure?

.filter(p -> !p.isFinished())
.filter(p -> p.getServerName() != null)
.forEach(p -> {
verifyWorkerAssigner.addUsedWorker(p.getServerName());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the region server is already dead?

@frostruanfrostruanFeb 25, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's harmless that we have dead servers in the WorkAssigner based on this consideration:

  1. If the remote server is dead, there will be a FailedRemoteDispatchException thrown when we dispatch remote procedures. The procedure-v2 framework is designed to be able to handle this. The upper layer procedure can notice that and reschedule a new procedure, or the procedure can retry itself.
  2. When acquire a new worker from WorkAssigner, we will always choose from the online server list. The dead server will not be able to become a candidate again.
  3. If there are too many dead servers in the WorkerAssigner, we can restart the master to clear it. This is a very lightweight operation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this is the behavior of the SplitWALManager, so I guess it is OK.


private void prepareSnapshot(MasterProcedureEnv env)
throws ProcedureSuspendedException, IOException {
if (isAnySplitOrMergeProcedureRunning(env)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So when arriving here, we can not schedule new split or merge procedures for this table any more as we have already record this procedure in the SnapshotManager?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. That's correct. In fact, in the current implementation, if a zk-coordinated snapshot is in progress, the split/merge procedure on the same table will be marked as FAILED.

@frostruan

Copy link
Copy Markdown
ContributorAuthor

Really thanks for your great suggestions and help, Duo. I will address it ASAP. @Apache9

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 4sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 22smaster passed
+1 💚compile3m 21smaster passed
+1 💚checkstyle1m 1smaster passed
+1 💚spotbugs4m 26smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 24sthe patch passed
+1 💚compile3m 24sthe patch passed
+1 💚cc3m 24sthe patch passed
+1 💚javac3m 24sthe patch passed
-0 ⚠️checkstyle0m 37shbase-server: The patch generated 1 new + 105 unchanged - 3 fixed = 106 total (was 108)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 19sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚hbaseprotoc1m 19sthe patch passed
+1 💚spotbugs4m 46sthe patch passed
_ Other Tests _
+1 💚asflicense0m 25sThe patch does not generate ASF License warnings.
42m 24s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4115
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux ddf41b59447b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5dc663e
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count60 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 1sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 36sMaven dependency ordering for branch
+1 💚mvninstall3m 15smaster passed
+1 💚compile1m 28smaster passed
+1 💚shadedjars4m 58sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 47smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 43sthe patch passed
+1 💚compile1m 29sthe patch passed
+1 💚javac1m 29sthe patch passed
+1 💚shadedjars4m 57spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45sthe patch passed
_ Other Tests _
+1 💚unit0m 31shbase-protocol-shaded in the patch passed.
+1 💚unit1m 47shbase-client in the patch passed.
+1 💚unit318m 58shbase-server in the patch passed.
345m 34s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 4a0ab58a2a7c 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5dc663e
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/testReport/
Max. process+thread count1719 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 48sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 23sMaven dependency ordering for branch
+1 💚mvninstall2m 20smaster passed
+1 💚compile1m 14smaster passed
+1 💚shadedjars4m 59sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall2m 16sthe patch passed
+1 💚compile1m 11sthe patch passed
+1 💚javac1m 11sthe patch passed
+1 💚shadedjars4m 55spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 41sthe patch passed
_ Other Tests _
+1 💚unit0m 24shbase-protocol-shaded in the patch passed.
+1 💚unit1m 34shbase-client in the patch passed.
-1 ❌unit333m 26shbase-server in the patch failed.
356m 49s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4115
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5ef1781f767c 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5dc663e
Default JavaAdoptOpenJDK-1.8.0_282-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/testReport/
Max. process+thread count1622 (vs. ulimit of 30000)
modulesC: hbase-protocol-shaded hbase-client hbase-server U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4115/5/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9Apache9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. Just need to fix some minor nits.

I think we can merge it to master and branch-2 first, and then polish other things.

Thanks @frostruan !

.filter(p -> !p.isFinished())
.filter(p -> p.getServerName() != null)
.forEach(p -> {
verifyWorkerAssigner.addUsedWorker(p.getServerName());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this is the behavior of the SplitWALManager, so I guess it is OK.

@frostruan

Copy link
Copy Markdown
ContributorAuthor

Really appreciate your help, Duo @Apache9

@Apache9
Apache9 merged commit fd301ad into apache:masterMar 12, 2022
Apache9 pushed a commit that referenced this pull request Mar 12, 2022
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@frostruan@Apache-HBase@Apache9