Skip to content

HBASE-26120 New replication gets stuck or data loss when multiwal gro… - #3528

Merged
apurtell merged 1 commit into
apache:branch-2from
Apache9:HBASE-26120
Jul 27, 2021
Merged

HBASE-26120 New replication gets stuck or data loss when multiwal gro…#3528
apurtell merged 1 commit into
apache:branch-2from
Apache9:HBASE-26120

Conversation

@Apache9

@Apache9Apache9 commented Jul 27, 2021

Copy link
Copy Markdown
Contributor

…ups more than 10

@Apache9Apache9 self-assigned this Jul 27, 2021
saintstack
saintstack approved these changes Jul 27, 2021
View reviewed changes

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed. Not familiar enough to say if this fixes the issue but LGTM.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec6m 29sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 1sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall3m 59sbranch-2 passed
+1 💚compile3m 23sbranch-2 passed
+1 💚checkstyle1m 15sbranch-2 passed
+1 💚spotbugs2m 17sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 38sthe patch passed
+1 💚compile3m 24sthe patch passed
+1 💚javac3m 24sthe patch passed
+1 💚checkstyle1m 14sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck13m 23sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 33sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
50m 14s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3528
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 234b2ed45b76 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count85 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec4m 15sDocker mode activated.
-0 ⚠️yetus0m 7sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 22sbranch-2 passed
+1 💚compile1m 9sbranch-2 passed
+1 💚shadedjars6m 53sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 2sthe patch passed
+1 💚compile1m 6sthe patch passed
+1 💚javac1m 6sthe patch passed
+1 💚shadedjars6m 43spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39sthe patch passed
_ Other Tests _
+1 💚unit134m 8shbase-server in the patch passed.
166m 27s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3528
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 2ca3049f8288 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/testReport/
Max. process+thread count4092 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec6m 21sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 2sbranch-2 passed
+1 💚compile0m 59sbranch-2 passed
+1 💚shadedjars6m 37sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 41sthe patch passed
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
+1 💚shadedjars6m 33spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
+1 💚unit229m 15shbase-server in the patch passed.
261m 43s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#3528
Optional Testsjavac javadoc unit shadedjars compile
unameLinux e260bb957bda 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/testReport/
Max. process+thread count2263 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec10m 59sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall4m 6sbranch-2 passed
+1 💚compile3m 43sbranch-2 passed
+1 💚checkstyle1m 14sbranch-2 passed
+1 💚spotbugs2m 14sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 30sthe patch passed
+1 💚compile3m 22sthe patch passed
+1 💚javac3m 22sthe patch passed
+1 💚checkstyle1m 6sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck13m 7sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 24sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
54m 18s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3528
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux fc0a5066f7bc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count96 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 7sDocker mode activated.
-0 ⚠️yetus0m 8sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall3m 36sbranch-2 passed
+1 💚compile0m 57sbranch-2 passed
+1 💚shadedjars6m 0sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 40sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 23sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚shadedjars6m 1spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37sthe patch passed
_ Other Tests _
+1 💚unit150m 27shbase-server in the patch passed.
176m 13s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#3528
Optional Testsjavac javadoc unit shadedjars compile
unameLinux adf3c3fa030c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/testReport/
Max. process+thread count3553 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jul 27, 2021

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec3m 58sDocker mode activated.
-0 ⚠️yetus0m 7sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 26sbranch-2 passed
+1 💚compile1m 8sbranch-2 passed
+1 💚shadedjars6m 56sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 44sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 8sthe patch passed
+1 💚compile1m 7sthe patch passed
+1 💚javac1m 7sthe patch passed
+1 💚shadedjars7m 2spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 42sthe patch passed
_ Other Tests _
-1 ❌unit153m 37shbase-server in the patch failed.
185m 58s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3528
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9371ac470444 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / 20a4aae
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/testReport/
Max. process+thread count4145 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3528/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtell
apurtell merged commit dd22fec into apache:branch-2Jul 27, 2021
asfgit pushed a commit that referenced this pull request Jul 27, 2021
@Apache9@apurtell
HBASE-26120New replication gets stuck or data loss when multiwal gro…
eb47281
…ups more than 10 (#3528)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
asfgit pushed a commit that referenced this pull request Jul 27, 2021
@Apache9@apurtell
HBASE-26120New replication gets stuck or data loss when multiwal gro…
b27ee46
…ups more than 10 (#3528)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
asfgit pushed a commit that referenced this pull request Jul 27, 2021
@Apache9@apurtell
HBASE-26120New replication gets stuck or data loss when multiwal gro…
8f82cbf
…ups more than 10 (#3528)
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Michael Stack <stack@apache.org>
Conflicts:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
saintstack
saintstack approved these changes Jul 27, 2021
View reviewed changes

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appreciate the reasoning up in the JIRA. Helped w/ the review here.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@saintstacksaintstacksaintstack approved these changes
@apurtellapurtellapurtell approved these changes

Assignees

@Apache9Apache9

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Apache9@Apache-HBase@saintstack@apurtell