Skip to content

HBASE-24379 CatalogJanitor misreports region holes when there are act… - #1741

Merged
huaxiangsun merged 1 commit into
apache:masterfrom
huaxiangsun:master.HBASE-24379
May 22, 2020
Merged

HBASE-24379 CatalogJanitor misreports region holes when there are act…#1741
huaxiangsun merged 1 commit into
apache:masterfrom
huaxiangsun:master.HBASE-24379

Conversation

@huaxiangsun

Copy link
Copy Markdown
Contributor

…ually over laps.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 41sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 17smaster passed
+1 💚checkstyle1m 17smaster passed
+1 💚spotbugs2m 16smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 50sthe patch passed
+1 💚checkstyle1m 13sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 32sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 42sthe patch passed
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
38m 44s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1741
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux 2ae1d519d620 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5fb9e51
Max. process+thread count84 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 40sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 18smaster passed
+1 💚compile1m 1smaster passed
+1 💚shadedjars6m 16sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 56sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
+1 💚shadedjars6m 5spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sthe patch passed
_ Other Tests _
+1 💚unit199m 29shbase-server in the patch passed.
226m 44s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#1741
Optional Testsjavac javadoc unit shadedjars compile
unameLinux feabdc161a4a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5fb9e51
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/testReport/
Max. process+thread count3165 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 12sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 45smaster passed
+1 💚compile1m 8smaster passed
+1 💚shadedjars6m 19sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 43shbase-server in master failed.
_ Patch Compile Tests _
+1 💚mvninstall4m 38sthe patch passed
+1 💚compile1m 8sthe patch passed
+1 💚javac1m 8sthe patch passed
+1 💚shadedjars6m 25spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 40shbase-server in the patch failed.
_ Other Tests _
-1 ❌unit218m 11shbase-server in the patch failed.
247m 4s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1741
Optional Testsjavac javadoc unit shadedjars compile
unameLinux c9075d69d90e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5fb9e51
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/testReport/
Max. process+thread count3281 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@huaxiangsun

Copy link
Copy Markdown
ContributorAuthor

I run the failed test locally and it passed. The test case timed out, I am kicking off another run.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 37sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 16smaster passed
+1 💚checkstyle1m 14smaster passed
+1 💚spotbugs2m 19smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 51sthe patch passed
+1 💚checkstyle1m 13sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 42sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 53sthe patch passed
_ Other Tests _
+1 💚asflicense0m 17sThe patch does not generate ASF License warnings.
39m 9s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1741
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux b912cb5d8a5e 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5fb9e51
Max. process+thread count84 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 31sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 16smaster passed
+1 💚compile1m 6smaster passed
+1 💚shadedjars5m 42sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 41shbase-server in master failed.
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile1m 2sthe patch passed
+1 💚javac1m 2sthe patch passed
+1 💚shadedjars5m 42spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 40shbase-server in the patch failed.
_ Other Tests _
+1 💚unit126m 35shbase-server in the patch passed.
152m 26s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1741
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 3de75476ab1f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 5fb9e51
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/testReport/
Max. process+thread count4256 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/2/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we try more combos here Huaxiang? I think this will get one case for sure but what happens if:

[a, g), [bb, cc), [dd, ee)

... will we report hole between a-bb or cc-dd or ee-g ?

Or will it just show as overlap?

Otherwise +1.

@saintstack

Copy link
Copy Markdown
Contributor

Or rather, while we are in here, maybe add some more test cases since they cheap to do.

@huaxiangsun

Copy link
Copy Markdown
ContributorAuthor

Should we try more combos here Huaxiang? I think this will get one case for sure but what happens if:

[a, g), [bb, cc), [dd, ee)

... will we report hole between a-bb or cc-dd or ee-g ?

Or will it just show as overlap?

Thanks Stack.

Yeah, they will be reported as overlaps. In this case, g is the region with highest key and it overlaps with the other two regions. The algorithm reporting holes only has one corner case which is fixed in this patch. I will add more test cases.

@huaxiangsun

Copy link
Copy Markdown
ContributorAuthor

Add more test case, updated the patch.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 5sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 44smaster passed
+1 💚compile1m 8smaster passed
+1 💚shadedjars6m 18sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 42shbase-server in master failed.
_ Patch Compile Tests _
+1 💚mvninstall4m 29sthe patch passed
+1 💚compile1m 7sthe patch passed
+1 💚javac1m 7sthe patch passed
+1 💚shadedjars6m 25spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 42shbase-server in the patch failed.
_ Other Tests _
-1 ❌unit9m 28shbase-server in the patch failed.
37m 11s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#1741
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a9d03d7d25f3 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e1d7dc8
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/testReport/
Max. process+thread count609 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 49sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 51smaster passed
+1 💚checkstyle1m 30smaster passed
+1 💚spotbugs2m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 21sthe patch passed
+1 💚checkstyle1m 25sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck13m 57sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 45sthe patch passed
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
41m 31s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#1741
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux baa27d8dcc12 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e1d7dc8
Max. process+thread count84 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 31sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 48smaster passed
+1 💚compile0m 54smaster passed
+1 💚shadedjars5m 34sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 39smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 36sthe patch passed
+1 💚compile0m 56sthe patch passed
+1 💚javac0m 56sthe patch passed
+1 💚shadedjars5m 38spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38sthe patch passed
_ Other Tests _
+1 💚unit136m 5shbase-server in the patch passed.
160m 8s
SubsystemReport/Notes
DockerClient=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#1741
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7ca8d45ac0d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / e1d7dc8
Default Java1.8.0_232
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/testReport/
Max. process+thread count4525 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1741/3/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@huaxiangsun

Copy link
Copy Markdown
ContributorAuthor

The test failure is HBASE-23976 (seems that it still happens).

@huaxiangsun
huaxiangsun merged commit 2b894dc into apache:masterMay 22, 2020
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 25, 2020
…ually over laps. (apache#1741)
Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 25, 2020
…ually over laps. (apache#1741)
Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 26, 2020
…ually over laps. (#1741) (#1777)
Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 26, 2020
…ually over laps. (#1741) (#1776)
Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit to huaxiangsun/hbase that referenced this pull request May 26, 2020
…ually over laps. (apache#1741)
Signed-off-by: stack <stack@apache.org>
huaxiangsun added a commit that referenced this pull request May 27, 2020
…ually over laps. (#1741) (#1785)
Signed-off-by: stack <stack@apache.org>
HomminLee pushed a commit to HomminLee/hbase that referenced this pull request Jun 12, 2020
…ually over laps. (apache#1741) (apache#1785)
Signed-off-by: stack <stack@apache.org>
infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
…ually over laps. (apache#1741) (apache#1776)
Signed-off-by: stack <stack@apache.org>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…ually over laps. (apache#1741) (apache#1777)
Signed-off-by: stack <stack@apache.org>
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
…ually over laps. (apache#1741) (apache#1785)
Author: huaxiangsun
Reason: Bug
Ref: CDPD-15964
Signed-off-by: stack <stack@apache.org>
Change-Id: Ic77366213b962dc28c428ac57dcdc19d5c2287f1
(cherry picked from commit b850603)
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request May 24, 2021
…ually over laps. (apache#1741) (apache#1785)
Signed-off-by: stack <stack@apache.org>
(cherry picked from commit 482956e)
Change-Id: Ic77366213b962dc28c428ac57dcdc19d5c2287f1
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@huaxiangsun@Apache-HBase@saintstack