Skip to content

HBASE-25634 The client scan frequently exceeds the quota, which cause… - #3030

Open
SiCheng-Zheng wants to merge 1 commit into
apache:branch-2from
SiCheng-Zheng:HBASE-25634-new
Open

HBASE-25634 The client scan frequently exceeds the quota, which cause…#3030
SiCheng-Zheng wants to merge 1 commit into
apache:branch-2from
SiCheng-Zheng:HBASE-25634-new

Conversation

@SiCheng-Zheng

Copy link
Copy Markdown
Contributor

…s the meta table scan to be too high

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 49sDocker mode activated.
-0 ⚠️yetus0m 7sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 4sbranch-2 passed
+1 💚compile0m 27sbranch-2 passed
+1 💚shadedjars6m 33sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 42sthe patch passed
+1 💚compile0m 26sthe patch passed
+1 💚javac0m 26sthe patch passed
+1 💚shadedjars6m 37spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 23sthe patch passed
_ Other Tests _
+1 💚unit2m 33shbase-client in the patch passed.
28m 5s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 9b44a734cbd2 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / e81ef34
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/testReport/
Max. process+thread count223 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 32sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall3m 45sbranch-2 passed
+1 💚compile1m 6sbranch-2 passed
+1 💚checkstyle0m 32sbranch-2 passed
+1 💚spotbugs1m 11sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 19sthe patch passed
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
-0 ⚠️checkstyle0m 32shbase-client: The patch generated 1 new + 5 unchanged - 0 fixed = 6 total (was 5)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck11m 43sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs1m 14sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
32m 25s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3030
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 66d477daf1fd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / e81ef34
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count96 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 7sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall5m 23sbranch-2 passed
+1 💚compile0m 38sbranch-2 passed
+1 💚shadedjars8m 13sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 36sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall5m 10sthe patch passed
+1 💚compile0m 38sthe patch passed
+1 💚javac0m 38sthe patch passed
+1 💚shadedjars8m 10spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 31sthe patch passed
_ Other Tests _
+1 💚unit3m 2shbase-client in the patch passed.
35m 35s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 148b8116fdac 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / e81ef34
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/testReport/
Max. process+thread count206 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

}
if (!(t instanceof RpcThrottlingException)) {
callable.prepare(tries != 0);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the idea here is that if we got an exception and we are retrying, do NOT reload cache if the exception was a because we were throttled?

This is a good idea. I wonder if there are other exceptions where we retry but do not need to reload the cache?

@saintstacksaintstack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would a test be hard to add?

@InterfaceAudience.Private
public static class ThrowableWithExtraContext {
private final Throwable throwable;
public final Throwable throwable;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't think we need an accessor here?

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 23sDocker mode activated.
-0 ⚠️yetus0m 8sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 8sbranch-2 passed
+1 💚compile0m 27sbranch-2 passed
+1 💚shadedjars6m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 45sthe patch passed
+1 💚compile0m 26sthe patch passed
+1 💚javac0m 26sthe patch passed
+1 💚shadedjars6m 35spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 24sthe patch passed
_ Other Tests _
+1 💚unit2m 57shbase-client in the patch passed.
28m 32s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7b5e2949f750 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / c52c091
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/testReport/
Max. process+thread count227 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
-0 ⚠️yetus0m 7sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 46sbranch-2 passed
+1 💚compile0m 34sbranch-2 passed
+1 💚shadedjars7m 34sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 32sthe patch passed
+1 💚compile0m 32sthe patch passed
+1 💚javac0m 32sthe patch passed
+1 💚shadedjars7m 31spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28sthe patch passed
_ Other Tests _
+1 💚unit2m 57shbase-client in the patch passed.
31m 56s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 32482b5d586c 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / c52c091
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/testReport/
Max. process+thread count216 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 36sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall4m 13sbranch-2 passed
+1 💚compile1m 10sbranch-2 passed
+1 💚checkstyle0m 34sbranch-2 passed
+1 💚spotbugs1m 17sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 1sthe patch passed
+1 💚compile1m 14sthe patch passed
+1 💚javac1m 14sthe patch passed
+1 💚checkstyle0m 37sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck14m 2sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs1m 25sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
37m 39s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3030
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux d4305a4dd284 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / c52c091
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count96 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
-0 ⚠️yetus0m 7sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 6sbranch-2 passed
+1 💚compile0m 26sbranch-2 passed
+1 💚shadedjars7m 1sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 44sthe patch passed
+1 💚compile0m 26sthe patch passed
+1 💚javac0m 26sthe patch passed
+1 💚shadedjars7m 6spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 23sthe patch passed
_ Other Tests _
+1 💚unit2m 40shbase-client in the patch passed.
28m 43s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a2eedb324b07 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / da9bcb6
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/testReport/
Max. process+thread count233 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 10sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2 Compile Tests _
+1 💚mvninstall4m 0sbranch-2 passed
+1 💚compile1m 5sbranch-2 passed
+1 💚checkstyle0m 34sbranch-2 passed
+1 💚spotbugs1m 13sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall3m 41sthe patch passed
+1 💚compile1m 3sthe patch passed
+1 💚javac1m 3sthe patch passed
+1 💚checkstyle0m 34sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck13m 32sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs1m 28sthe patch passed
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
36m 53s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3030
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux a03ebfa62d01 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / da9bcb6
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 33sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚mvninstall4m 17sbranch-2 passed
+1 💚compile0m 32sbranch-2 passed
+1 💚shadedjars7m 9sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 32sbranch-2 passed
_ Patch Compile Tests _
+1 💚mvninstall4m 11sthe patch passed
+1 💚compile0m 32sthe patch passed
+1 💚javac0m 32sthe patch passed
+1 💚shadedjars7m 19spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30sthe patch passed
_ Other Tests _
+1 💚unit2m 48shbase-client in the patch passed.
29m 30s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#3030
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b6cfb90cf8d1 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2 / da9bcb6
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/testReport/
Max. process+thread count302 (vs. ulimit of 12500)
modulesC: hbase-client U: hbase-client
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3030/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@SiCheng-Zheng@Apache-HBase@saintstack@yuanheng@liuzhangqing@gottagogottagoGxj