Skip to content

HBASE-27676 Scan handlers in the RPC executor should match at least o… - #5074

Merged
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-27676
Mar 22, 2023
Merged

HBASE-27676 Scan handlers in the RPC executor should match at least o…#5074
sunhelly merged 1 commit into
apache:masterfrom
sunhelly:HBASE-27676

Conversation

@sunhelly

@sunhellysunhelly commented Mar 2, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

…ne scan queues

This issue is try to avoid NO scan queues for some scan handlers.
For example, if we set hbase.regionserver.handler.count=150, hbase.ipc.server.callqueue.scan.ratio=0.1, hbase.ipc.server.callqueue.read.ratio=0.5, hbase.ipc.server.callqueue.handler.factor=0.1, then there will be 1500.50.1=7 scan handlers, but there are 1500.10.5*0.1=0 scan RPC queues.
When there are no scan rpc queues, all the scan and get requests will be dispatched to the read rpc queues, while we we thought they had been dealt with separately, since the scan handler count is not 0. When there are not enough handlers for large scan requests under this circumstance, the small get requests will be blocked in the rpc queues.

@Apache-HBase

Apache-HBase commented Mar 2, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 8sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 42smaster passed
+1 💚compile2m 22smaster passed
+1 💚checkstyle0m 36smaster passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 16sthe patch passed
+1 💚compile2m 22sthe patch passed
+1 💚javac2m 22sthe patch passed
+1 💚checkstyle0m 36sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 12sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs1m 36sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
39m 15s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5074
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux d1b6c0262061 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count85 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 2, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 32sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 48smaster passed
+1 💚compile0m 55smaster passed
+1 💚shadedjars5m 42sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 15sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚shadedjars6m 30spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
-1 ❌unit13m 52shbase-server in the patch failed.
40m 37s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5074
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5828b5f34c0f 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/testReport/
Max. process+thread count593 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 2, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 57sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 3smaster passed
+1 💚compile0m 41smaster passed
+1 💚shadedjars4m 15sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile0m 40sthe patch passed
+1 💚javac0m 40sthe patch passed
+1 💚shadedjars4m 14spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 22sthe patch passed
_ Other Tests _
-1 ❌unit8m 31shbase-server in the patch failed.
27m 55s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5074
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 4c36e9caffdc 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / d2b0074
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/testReport/
Max. process+thread count1107 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 9, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 24sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall6m 18smaster passed
+1 💚compile3m 14smaster passed
+1 💚checkstyle0m 49smaster passed
+1 💚spotless0m 42sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 29smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 18sthe patch passed
+1 💚compile2m 22sthe patch passed
+1 💚javac2m 22sthe patch passed
+1 💚checkstyle0m 36sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 7sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs1m 38sthe patch passed
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
41m 13s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5074
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 9c24f928089a 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0491524
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 9, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec5m 11sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 4smaster passed
+1 💚compile0m 40smaster passed
+1 💚shadedjars4m 16sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 46sthe patch passed
+1 💚compile0m 39sthe patch passed
+1 💚javac0m 39sthe patch passed
+1 💚shadedjars4m 17spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 23sthe patch passed
_ Other Tests _
+1 💚unit219m 37shbase-server in the patch passed.
245m 50s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5074
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6818308a1df1 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0491524
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/testReport/
Max. process+thread count2575 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Mar 9, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 45sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall5m 30smaster passed
+1 💚compile1m 6smaster passed
+1 💚shadedjars6m 0sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 37smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 52sthe patch passed
+1 💚compile0m 58sthe patch passed
+1 💚javac0m 58sthe patch passed
+1 💚shadedjars5m 58spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 31sthe patch passed
_ Other Tests _
+1 💚unit237m 32shbase-server in the patch passed.
268m 40s
SubsystemReport/Notes
DockerClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5074
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 8d57064b95d3 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 0491524
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/testReport/
Max. process+thread count2673 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5074/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@bbeaudreault

bbeaudreault commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

Makes sense to me, but the UT failures look related

@sunhelly

sunhelly commented Mar 21, 2023

Copy link
Copy Markdown
ContributorAuthor

Thanks, @bbeaudreault . The last failed UT is related, I have fixed it.

@bbeaudreault

bbeaudreault commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

Oh sorry, my GitHub was outdated. I see pre-commit is green now. LGTM

@sunhelly
sunhelly merged commit 7d6a79b into apache:masterMar 22, 2023
asfgit pushed a commit that referenced this pull request Mar 22, 2023
@sunhelly
HBASE-27676Scan handlers in the RPC executor should match at least o…
d4edfe4
…ne scan queues (#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
asfgit pushed a commit that referenced this pull request Mar 22, 2023
@sunhelly
HBASE-27676Scan handlers in the RPC executor should match at least o…
8b0ae80
…ne scan queues (#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
asfgit pushed a commit that referenced this pull request Mar 22, 2023
@sunhelly
HBASE-27676Scan handlers in the RPC executor should match at least o…
40d2545
…ne scan queues (#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
wesiq pushed a commit to web-iq/hbase that referenced this pull request Mar 27, 2023
@sunhelly@wesiq
HBASE-27676 Scan handlers in the RPC executor should match at least o…
610e1ce
…ne scan queues (apache#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
@sunhelly
HBASE-27676 Scan handlers in the RPC executor should match at least o…
70459a1
…ne scan queues (apache#5074)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
(cherry picked from commit 8b0ae80)
Change-Id: I35de74ba99deb0adac3795d769b7eeaea4b97769
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bbeaudreaultbbeaudreaultbbeaudreault approved these changes
@Apache9Apache9Awaiting requested review from Apache9

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sunhelly@Apache-HBase@bbeaudreault