Skip to content

HBASE-26479: Print too slow/big scan's operation_id in region server log - #3870

Merged
Apache9 merged 2 commits into
apache:masterfrom
liangxs:HBASE-26479
Nov 24, 2021
Merged

HBASE-26479: Print too slow/big scan's operation_id in region server log#3870
Apache9 merged 2 commits into
apache:masterfrom
liangxs:HBASE-26479

Conversation

@liangxs

@liangxsliangxs commented Nov 22, 2021

Copy link
Copy Markdown
Contributor

Tracing is very important in large-scale distributed systems.

The attribute _operation.attributes.id of a scan request can be used as trace id between hbase client and region server.

We should print operation id in region server's log if the scan request is too slow or too big.

It will be very helpful for finding problematic requests.

@Apache-HBase

Apache-HBase commented Nov 22, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 25sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 10smaster passed
+1 💚compile3m 17smaster passed
+1 💚checkstyle1m 4smaster passed
+1 💚spotbugs2m 5smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 52sthe patch passed
+1 💚compile3m 13sthe patch passed
+1 💚javac3m 13sthe patch passed
-0 ⚠️checkstyle1m 3shbase-server: The patch generated 1 new + 32 unchanged - 0 fixed = 33 total (was 32)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 36sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs2m 18sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
49m 29s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 8a8dcfd9ca18 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 04635f4
Default JavaAdoptOpenJDK-1.8.0_282-b08
checkstylehttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count96 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9
Apache9 reviewed Nov 22, 2021
View reviewed changes

@Apache9Apache9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind adding a simple UT for this?

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
@Apache-HBase

Apache-HBase commented Nov 22, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 25sDocker mode activated.
-0 ⚠️yetus0m 4sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 14smaster passed
+1 💚compile1m 2smaster passed
+1 💚shadedjars8m 18sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 40smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 53sthe patch passed
+1 💚compile1m 1sthe patch passed
+1 💚javac1m 1sthe patch passed
+1 💚shadedjars8m 17spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38sthe patch passed
_ Other Tests _
+1 💚unit153m 6shbase-server in the patch passed.
183m 41s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 067b5a2dbe59 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 04635f4
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/testReport/
Max. process+thread count4821 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Nov 23, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 7sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall8m 57smaster passed
+1 💚compile3m 30smaster passed
+1 💚checkstyle1m 48smaster passed
+1 💚spotbugs2m 19smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 14sthe patch passed
+1 💚compile3m 25sthe patch passed
+1 💚javac3m 25sthe patch passed
+1 💚checkstyle1m 11sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck22m 23sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs2m 23sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
60m 33s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 457fb5cafbd6 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b870b6d
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Nov 23, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 2sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall10m 3smaster passed
+1 💚compile1m 5smaster passed
+1 💚shadedjars9m 9sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 17sthe patch passed
+1 💚compile1m 3sthe patch passed
+1 💚javac1m 3sthe patch passed
+1 💚shadedjars9m 0spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
+1 💚unit239m 35shbase-server in the patch passed.
278m 30s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b5d7c739b6ba 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b870b6d
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/testReport/
Max. process+thread count2398 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Nov 23, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 12smaster passed
+1 💚compile3m 11smaster passed
+1 💚checkstyle1m 5smaster passed
+1 💚spotbugs2m 6smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 49sthe patch passed
+1 💚compile3m 11sthe patch passed
+1 💚javac3m 11sthe patch passed
+1 💚checkstyle1m 3sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 27sPatch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚spotbugs2m 17sthe patch passed
_ Other Tests _
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
49m 8s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
unameLinux 7d3e7d1ed60d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b870b6d
Default JavaAdoptOpenJDK-1.8.0_282-b08
Max. process+thread count95 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Nov 23, 2021

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 9sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 35smaster passed
+1 💚compile1m 7smaster passed
+1 💚shadedjars9m 30sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 45smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 19sthe patch passed
+1 💚compile1m 6sthe patch passed
+1 💚javac1m 6sthe patch passed
+1 💚shadedjars9m 6spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 38sthe patch passed
_ Other Tests _
+1 💚unit220m 41shbase-server in the patch passed.
254m 47s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#3870
JIRA IssueHBASE-26479
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a607f20de5a7 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / b870b6d
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/testReport/
Max. process+thread count3540 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Apache9 merged commit c148d5b into apache:masterNov 24, 2021
Apache9 pushed a commit that referenced this pull request Nov 24, 2021
@liangxs@Apache9
HBASE-26479Print too slow/big scan's operation_id in region server l…
b437a96
…og (#3870)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
asfgit pushed a commit that referenced this pull request Nov 24, 2021
@apurtell
Revert "HBASE-26479Print too slow/big scan's operation_id in region …
52ce877
…server log (#3870)"
This reverts commit b437a96.
Apache9 pushed a commit that referenced this pull request Nov 25, 2021
@liangxs@Apache9
HBASE-26479Print too slow/big scan's operation_id in region server l…
2798ef1
…og (#3870)
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@liangxs@Apache-HBase@Apache9