Skip to content

HBASE-29233: Capture scan metrics at region level - #7132

Merged
virajjasani merged 3 commits into
apache:branch-2.6from
sanjeet006py:scan-metrics-by-region-2.6
Jul 2, 2025
Merged

HBASE-29233: Capture scan metrics at region level#7132
virajjasani merged 3 commits into
apache:branch-2.6from
sanjeet006py:scan-metrics-by-region-2.6

Conversation

@sanjeet006py

@sanjeet006pysanjeet006py commented Jun 29, 2025

Copy link
Copy Markdown
Contributor

Cherry-pick of #6868 for HBase 2.

Modifications done for HBase 2:

  • Sync HBase client read path is different in HBase 2 from HBase 3 and 4. Thus, modifications have been done to handle only this part.
  • In HBase 3 and 4, scan metrics by region are available for sync and async client both for STRONG consistency and TIMELINE consistency. But in HBase 2, for sync client with TIMELINE consistency, scan metrics by region is not supported. If users enable the flag then in constructor of ClientScanner we disable it and log a warning stating the same.
  • In ClientScanner used by sync clients, two modifications have been done to add support for scan metrics by region:
    - In moveToNextRegion() we conditionally call scanMetrics.moveToNextRegion().
    - We initialize the ScanMetricsRegionInfo at same time when we capture current region being scanned in ClientScanner. Code to capture current region being scanned was already there in ClientScanner.
  • From HBase 3 onwards, scan metrics for NSRE count and milliseconds spent b/w two consecutive next() calls, are not used. But in HBase 2 both above metrics are used so, they also had to be initialized in ScanMetrics.moveToNextRegion() and ITs had to be adjusted accordingly.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 55sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ branch-2.6 Compile Tests _
+0 🆗mvndep0m 35sMaven dependency ordering for branch
+1 💚mvninstall4m 12sbranch-2.6 passed
+1 💚compile7m 36sbranch-2.6 passed
+1 💚checkstyle1m 24sbranch-2.6 passed
+1 💚spotbugs12m 5sbranch-2.6 passed
+1 💚spotless1m 7sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall3m 37sthe patch passed
+1 💚compile7m 25sthe patch passed
-0 ⚠️javac7m 25s/results-compile-javac-root.txtroot generated 2 new + 1268 unchanged - 2 fixed = 1270 total (was 1270)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle1m 40sthe patch passed
+1 💚spotbugs13m 57sthe patch passed
+1 💚hadoopcheck19m 21sPatch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚spotless0m 48spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 37sThe patch does not generate ASF License warnings.
78m 45s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#7132
Optional Testsdupname asflicense codespell detsecrets spotless javac spotbugs checkstyle compile hadoopcheck hbaseanti
unameLinux 19896351b483 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.6 / 19457bb
Default JavaEclipse Adoptium-11.0.23+9
Max. process+thread count179 (vs. ulimit of 30000)
modulesC: hbase-client hbase-server . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 58sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall3m 18sbranch-2.6 passed
+1 💚compile2m 15sbranch-2.6 passed
+1 💚javadoc2m 45sbranch-2.6 passed
+1 💚shadedjars6m 37sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall3m 18sthe patch passed
+1 💚compile2m 12sthe patch passed
+1 💚javac2m 12sthe patch passed
+1 💚javadoc2m 51sthe patch passed
+1 💚shadedjars6m 14spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit37m 40s/patch-unit-root.txtroot in the patch failed.
71m 27s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#7132
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 319b98ff678a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.6 / 19457bb
Default JavaTemurin-1.8.0_412-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/testReport/
Max. process+thread count2439 (vs. ulimit of 30000)
modulesC: hbase-client hbase-server . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 52sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for branch
+1 💚mvninstall3m 25sbranch-2.6 passed
+1 💚compile2m 23sbranch-2.6 passed
+1 💚javadoc3m 11sbranch-2.6 passed
+1 💚shadedjars7m 51sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall3m 25sthe patch passed
+1 💚compile3m 9sthe patch passed
+1 💚javac3m 9sthe patch passed
-0 ⚠️javadoc0m 28s/results-javadoc-javadoc-hbase-client.txthbase-client generated 11 new + 83 unchanged - 0 fixed = 94 total (was 83)
-0 ⚠️javadoc2m 23s/results-javadoc-javadoc-root.txtroot generated 11 new + 199 unchanged - 0 fixed = 210 total (was 199)
+1 💚shadedjars6m 51spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit304m 45sroot in the patch passed.
347m 31s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#7132
Optional Testsjavac javadoc unit compile shadedjars
unameLinux a49221a91cea 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.6 / 19457bb
Default JavaEclipse Adoptium-11.0.23+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/testReport/
Max. process+thread count9047 (vs. ulimit of 30000)
modulesC: hbase-client hbase-server . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 53sDocker mode activated.
-0 ⚠️yetus0m 6sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.6 Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for branch
+1 💚mvninstall4m 17sbranch-2.6 passed
+1 💚compile3m 6sbranch-2.6 passed
+1 💚javadoc3m 22sbranch-2.6 passed
+1 💚shadedjars7m 1sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 13sMaven dependency ordering for patch
+1 💚mvninstall3m 24sthe patch passed
+1 💚compile2m 23sthe patch passed
+1 💚javac2m 23sthe patch passed
+1 💚javadoc3m 11sthe patch passed
+1 💚shadedjars7m 3spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit317m 44sroot in the patch passed.
360m 45s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#7132
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 75ef32672d75 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.6 / 19457bb
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/testReport/
Max. process+thread count8803 (vs. ulimit of 30000)
modulesC: hbase-client hbase-server . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7132/2/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@virajjasanivirajjasani left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@virajjasani
virajjasani merged commit 4fde3b2 into apache:branch-2.6Jul 2, 2025
virajjasani pushed a commit that referenced this pull request Jul 2, 2025
Signed-off-by: Viraj Jasani <vjasani@apache.org>
sanjeet006py added a commit to sanjeet006py/hbase that referenced this pull request Jul 3, 2025
mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@sanjeet006py@Apache-HBase@virajjasani