Skip to content

HBASE-23210 Backport HBASE-15519 (Add per-user metrics) to branch-1 - #755

Merged
apurtell merged 1 commit into
apache:branch-1from
apurtell:HBASE-23210-branch-1
Oct 28, 2019
Merged

HBASE-23210 Backport HBASE-15519 (Add per-user metrics) to branch-1#755
apurtell merged 1 commit into
apache:branch-1from
apurtell:HBASE-23210-branch-1

Conversation

@apurtell

Copy link
Copy Markdown
Contributor

HBASE-15519 Add per-user metrics with lossy counting

Introducing property hbase.regionserver.user.metrics.enabled(Default:true)
to disable user metrics in case it accounts for any performance issues

@apurtell

Copy link
Copy Markdown
ContributorAuthor

New unit tests pass

@apurtell

Copy link
Copy Markdown
ContributorAuthor

@joshelser I couldn't ask @ankitsinghal for a review for some reason, but i see you recently reviewed similar

@apurtell
apurtellforce-pushed the HBASE-23210-branch-1 branch from b770177 to b31f442CompareOctober 24, 2019 22:47

@joshelserjoshelser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things, but LGTM.

class SweepRunnable implements Runnable {
@Override public void run() {
if (LOG.isTraceEnabled()) {
LOG.trace("Starting sweep of lossyCounting-" + name);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, could use the {} markers and drop the isTraceEnabled()

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This came in from branch-2 change. I agree but try not to second guess the original contributor on backports.

@joshelserjoshelser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things, but LGTM.

try {
sweep();
} catch (Exception exception) {
LOG.debug("Error while sweeping of lossyCounting-{}", name, exception);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(damnit, lost the previous comment)

Make sure you're getting the right debug method here. The debug(String, Object, Exception) not debug(String, Object, Object). Hit this the other day.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing

@apurtell
apurtellforce-pushed the HBASE-23210-branch-1 branch from b31f442 to ba2d5ffCompareOctober 24, 2019 23:09
@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec2m 29sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 4 new or modified test files.
_ branch-1 Compile Tests _
💙mvndep1m 27sMaven dependency ordering for branch
💚mvninstall8m 35sbranch-1 passed
💚compile1m 27sbranch-1 passed with JDK v1.8.0_232
💚compile1m 32sbranch-1 passed with JDK v1.7.0_242
💚checkstyle2m 25sbranch-1 passed
💚shadedjars4m 23sbranch has no errors when building our shaded downstream artifacts.
💚javadoc1m 14sbranch-1 passed with JDK v1.8.0_232
💚javadoc1m 32sbranch-1 passed with JDK v1.7.0_242
💙spotbugs3m 20sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 43sbranch-1 passed
_ Patch Compile Tests _
💙mvndep0m 20sMaven dependency ordering for patch
💚mvninstall2m 38sthe patch passed
💚compile1m 31sthe patch passed with JDK v1.8.0_232
💚javac1m 31sthe patch passed
💚compile1m 42sthe patch passed with JDK v1.7.0_242
💚javac1m 42sthe patch passed
💔checkstyle1m 58shbase-server: The patch generated 6 new + 4 unchanged - 0 fixed = 10 total (was 4)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars4m 14spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck6m 38sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
💚javadoc1m 17sthe patch passed with JDK v1.8.0_232
💚javadoc1m 31sthe patch passed with JDK v1.7.0_242
💚findbugs5m 19sthe patch passed
_ Other Tests _
💚unit0m 32shbase-hadoop-compat in the patch passed.
💚unit0m 44shbase-hadoop2-compat in the patch passed.
💔unit180m 41shbase-server in the patch failed.
💚asflicense1m 7sThe patch does not generate ASF License warnings.
246m 18s
ReasonTests
Failed junit testshadoop.hbase.master.TestWarmupRegion
hadoop.hbase.master.TestMasterBalanceThrottling
SubsystemReport/Notes
DockerClient=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/1/artifact/out/Dockerfile
GITHUB PR#755
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux a8218eedfc70 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-755/out/precommit/personality/provided.sh
git revisionbranch-1 / 41f6713
Default Java1.7.0_242
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/1/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/1/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/1/testReport/
Max. process+thread count3624 (vs. ulimit of 10000)
modulesC: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/1/console
versionsgit=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@apurtell

Copy link
Copy Markdown
ContributorAuthor

Back soon to fix checkstyle issues

@apurtell
apurtellforce-pushed the HBASE-23210-branch-1 branch from ba2d5ff to dc96c24CompareOctober 25, 2019 04:25
@apurtell

Copy link
Copy Markdown
ContributorAuthor

Fixed some checkstyle nits, will merge tomorrow

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec1m 25sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 4 new or modified test files.
_ branch-1 Compile Tests _
💙mvndep1m 25sMaven dependency ordering for branch
💚mvninstall7m 32sbranch-1 passed
💚compile1m 13sbranch-1 passed with JDK v1.8.0_232
💚compile1m 20sbranch-1 passed with JDK v1.7.0_242
💚checkstyle2m 12sbranch-1 passed
💚shadedjars3m 16sbranch has no errors when building our shaded downstream artifacts.
💚javadoc1m 1sbranch-1 passed with JDK v1.8.0_232
💚javadoc1m 14sbranch-1 passed with JDK v1.7.0_242
💙spotbugs3m 0sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 9sbranch-1 passed
_ Patch Compile Tests _
💙mvndep0m 15sMaven dependency ordering for patch
💚mvninstall2m 9sthe patch passed
💚compile1m 8sthe patch passed with JDK v1.8.0_232
💚javac1m 8sthe patch passed
💚compile1m 21sthe patch passed with JDK v1.7.0_242
💚javac1m 21sthe patch passed
💔checkstyle1m 44shbase-server: The patch generated 6 new + 4 unchanged - 0 fixed = 10 total (was 4)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars3m 8spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck5m 39sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
💚javadoc0m 58sthe patch passed with JDK v1.8.0_232
💚javadoc1m 19sthe patch passed with JDK v1.7.0_242
💚findbugs4m 41sthe patch passed
_ Other Tests _
💚unit0m 27shbase-hadoop-compat in the patch passed.
💚unit0m 36shbase-hadoop2-compat in the patch passed.
💔unit164m 24shbase-server in the patch failed.
💚asflicense1m 11sThe patch does not generate ASF License warnings.
218m 57s
ReasonTests
Failed junit testshadoop.hbase.master.TestMasterBalanceThrottling
hadoop.hbase.master.cleaner.TestReplicationZKNodeCleaner
SubsystemReport/Notes
DockerClient=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/2/artifact/out/Dockerfile
GITHUB PR#755
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 72d01cb3876c 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-755/out/precommit/personality/provided.sh
git revisionbranch-1 / 41f6713
Default Java1.7.0_242
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/2/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/2/testReport/
Max. process+thread count3655 (vs. ulimit of 10000)
modulesC: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/2/console
versionsgit=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec2m 15sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 4 new or modified test files.
_ branch-1 Compile Tests _
💙mvndep1m 23sMaven dependency ordering for branch
💚mvninstall7m 57sbranch-1 passed
💚compile1m 32sbranch-1 passed with JDK v1.8.0_232
💚compile1m 39sbranch-1 passed with JDK v1.7.0_242
💚checkstyle2m 25sbranch-1 passed
💚shadedjars4m 10sbranch has no errors when building our shaded downstream artifacts.
💚javadoc1m 15sbranch-1 passed with JDK v1.8.0_232
💚javadoc1m 31sbranch-1 passed with JDK v1.7.0_242
💙spotbugs3m 20sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs4m 42sbranch-1 passed
_ Patch Compile Tests _
💙mvndep0m 18sMaven dependency ordering for patch
💚mvninstall2m 34sthe patch passed
💚compile1m 31sthe patch passed with JDK v1.8.0_232
💚javac1m 31sthe patch passed
💚compile1m 42sthe patch passed with JDK v1.7.0_242
💚javac1m 42sthe patch passed
💔checkstyle2m 7shbase-server: The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars3m 49spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck6m 20sPatch does not cause any errors with Hadoop 2.8.5 2.9.2.
💚javadoc1m 9sthe patch passed with JDK v1.8.0_232
💚javadoc1m 30sthe patch passed with JDK v1.7.0_242
💚findbugs5m 32sthe patch passed
_ Other Tests _
💚unit0m 31shbase-hadoop-compat in the patch passed.
💚unit0m 41shbase-hadoop2-compat in the patch passed.
💚unit144m 16shbase-server in the patch passed.
💚asflicense0m 57sThe patch does not generate ASF License warnings.
207m 44s
SubsystemReport/Notes
DockerClient=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/3/artifact/out/Dockerfile
GITHUB PR#755
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 103ee95c33b3 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-755/out/precommit/personality/provided.sh
git revisionbranch-1 / 41f6713
Default Java1.7.0_242
Multi-JDK versions/usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/3/artifact/out/diff-checkstyle-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/3/testReport/
Max. process+thread count4510 (vs. ulimit of 10000)
modulesC: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-755/3/console
versionsgit=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

HBASE-15519 Add per-user metrics with lossy counting
Introducing property hbase.regionserver.user.metrics.enabled(Default:true)
to disable user metrics in case it accounts for any performance issues
@apurtell
apurtellforce-pushed the HBASE-23210-branch-1 branch from dc96c24 to 5127c2eCompareOctober 28, 2019 16:03
@apurtell

Copy link
Copy Markdown
ContributorAuthor

Fixed the other checkstyle indent nit

@apurtell
apurtell merged commit d16cbfe into apache:branch-1Oct 28, 2019
@apurtell
apurtell deleted the HBASE-23210-branch-1 branch October 28, 2019 16:05
this.data = new ConcurrentHashMap<>();
this.listener = listener;
calculateCurrentTerm();
executor = Executors.newSingleThreadExecutor();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this causes the RegionServer process to hang upon shutdown since we're not using daemon threads.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@apurtell@Apache-HBase@joshelser@lhofhansl