Skip to content

HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRef… - #950

Closed
virajjasani wants to merge 1 commit into
apache:masterfrom
virajjasani:HBASE-23590-master
Closed

HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRef…#950
virajjasani wants to merge 1 commit into
apache:masterfrom
virajjasani:HBASE-23590-master

Conversation

@virajjasani

@virajjasanivirajjasani commented Dec 18, 2019

Copy link
Copy Markdown
Contributor

…Count for auto region recovery based on old reader references

@virajjasani
HBASE-23590 : Update maxStoreFileRefCount to maxCompactedStoreFileRef…
c405b4c
…Count for auto region recovery based on old reader references
@Apache-HBase

Apache-HBase commented Dec 18, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 0sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 3 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep0m 37sMaven dependency ordering for branch
+1 💚mvninstall5m 19smaster passed
+1 💚compile3m 0smaster passed
+1 💚checkstyle2m 30smaster passed
+0 🆗refguide5m 32sbranch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars4m 36sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc5m 2smaster passed
+0 🆗spotbugs3m 53sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs24m 13smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
+1 💚mvninstall4m 52sthe patch passed
+1 💚compile3m 1sthe patch passed
+1 💚cc3m 1sthe patch passed
+1 💚javac3m 1sthe patch passed
+1 💚checkstyle2m 36sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚xml0m 1sThe patch has no ill-formed XML file.
+0 🆗refguide5m 30spatch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚shadedjars4m 37spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck15m 42sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚hbaseprotoc8m 42sthe patch passed
+1 💚javadoc5m 13sthe patch passed
+1 💚findbugs25m 26sthe patch passed
_ Other Tests _
-1 ❌unit228m 42sroot in the patch failed.
+1 💚asflicense4m 36sThe patch does not generate ASF License warnings.
370m 23s
ReasonTests
Failed junit testshadoop.hbase.snapshot.TestExportSnapshotNoCluster
SubsystemReport/Notes
DockerClient=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/artifact/out/Dockerfile
GITHUB PR#950
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc prototool
unameLinux 0952b0c2c878 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-950/out/precommit/personality/provided.sh
git revisionmaster / 17e180e
Default Java1.8.0_181
refguidehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/artifact/out/branch-site/book.html
refguidehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/artifact/out/patch-site/book.html
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/artifact/out/patch-unit-root.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/testReport/
Max. process+thread count5345 (vs. ulimit of 10000)
modulesC: hbase-protocol-shaded hbase-common hbase-hadoop-compat hbase-hadoop2-compat hbase-protocol hbase-client hbase-server . U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-950/1/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

HorizonNet
HorizonNet requested changes Dec 30, 2019
View reviewed changes

@HorizonNetHorizonNet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, but left a comment regarding version compatibility.

Comment threadhbase-client/src/main/java/org/apache/hadoop/hbase/RegionMetrics.java
* of this region
*/
int getMaxStoreFileRefCount();
int getMaxCompactedStoreFileRefCount();

@HorizonNetHorizonNetDec 30, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because RegionMetrics is a public interface, could you please deprecate getMaxStoreFileRefCount and add getMaxCompactedStoreFileRefCount as a new method?

@virajjasanivirajjasaniDec 30, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh okk, yes ideally we would do that but in this case, this change int getMaxStoreFileRefCount(); has not yet landed to any release so far and is available to 2.3.0 and 1.6.0 only, no other release branch. Hence, the plan is to update this method now only before 2.3.0 or 1.6.0 makes it to releases.
Thanks

@HorizonNetHorizonNetDec 30, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok. This seems like a minor gap in our documentation on how we handle deprecations. Would be good to have a common sense and a documentation for it. @saintstack Any thoughts?

@virajjasanivirajjasaniDec 30, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with the suggestion.
This new IA method was introduced as part of Jira HBASE-22460 and HBASE-23213 which are not part of any active release so far. Hence IMO, before 2.3.0 or 1.6.0 goes live, ideally we should be good to make this change, but yes agree to follow standard best practices reg such cases if we have common practice available.
It could also be argued that we can change any Public IA method signatures as long as they are not yet present in any active release.

cc: @apurtell@anoopsjohn

anoopsjohn
anoopsjohn approved these changes Dec 31, 2019
View reviewed changes

@anoopsjohnanoopsjohn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now we have this max compacted file ref count metric only been sending to HM right? No other metric to HM right regarding the store file ref counts

@virajjasani

virajjasani commented Dec 31, 2019

Copy link
Copy Markdown
ContributorAuthor

So now we have this max compacted file ref count metric only been sending to HM right? No other metric to HM right regarding the store file ref counts

That is correct, this is the only one we are concerned about

@virajjasani

virajjasani commented Dec 31, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@HorizonNet should be good to go? Please let me know.
Thanks

@virajjasani
virajjasani deleted the HBASE-23590-master branch January 1, 2020 18:36
@JeongDaeKimJeongDaeKim mentioned this pull request Jan 2, 2020
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HorizonNetHorizonNetHorizonNet requested changes
@anoopsjohnanoopsjohnanoopsjohn approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@virajjasani@Apache-HBase@HorizonNet@anoopsjohn