Uh oh!
There was an error while loading. Please reload this page.
HBASE-24659 Calculate FIXED_OVERHEAD automatically - #2018
Merged
Conversation
Apache-HBase
commented
Jul 5, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 5, 2020
🎊 +1 overall
This message was automatically generated. |
Apache9
requested changes
Jul 5, 2020
Apache9
left a comment
Contributor
There was a problem hiding this comment.
I wonder do we still need the testSizes method in TestHeapSize after this change?
And we need a performance test to see if there are any impact on performance(although I believe no)
Apache-HBase
commented
Jul 5, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 5, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 5, 2020
🎊 +1 overall
This message was automatically generated. |
| 3 * Bytes.SIZEOF_INT + | ||
| 14 * Bytes.SIZEOF_LONG + | ||
| 3 * Bytes.SIZEOF_BOOLEAN); | ||
| public static final long FIXED_OVERHEAD = ClassSize.estimateBase(HRegion.class, false); |
Contributor
There was a problem hiding this comment.
Does ClassSize come up w/ same general numbers as old manual technique. It does deep size rather than shallow?
ContributorAuthor
There was a problem hiding this comment.
I think ClassSize.estimateBase only calculate shallow size
Contributor
There was a problem hiding this comment.
I thought it critical it did deep size? Do you get the same numbers roughly?
ContributorAuthor
Yes, this change only affect class loading phase. And the calculation of FIXED_OVERHEAD cost less than 5 milliseconds |
Apache-HBase
commented
Jul 12, 2020
🎊 +1 overall
This message was automatically generated. |
Apache9
commented
Jul 12, 2020
Contributor
Let's run a YCSB or at least a PE to see the performance impact? |
nyl3532016
commented
Jul 12, 2020
ContributorAuthor
yes,it is necessary, Let me do it |
Apache-HBase
commented
Jul 12, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 12, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 13, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 13, 2020
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 13, 2020
🎊 +1 overall
This message was automatically generated. |
nyl3532016
commented
Aug 1, 2020
ContributorAuthor
@Apache9 sir,I have done the performance test, and attaches on jira page, please take a look ? |
Apache9
approved these changes
Aug 1, 2020
Apache9
left a comment
Contributor
There was a problem hiding this comment.
Seems no big difference.
+1.
Apache9
commented
Aug 1, 2020
Contributor
@saintstack PTAL. If no objections, I will push this to master and branch-2. Thanks. |
nyl3532016
commented
Aug 6, 2020
ContributorAuthor
@saintstack please have a look, thanks |
Apache9
commented
Aug 6, 2020
Contributor
Oh, forgot to merge this one. Since you have pinged @saintstack again, let's wait until tomorrow... |
saintstack
commented
Aug 6, 2020
Contributor
Pardon me for missing your pings @nyl3532016 ... let me look. |
saintstack
approved these changes
Aug 6, 2020
saintstack
left a comment
Contributor
There was a problem hiding this comment.
Thank you for waiting on my input. I had concerns but after checking this patch again and going back through git history, I see better what is going on and +1 this approach. Thank you for running the PE check. Good. This PR might make our class loading take a little longer but should have no effect on general perf I'd think. Thanks for the patch.
| 4 * Bytes.SIZEOF_BOOLEAN + Bytes.SIZEOF_LONG + | ||
| //byte[] headers for column family and table name | ||
| 2 * ClassSize.ARRAY + 2 * ClassSize.REFERENCE); | ||
| public static final long FIXED_OVERHEAD = ClassSize.estimateBase(HFileContext.class, false); |
Contributor
There was a problem hiding this comment.
Or, sorry, yeah, FIXED_OVERHEAD is 'this' classes size; i.e. the shallow size (I think).
Contributor
There was a problem hiding this comment.
I went back through history of ClassSize... its so old.
saintstack pushed a commit
that referenced
this pull request
Aug 6, 2020
Co-authored-by: niuyulin <niuyulin@xiaomi.com> SIgned-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: stack <stack@apache.org>
clarax pushed a commit
to clarax/hbase
that referenced
this pull request
Nov 15, 2020
Co-authored-by: niuyulin <niuyulin@xiaomi.com> SIgned-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: stack <stack@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.