Uh oh!
There was an error while loading. Please reload this page.
HBASE-26340 - fix RegionSizeCalculator getLEngth to bytes instead of … - #3872
Merged
Conversation
Apache-HBase
commented
Nov 22, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 22, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 22, 2021
🎊 +1 overall
This message was automatically generated. |
petersomogyi
approved these changes
Nov 24, 2021
Apache9
reviewed
Nov 24, 2021
| totalStaticBloomSizeKB += (int) (store.getTotalStaticBloomSize() / 1024); | ||
| } | ||
| //HBASE-26340 Fix false "0" size under 1MB | ||
| if(storefileSizeMB < 1 && nonEmptyStoreExist) { |
Contributor
There was a problem hiding this comment.
Do we need to apply this trick to other fields as well? Just introduce a round method
long round(long size, long unit) {
// normally return size / unit, if size is smaller than unit but greater than 0, return 1
}
And all the size should be calculated by this method
ContributorAuthor
There was a problem hiding this comment.
storefileSizeMB is integer, it is either 0 or 1 here (well, or any other positive integer, but staying at the less then 1MB example), and we go through all the storelist above, if we do the rounding there, we keep adding 1MB every time there is like 10KB of data, that would result in a false size as well. We need "nonEmptyStoreExist" to tell us if there were storefile anywhere, or else we'll just see 0 here and cant round up to 1 bc it was either really empty or size didn't reach the 1MB
ContributorAuthor
There was a problem hiding this comment.
I could use a new variable "long storefileSizeMBTemp" and create a function "int round(long size, int unit)" but it just seems kind of a special case to create a function for it.
Contributor
There was a problem hiding this comment.
At region server side we should know the accurate size in bytes, so when calculating we could just cumulate the size in bytes, and at last we convert them to size in KB or size in MB or what ever unit. WDYT?
ContributorAuthor
There was a problem hiding this comment.
That works, sure, at least I think we are thinking the same when I said above we could use long to track "storefileSize(MB)(Temp)" - no need for Temp though just remove MB - but at the end we set this as MB just like before. I'll do the change today.
…ean check Change-Id: Idce876f4b95b0f28e3e84a08b367f77bc76fb268
Apache9
approved these changes
Dec 8, 2021
Apache9
left a comment
Contributor
There was a problem hiding this comment.
It is OK for fixing the problem in HBASE-26340, so +1 on merging.
But I think we'd better have a follow on issue to change all other size calculating using the same pattern, such as the storeUncompressedSizeMB, memstoreSizeMB, etc.
Thanks~
Apache-HBase
commented
Dec 8, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 8, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 8, 2021
🎊 +1 overall
This message was automatically generated. |
Apache9
commented
Dec 17, 2021
Contributor
The pre commit is still running? Anyway, let's wait for the newest result... |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Apache9 pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <psomogyi@apache.org Signed-off-by: Duo Zhang <zhangduo@apache.org>
Apache9 pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <psomogyi@apache.org Signed-off-by: Duo Zhang <zhangduo@apache.org>
Apache9 pushed a commit
that referenced
this pull request
Dec 18, 2021
Signed-off-by: Peter Somogyi <psomogyi@apache.org Signed-off-by: Duo Zhang <zhangduo@apache.org>
vinayakphegde pushed a commit
to vinayakphegde/hbase
that referenced
this pull request
Apr 4, 2024
Signed-off-by: Peter Somogyi <psomogyi@apache.org Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit 8fa8344) Change-Id: I380b2e9acd4385a2a1228e1115f686c63410ef6e
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If we have store files under 1MB, list_regions will report region size of 0. This fix will use 1MB of value if there are store files available for a region. It makes more sense than using 0MB if store file size are less than 1MB.