Uh oh!
There was an error while loading. Please reload this page.
HBASE-26566 Optimize encodeNumeric in OrderedBytes - #3940
Conversation
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 13, 2021
🎊 +1 overall
This message was automatically generated. |
Reidddddd
commented
Dec 15, 2021
Is there any test or unit test for verifying the behaviour of before-change and after-change is equal? |
YutSean
commented
Dec 15, 2021
There is a UT named TestOrderedNumeric. The changes passed this UT. |
Reidddddd
commented
Dec 15, 2021
I feel it is better to give more details, even math theoretics, to make me or other reviewers more clear and understand your changes... |
YutSean
commented
Dec 15, 2021
Have added details in the corresponding jira ticket. |
Apache9
commented
Dec 15, 2021
I think we'd better have some JMH tests to show the performance impact. |
YutSean
commented
Dec 15, 2021
Did a JMH test and attached the result in the jira ticket. |
YutSean
commented
Dec 16, 2021
Found the function could be optimized one step forward. Has update the code in the latest commit and upload the jmh test results in the jira ticket. |
Apache-HBase
commented
Dec 16, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 16, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 16, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 17, 2021
🎊 +1 overall
This message was automatically generated. |
Reidddddd
commented
Dec 17, 2021
The result seems be very impressive.
|
@Reidddddd After checking the current UT and some configurations. I found that, although the method encodes BigDecimal, but current MAX_PRECISION is really limited (31). It can only fit the domain of Long. I added some test cases beyond this domain. The old implementation will also fail. The new change just follows the old logic. If we do not need to fit some really big decimal, we do not need new UT I think. On the contrary if we want to support them, the function logic should be reconstructed and more large decimal cases should be added in current UTs. What do you think? |
| lengthToMoveLeft = 702; | ||
| } | ||
| abs = abs.movePointLeft(lengthToMoveLeft); | ||
There was a problem hiding this comment.
This old 350 limit is problematic, for value > 100^350, the abs will > 1, which will make the following steps have an incorrect encoding for M . And for value < 100^350, this limit is useless. This limit should be removed in my opinion.
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Dec 18, 2021
🎊 +1 overall
This message was automatically generated. |
ndimiduk
commented
Dec 23, 2021
Improvements to this code path are welcome! Thank you for putting together the jmh benchmark. For context, know that the original implementation was ported more-or-less directly from sqlite. At this point, the code is rather opaque to me. Is it the case that the new encoder is forward and backward compatible with the previous implementation? That is, can a byte[] written by the new encoder be read correctly by the old implementation? Can a byte[] written by the old implementation be read correctly by the new? Thanks. |
YutSean
commented
Dec 23, 2021
Yes. The new one is compatible with the previous implementation. The new encoder just optimized the performance but not changed the encoding logic and this commit only made changes at the encoding part. The old read function can correctly read the bytes encoded by the new encoding in our UTs. Besides, I also randomly generated 200 examples to check the correctness. The results of the both encoding are matched. |
Signed-off-by: Reid Chan <reidchan@apache.org>
Signed-off-by: Reid Chan <reidchan@apache.org>
https://issues.apache.org/jira/browse/HBASE-26566