Uh oh!
There was an error while loading. Please reload this page.
HBASE-27144 Improvement bulkload rpc - #4558
Conversation
Apache-HBase
commented
Jun 22, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 22, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 23, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 23, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 23, 2022
💔 -1 overall
This message was automatically generated. |
SiCheng-Zheng
commented
Jun 27, 2022
Bulkload will consume a lot of resources in the cluster. We try to reduce the impact of bulkload on online services and do simple resource isolation for bulkload. This feature has been running online for 2 years and works well @Apache9 |
Apache9
left a comment
There was a problem hiding this comment.
Overall LGTM.
Just some concerns about the implementation details.
| * @param highPriorityLevel high Priority Level | ||
| * @param priority Function to extract request priority. | ||
| */ | ||
| public SimpleRpcScheduler(Configuration conf, int handlerCount, int priorityHandlerCount, |
There was a problem hiding this comment.
Is the below constructor still valid? In general I suggest we move the code in the below method to this method and make the below method call this method, with bulkLoadHandlerCount = 0, and better also make the below method as deprecated, and remove it in 3.0.0, for example, as it is IA.LimitedPrivate.
| highPriorityLevel); | ||
| } | ||
| public SimpleRpcScheduler(Configuration conf, int handlerCount, int priorityHandlerCount, |
| conf.getInt(HConstants.MASTER_META_TRANSITION_HANDLER_COUNT, | ||
| HConstants.MASTER__META_TRANSITION_HANDLER_COUNT_DEFAULT), | ||
| priority, server, HConstants.QOS_THRESHOLD); | ||
| boolean bulkLoadIndependenceQueueEnabled = |
There was a problem hiding this comment.
Do we need a separated config to enable this feature? I suppose we just set the handler count to 0 is enough?
SiCheng-Zheng
commented
Jun 29, 2022
@Apache9 Edit as you suggested |
Apache-HBase
commented
Jun 29, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 29, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 29, 2022
💔 -1 overall
This message was automatically generated. |
SiCheng-Zheng
commented
Jul 4, 2022
Can this PR be merged into the master branch? @Apache9 |
SiCheng-Zheng
commented
Jul 11, 2022
Independent bulkload rpc is not enabled by default bulkLoadHandlerCount = 0 |
Apache-HBase
commented
Jul 11, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 11, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jul 11, 2022
🎊 +1 overall
This message was automatically generated. |
Co-authored-by: SiCheng-Zheng <zhengsicheng@jd.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> (cherry picked from commit ff8eb59) Conflicts: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcScheduler.java hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/AnnotationReadingPriorityFunction.java
No description provided.