Uh oh!
There was an error while loading. Please reload this page.
Introduce a Snapshot Procedure - #4115
Conversation
Apache-HBase
commented
Feb 15, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 15, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 15, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 16, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 17, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 17, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 17, 2022
🎊 +1 overall
This message was automatically generated. |
frostruan
commented
Feb 21, 2022
hi duo, I've made some progress recently.
If you have time, would you mind taking a look on this at your convenience and see if there is any problems ? Thanks. @Apache9 |
Apache9
left a comment
There was a problem hiding this comment.
Overall LGTM.
Only need to confirm that the fencing method is correct.
Thanks for the hard work @frostruan !
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| } | ||
| @RestrictedApi(explanation = "Should only be called in tests", link = "", | ||
| allowedOnPath = ".*(/src/test/.*|TestSnapshotProcedure).java") |
There was a problem hiding this comment.
'src/test/.* ' contains TestSnapshotProcedure?
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| .filter(p -> !p.isFinished()) | ||
| .filter(p -> p.getServerName() != null) | ||
| .forEach(p -> { | ||
| verifyWorkerAssigner.addUsedWorker(p.getServerName()); |
There was a problem hiding this comment.
What if the region server is already dead?
There was a problem hiding this comment.
I think it's harmless that we have dead servers in the WorkAssigner based on this consideration:
- If the remote server is dead, there will be a FailedRemoteDispatchException thrown when we dispatch remote procedures. The procedure-v2 framework is designed to be able to handle this. The upper layer procedure can notice that and reschedule a new procedure, or the procedure can retry itself.
- When acquire a new worker from WorkAssigner, we will always choose from the online server list. The dead server will not be able to become a candidate again.
- If there are too many dead servers in the WorkerAssigner, we can restart the master to clear it. This is a very lightweight operation.
There was a problem hiding this comment.
Seems this is the behavior of the SplitWALManager, so I guess it is OK.
| private void prepareSnapshot(MasterProcedureEnv env) | ||
| throws ProcedureSuspendedException, IOException { | ||
| if (isAnySplitOrMergeProcedureRunning(env)) { |
There was a problem hiding this comment.
So when arriving here, we can not schedule new split or merge procedures for this table any more as we have already record this procedure in the SnapshotManager?
There was a problem hiding this comment.
Yes. That's correct. In fact, in the current implementation, if a zk-coordinated snapshot is in progress, the split/merge procedure on the same table will be marked as FAILED.
frostruan
commented
Feb 25, 2022
Really thanks for your great suggestions and help, Duo. I will address it ASAP. @Apache9 |
Apache-HBase
commented
Feb 26, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 26, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 26, 2022
💔 -1 overall
This message was automatically generated. |
Apache9
left a comment
There was a problem hiding this comment.
Overall LGTM. Just need to fix some minor nits.
I think we can merge it to master and branch-2 first, and then polish other things.
Thanks @frostruan !
| .filter(p -> !p.isFinished()) | ||
| .filter(p -> p.getServerName() != null) | ||
| .forEach(p -> { | ||
| verifyWorkerAssigner.addUsedWorker(p.getServerName()); |
There was a problem hiding this comment.
Seems this is the behavior of the SplitWALManager, so I guess it is OK.
frostruan
commented
Mar 12, 2022
Really appreciate your help, Duo @Apache9 |
Signed-off-by: Duo Zhang <zhangduo@apache.org>
No description provided.