HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API

[jojochuang]

What changes were proposed in this pull request?

Implement new Hadoop file system APIs to support HBase.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8436 Support setSafeMode(), isFileClosed() FileSystem API

How was this patch tested?

Unit tests

[szetszwo]

@jojochuang , thanks a lot for working on this! The change looks good. Please see the comments inlined.

[szetszwo]

Currently, we only support FORCE_EXIT. We should support LEAVE later.

[szetszwo]

For LEAVE, it should exit safe mode only if preCheck is true. We should make the change separately.

//SCMSafeModeManager@@ -237,20 +237,25@@publicvoidcompletePreCheck(EventPublishereventQueue) {
* 2.EmitsSTART_REPLICATIONforReplicationManager.
* 3.Cleanupresources.
* 4.Emitsafemodestatus.
- * @parameventQueue
*/
@VisibleForTesting
- publicvoidexitSafeMode(EventPublishereventQueue) {
- LOG.info("SCM exiting safe mode.");
- // If safemode is exiting, then pre check must also have passed so
- // set it to true.
- setPreCheckComplete(true);
+ publicbooleanexitSafeMode(booleanforce) {
+ LOG.info("SCM exiting safe mode, force? {}", force);
+ if (force) {
+ // If force exiting, set preCheck to true.
+ setPreCheckComplete(true);
+ } else {
+ if (!getPreCheckComplete()) {
+ returnfalse;
+ }
+ }
setInSafeMode(false);
// TODO: Remove handler registration as there is no need to listen to// register events anymore.emitSafeModeStatus();
+ returntrue;
}

[szetszwo]

We should check status.isFile().

[szetszwo]

Check status.isFile().

[szetszwo]

Check if the action is not GET before incrementWriteOps.

[szetszwo]

Check if the action is not GET before incrementWriteOps.

[szetszwo]

Check if the action is not GET before incrementWriteOps.

[szetszwo]

@jojochuang , thanks for the update!

We need to check GET before incrementWriteOp in two of RootedOzoneFileSystems. BTW, is it possible to move the setSafeMode method to Basic RootedOzoneFileSystem?

[szetszwo]

Check if the action is not GET before incrementWriteOps.

[jojochuang]

BTW, is it possible to move the setSafeMode method to Basic RootedOzoneFileSystem?

We can't, because Hadoop 2 does not have SafeMode interface (introduced in Hadoop 3.3.6). So I can only put it where Ozone is not run with Hadoop 2 runtime.

[jojochuang]

But as it turns out, we test MR jobs with Hadoop 3.1 3.2 and 3.3 runtime, and the MR job tests fail with Hadoop 3.1 and 3.2 (due to the lacking of SafeMode and LeaseRecoverable interface). So either we find a workaround or remove the Hadoop 3.1 3.2 tests.

[szetszwo]

+1 the change looks good.