HDDS-14871. DataNode: tolerate per-volume health-check latch timeouts before marking volumes failed.

[devmadhuu]

What changes were proposed in this pull request?

This PR addresses the problem of latch timeout for pending volumes not reported any result.

StorageVolumeChecker.checkAllVolumes() waits on a single CountDownLatch for all volume health checks to complete. If the latch expires before any volume finishes — due to any transient stall — every pending volume is immediately marked FAILED with zero tolerance, producing false-positive volume failures.

The existing per-volume IO-failure sliding window in StorageVolume.check() does not address this because it only applies when a check completes, not when the latch times out.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14871

How was this patch tested?

This patch has been tested by extending 3 unit tests in existing test class : TestStorageVolumeHealthChecks

[ptlrs]

Thanks for the PR @devmadhuu

[devmadhuu]

@ptlrs Thanks for your review. So lets wait for your #8843 to get merged, so that I can revisit this PR and do the changes accordingly.

[priyeshkaratha]

Thanks @devmadhuu for the patch. Changes LGTM

[ptlrs]

Hi @devmadhuu, #8843 has been merged. Could you please revisit this PR?

[ptlrs]

Thanks for the update @devmadhuu. I have some comments.

[ptlrs]

Thank you for the updates @devmadhuu. I just have one concern about the callback. Otherwise, the code mostly looks good to me.

[jojochuang]

hi thanks a lot for the PR. I see that it has several rounds of through reviews and it's been a month. If there are no major concerns we should merge it, and fix remaining minor issues in a follow-on task.

[ptlrs]

Thanks for the changes @devmadhuu. LGTM

[jojochuang]

Merged. Thanks @devmadhuu@ptlrs@ChenSammi@priyeshkaratha