Skip to content

HDDS-14687. Add test that OM can finalize from a snapshot - #10832

Merged
errose28 merged 3 commits into
apache:HDDS-14496-zdufrom
errose28:worktree/restore-om-snapshot-test
Jul 23, 2026
Merged

HDDS-14687. Add test that OM can finalize from a snapshot#10832
errose28 merged 3 commits into
apache:HDDS-14496-zdufrom
errose28:worktree/restore-om-snapshot-test

Conversation

@errose28

@errose28errose28 commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Prior to HDDS-14580, TestOMUpgradeFinalization used prepare for upgrade to clear OM logs and check that a follower could finalize from a snapshot. HDDS-14580 removed prepare for upgrade and that test as well. This PR restores testing for OM snapshot finalization based on reference implementations in TestSCMHAFinalization#testSnasphotFinalization and TestOMRatisSnapshots.

What is the link to the Apache JIRA

HDDS-14687

How was this patch tested?

New test should pass.

@errose28errose28 added the zdu Pull requests for Zero Downtime Upgrade (ZDU) https://issues.apache.org/jira/browse/HDDS-14496 label Jul 21, 2026
@errose28
errose28 requested a review from dombizitaJuly 21, 2026 20:10
@errose28errose28 changed the title Add passing snapshot finalization testHDDS-14687. Add test that OM can finalize from a snapshotJul 21, 2026
@errose28errose28 mentioned this pull request Jul 21, 2026
Merged
@errose28

errose28 commented Jul 21, 2026

Copy link
Copy Markdown
ContributorAuthor

The original implementation was flaky because it used the default 15 minute client timeout. If the client connected to the inactive OM first, it would retry for 15 minutes instead of failing over to the leader. This probably needs to be fixed in the client in a separate change.

The second issue was that the test needs to wait for the snapshot install flow to fully complete before shutting down, not just for the snapshot to show up on the follower. Otherwise the OM providing the snapshot gets shut down and the OM installing it blocks its shutdown until it can complete the snapshot install.

@errose28
errose28 marked this pull request as ready for review July 22, 2026 00:09
dombizita
dombizita reviewed Jul 22, 2026
View reviewed changes

@dombizitadombizita left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @errose28, few smaller comments poster.

Comment thread...one/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java Outdated
Comment thread...one/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
Comment thread...one/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java Outdated
@errose28

errose28 commented Jul 22, 2026

Copy link
Copy Markdown
ContributorAuthor

Thanks for the review, all comments are addressed in the latest commit.

dombizita
dombizita approved these changes Jul 22, 2026
View reviewed changes

@dombizitadombizita left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good!

@errose28
errose28 merged commit 2f149e4 into apache:HDDS-14496-zduJul 23, 2026
58 of 59 checks passed
errose28 added a commit to errose28/ozone that referenced this pull request Jul 23, 2026
@errose28
Merge branch 'HDDS-14496-zdu' into worktree/om-finalize-version-valid…
9e77af6
…ation
* HDDS-14496-zdu: (57 commits)
HDDS-14687. Add test that OM can finalize from a snapshot (apache#10832)
HDDS-14580. Remove OM Prepare for Upgrade Code for ZDU (apache#9723)
HDDS-15848. Add new upgrade action annotations for component versions (apache#10744)
HDDS-15876. Remove ozone-csi (apache#10780)
HDDS-14825. Add Grafana Dashboard and Metrics for ZDU (apache#10602)
HDDS-15508. Add 2.2.0 to compatibility/upgrade test (apache#10782)
HDDS-15867. [JDK25] Bump byte-buddy to 1.18.11 (apache#10776)
HDDS-15809. Validate x-amz-copy-source-range on UploadPartCopy (apache#10711).
HDDS-15678. OFS isDirectory/isFile should not trigger pipeline refresh or return block locations (apache#10741)
HDDS-15171. Add available space check on follower during bootstrap. (apache#10185)
HDDS-14043. Fix ls -e UnsupportedOperationException on ofs/o3fs (apache#10209)
HDDS-15709. Replace random name suffix with uniqueObjectName (apache#10773)
HDDS-15849. Force channel shutdown in XceiverClientGrpc.close (apache#10747)
HDDS-15765. Fail fast when --node-id is omitted for OM compact/defrag on an HA cluster (apache#10699)
HDDS-14989. Delay follower SCM DN server start until Ratis log catch-up. (apache#10617).
HDDS-15456. Add SCM DB lookup and identify orphan(wrt SCM) and deleted-but-present containers (apache#10547)
HDDS-15812. Rename test utils for naming convention (apache#10766)
HDDS-15600. Fix ListObjects response for encoding-type and empty delimiter (apache#10586).
HDDS-15656. Fix non-atomic data size accumulation in ContainerBalancer move callback (apache#10708)
HDDS-15667. `containerbalancer status`: show stop reason and iteration details after stop (apache#10681)
...
Conflicts:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
errose28 added a commit to errose28/ozone that referenced this pull request Jul 31, 2026
@errose28
Merge branch 'HDDS-14496-zdu' into worktree/scm-finalize-version-vali…
333d306
…dation
* HDDS-14496-zdu: (62 commits)
HDDS-15875. OM should validate software versions of peers before accepting finalize command (apache#10777)
HDDS-15417. Handle mixed datanode versions on the replication path (apache#10570)
HDDS-15716. Replace layout version protos with new version-named protos in the DN–SCM protocol (apache#10817)
HDDS-15715. Add helpful CLI flags to finalization CLIs (apache#10678)
HDDS-14687. Add test that OM can finalize from a snapshot (apache#10832)
HDDS-14580. Remove OM Prepare for Upgrade Code for ZDU (apache#9723)
HDDS-15848. Add new upgrade action annotations for component versions (apache#10744)
HDDS-15876. Remove ozone-csi (apache#10780)
HDDS-14825. Add Grafana Dashboard and Metrics for ZDU (apache#10602)
HDDS-15508. Add 2.2.0 to compatibility/upgrade test (apache#10782)
HDDS-15867. [JDK25] Bump byte-buddy to 1.18.11 (apache#10776)
HDDS-15809. Validate x-amz-copy-source-range on UploadPartCopy (apache#10711).
HDDS-15678. OFS isDirectory/isFile should not trigger pipeline refresh or return block locations (apache#10741)
HDDS-15171. Add available space check on follower during bootstrap. (apache#10185)
HDDS-14043. Fix ls -e UnsupportedOperationException on ofs/o3fs (apache#10209)
HDDS-15709. Replace random name suffix with uniqueObjectName (apache#10773)
HDDS-15677. Update client side version checks to use `isSupportedBy`. (apache#10611)
HDDS-15849. Force channel shutdown in XceiverClientGrpc.close (apache#10747)
HDDS-15765. Fail fast when --node-id is omitted for OM compact/defrag on an HA cluster (apache#10699)
HDDS-14989. Delay follower SCM DN server start until Ratis log catch-up. (apache#10617).
...
Conflicts:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManager.java
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dombizitadombizitadombizita approved these changes

Assignees

No one assigned

Labels

zduPull requests for Zero Downtime Upgrade (ZDU) https://issues.apache.org/jira/browse/HDDS-14496

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@errose28@dombizita