Skip to content

HDDS-9802. Tool to fix corrupted snapshot chain - #6386

Merged
hemantk-12 merged 3 commits into
apache:masterfrom
hemantk-12:HDDS-9802
Mar 19, 2024
Merged

HDDS-9802. Tool to fix corrupted snapshot chain#6386
hemantk-12 merged 3 commits into
apache:masterfrom
hemantk-12:HDDS-9802

Conversation

@hemantk-12

@hemantk-12hemantk-12 commented Mar 16, 2024

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

In the past, we have seen many instances (HDDS-8530, HDDS-8832, and recently HDDS-10524) when SnapshotChain corruption was causing OM to get stuck on restart. Lots of bugs related to snapshot chain corruption have been fixed but we still issue some occurrences here and there (HDDS-10524 is a very recent one).
This change is to implement a tool that can be used to repair the snapshot chain corruption once a corrupted entry is identified.
The idea for this tool is from HDDS-8100. This change also provides a base setup for HDDS-8100 and can be extended to provide support for other cases HDDS-8101, HDDS-8824, HDDS-10295, etc.

What is the link to the Apache JIRA

HDDS-9802

How was this patch tested?

Tested it on the docker.

@hemantk-12hemantk-12 changed the title HDDS-9802: Tool to fix corrupted snapshot chainHDDS-9802. Tool to fix corrupted snapshot chainMar 16, 2024
adoroszlai
adoroszlai reviewed Mar 16, 2024
View reviewed changes

@adoroszlaiadoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hemantk-12 for the patch. Found some typos, and suggested some minor improvements.

Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
Comment threadhadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/SnapshotInfo.java Outdated
Comment threadhadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/repair/om/SnapshotRepair.java Outdated
aswinshakil
aswinshakil approved these changes Mar 19, 2024
View reviewed changes

@aswinshakilaswinshakil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this tool @hemantk-12. Really helpful in fixing snapshot chain corruption. LGTM+1

@hemantk-12
hemantk-12 merged commit e907316 into apache:masterMar 19, 2024
@hemantk-12

hemantk-12 commented Mar 19, 2024

Copy link
Copy Markdown
ContributorAuthor

Thanks, @adoroszlai and @aswinshakil for the review.

xichen01 pushed a commit to xichen01/ozone that referenced this pull request Apr 17, 2024
@hemantk-12@xichen01
HDDS-9802. Tool to fix corrupted snapshot chain (apache#6386)
b8a01fb
(cherry picked from commit e907316)
xichen01 pushed a commit to xichen01/ozone that referenced this pull request Apr 18, 2024
@hemantk-12@xichen01
HDDS-9802. Tool to fix corrupted snapshot chain (apache#6386)
9a2bde1
(cherry picked from commit e907316)
@xichen01xichen01 mentioned this pull request Apr 18, 2024
Merged
@DaveTeng0DaveTeng0 mentioned this pull request Apr 22, 2024
Merged
xichen01 pushed a commit to xichen01/ozone that referenced this pull request Apr 23, 2024
@hemantk-12@xichen01
HDDS-9802. Tool to fix corrupted snapshot chain (apache#6386)
2f5db3a
(cherry picked from commit e907316)
swamirishi pushed a commit to swamirishi/ozone that referenced this pull request Jun 10, 2024
@hemantk-12@swamirishi
CDPD-64153. HDDS-9802. Tool to fix corrupted snapshot chain (apache#6386
b7b0cf2
)
(cherry picked from commit e907316)
Change-Id: I67950ede21d47449ce433b629e1a8f24a1d88a19
@hemantk-12
hemantk-12 deleted the HDDS-9802 branch October 28, 2024 18:40
@ivandika3ivandika3 added snapshot https://issues.apache.org/jira/browse/HDDS-6517 tools Tools that helps with debugging labels Jul 10, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adoroszlaiadoroszlaiadoroszlai left review comments
@aswinshakilaswinshakilaswinshakil approved these changes
@errose28errose28Awaiting requested review from errose28

Assignees

No one assigned

Labels

snapshothttps://issues.apache.org/jira/browse/HDDS-6517toolsTools that helps with debugging

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hemantk-12@adoroszlai@aswinshakil@ivandika3