HDDS-6091. Add file checksum to OmKeyInfo by jojochuang · Pull Request #3201 · apache/ozone

jojochuang · 2022-03-16T06:01:26Z

What changes were proposed in this pull request?

Persist file checksum in OM rocksdb.

What is the link to the Apache JIRA

How was this patch tested?

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

rakeshadr

Thanks @jojochuang for the patch. Added few comments.

Change-Id: I5912fd35d54796f1269567ea1113103cd1ff740c

Change-Id: Idf93c373e6391cf94dcedc9d44adad11a6ad5b42

Change-Id: I62732916cb93b32028cc0ebddda08c90f5e077c0

Change-Id: I065e7e09f5cf28ccf0ba67d4dc64d685ad4dc918

Change-Id: Iaec84961d9ab68a989e942a8af527efbd62fe328

Change-Id: Ida95a0a970f0312bfc74ee287cc9bae197219f33

Change-Id: Iec1c7bddda23b046c5887ae44178ac090501946f

rakeshadr

+1 LGTM, thanks @jojochuang for the contribution.

Pending one open comment pls take care that. Also, we can see the latest CI build report as the previous report was ran 9 days back.

Change-Id: I1d688e98dde5620c520baf0cd23031fd783ec504

jojochuang · 2022-06-05T19:23:37Z

Thanks for the review. I thought I updated the patch but missed that one. The new patch is up and I'll merge it once the build comes back clean.

* master: (87 commits) HDDS-6686. Do Leadship check before SASL token verification. (apache#3382) HDDS-4364: [FSO]List FileStatus : startKey can be a non-existed path (apache#3481) HDDS-6091. Add file checksum to OmKeyInfo (apache#3201) HDDS-6706. Exposing Volume Information Metrics to the DataNode UI (apache#3478) HDDS-6759: Add listblock API in MockDatanodeStorage (apache#3452) HDDS-5821 Container cache management for closing RockDB (apache#3426) HDDS-6683. Refactor OM server bucket layout configuration usage (apache#3477) HDDS-6824. Revert changes made in proto.lock by HDDS-6768. (apache#3480) HDDS-6811. Bucket create message with layout type (apache#3479) HDDS-6810. Add a optional flag to trigger listStatus as part of listKeys for FSO buckets. (apache#3461) HDDS-6828. Revert RockDB version pending leak fixes (apache#3475) HDDS-6764: EC: DN ability to create RECOVERING containers for EC reconstruction. (apache#3458) HDDS-6795: EC: PipelineStateMap#addPipeline should not have precondition checks post db updates (apache#3453) HDDS-6823. Intermittent failure in TestOzoneECClient#testExcludeOnDNMixed (apache#3476) HDDS-6820. Bucket Layout Post-Finalization Validators for ACL Requests. (apache#3472) HDDS-6819. Add LEGACY to AllowedBucketLayouts in CreateBucketHandler (apache#3473) HDDS-4859. [FSO]ListKeys: seek all the files/dirs from startKey to keyPrefix (apache#3466) HDDS-6705 Add metrics for volume statistics including disk capacity, usage, Reserved (apache#3430) HDDS-6474. Add test to cover the FSO bucket list status with beyond batch boundary and cache. (apache#3379). Contributed by aswinshakil HDDS-6280. Support Container Balancer HA (apache#3423) ...

jojochuang force-pushed the persist_checksum branch 3 times, most recently from 1e47b01 to 4126a88Compare March 23, 2022 08:01

jojochuang marked this pull request as ready for review March 24, 2022 03:17

jojochuang requested a review from rakeshadr March 24, 2022 03:17

rakeshadr reviewed Apr 13, 2022
View reviewed changes

jojochuang added 4 commits April 26, 2022 22:40

HDDS-6091. Add file checksum to OmKeyInfo.
0f2c305
Change-Id: I5912fd35d54796f1269567ea1113103cd1ff740c

Harden serialization/deserialization.
55ed436
Change-Id: Idf93c373e6391cf94dcedc9d44adad11a6ad5b42

Refactor test code.
b24ecb2
Change-Id: I62732916cb93b32028cc0ebddda08c90f5e077c0

Address review comments.
adbd946
Change-Id: I065e7e09f5cf28ccf0ba67d4dc64d685ad4dc918

jojochuang force-pushed the persist_checksum branch from 80ed017 to adbd946Compare May 2, 2022 21:41