Skip to content

HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. - #4519

Merged
dombizita merged 1 commit into
apache:masterfrom
smitajoshi12:HDDS-8339
May 15, 2023
Merged

HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI.#4519
dombizita merged 1 commit into
apache:masterfrom
smitajoshi12:HDDS-8339

Conversation

@smitajoshi12

@smitajoshi12smitajoshi12 commented Apr 3, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This information will be displayed on the overview page of recon which gets populated with the information provided by the ClusterStateEndpoint This information can be useful for auditing, compliance, and troubleshooting purposes. Hence in short the main purpose is to get a quick idea of the deletion progress.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8339

How was this patch tested?

Manually

image

@smitajoshi12smitajoshi12 changed the title HDDS-8399. Show the number of keys marked for Deletion in Recon UIHDDS-8399. Recon Show the number of keys marked for Deletion in Recon UI.Apr 3, 2023
@ArafatKhan2198

ArafatKhan2198 commented Apr 3, 2023

Copy link
Copy Markdown
Contributor

@smitajoshi12 Instead of naming the tile "Number of Deleted Keys," let's call it "Pending key Deletions" as this number represents the keys marked for deletion. They will be deleted eventually by the KeyDeletionService task running in the background for SCM.

@ArafatKhan2198

ArafatKhan2198 commented Apr 3, 2023

Copy link
Copy Markdown
Contributor

@smitajoshi12, please position the "Deleted Keys" tile after the "Keys" tile, allowing users to easily compare both numbers simultaneously.

@kerneltime

kerneltime commented Apr 3, 2023

Copy link
Copy Markdown
Contributor

@devmadhuu can you please take a look?

dombizita
dombizita reviewed Apr 4, 2023
View reviewed changes

@dombizitadombizita left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this @smitajoshi12! I left a few comments inline, please take a look

Comment threadhadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/api/db.json Outdated
Comment threadhadoop-ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/api/db.json Outdated
Comment thread...ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/overview/overview.tsx Outdated
devmadhuu
devmadhuu reviewed Apr 4, 2023
View reviewed changes
Comment thread...ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/overview/overview.tsx Outdated
Comment thread...ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/overview/overview.tsx Outdated
Comment thread...ozone/recon/src/main/resources/webapps/recon/ozone-recon-web/src/views/overview/overview.tsx Outdated
@smitajoshi12

smitajoshi12 commented Apr 5, 2023

Copy link
Copy Markdown
ContributorAuthor

@smitajoshi12, please position the "Deleted Keys" tile after the "Keys" tile, allowing users to easily compare both numbers simultaneously.

Done

@ArafatKhan2198ArafatKhan2198 mentioned this pull request Apr 12, 2023
Merged
@smitajoshi12smitajoshi12 changed the title HDDS-8399. Recon Show the number of keys marked for Deletion in Recon UI.HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI.Apr 14, 2023
@dombizita

dombizita commented Apr 14, 2023

Copy link
Copy Markdown
Contributor

thanks for updating your patch @smitajoshi12, now for me with this check it is not showing 0 in case there is no key/bucket/pending delete key. I attached a screenshot (I did a docker cluster locally with your changes), please check it.

recon

@smitajoshi12

smitajoshi12 commented Apr 14, 2023•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@dombizita
Thanks Zita completed changes suggested by you currently there are no changes integrated for pendingDeleteKeys in clusterState API so coming empty buckets.

image

dombizita
dombizita approved these changes Apr 28, 2023
View reviewed changes

@dombizitadombizita left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for working on this @smitajoshi12, it looks good to me!

@dombizita
dombizita requested a review from smengclApril 28, 2023 11:02
@smitajoshi12
HDDS-8339. Recon Show the number of keys marked for Deletion in Recon…
bdcc877
… UI. Revert Back keysPendingDeletion key chnages
smengcl
smengcl approved these changes May 5, 2023
View reviewed changes

@smengclsmengcl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 pending CI

@ArafatKhan2198

ArafatKhan2198 commented May 9, 2023

Copy link
Copy Markdown
Contributor

@smitajoshi12 can you please check the UT's that are failing.

@smitajoshi12

smitajoshi12 commented May 9, 2023

Copy link
Copy Markdown
ContributorAuthor

@smitajoshi12 can you please check the UT's that are failing.

@dombizita@smengcl@devmadhuu
Hi Zita , In Local build issuccessful can you rerun

@dombizita
dombizita merged commit 257a64f into apache:masterMay 15, 2023
@dombizita

dombizita commented May 15, 2023

Copy link
Copy Markdown
Contributor

thanks for working on this patch @smitajoshi12! thanks for the review @ArafatKhan2198, @devmadhuu and @smengcl!

errose28 added a commit to errose28/ozone that referenced this pull request May 17, 2023
@errose28
Merge branch 'master' into HDDS-7366
fb44948
* master: (78 commits)
HDDS-8575. Intermittent failure in TestCloseContainerEventHandler.testCloseContainerWithDelayByLeaseManager (apache#4688)
HDDS-7241. EC: Reconstruction could fail with orphan blocks. (apache#4718)
HDDS-8577. [Snapshot] Disable compaction log when loading metadata for snapshot (apache#4697)
HDDS-7080. EC: Offline reconstruction needs better logging (apache#4719)
HDDS-8626. Config thread pool in ReplicationServer (apache#4715)
HDDS-8616. Underreplication not fixed if all replicas start decommissioning (apache#4711)
HDDS-8254. Close containers when volume reaches utilisation threshold (apache#4583)
HDDS-8254. Close containers when volume reaches utilisation threshold (apache#4583)
HDDS-8615. Explicitly show EC block type in 'ozone debug chunkinfo' command output (apache#4706)
HDDS-8623. Delete duplicate getBucketInfo in OMKeyCommitRequest (apache#4712)
HDDS-8339. Recon Show the number of keys marked for Deletion in Recon UI. (apache#4519)
HDDS-8572. Support CodecBuffer for protobuf v3 codecs. (apache#4693)
HDDS-8010. Improve DN warning message when getBlock does not find the block. (apache#4698)
HDDS-8621. IOException is never thrown in SCMRatisServer.getRatisRoles(). (apache#4710)
HDDS-8463. S3 key uniqueness in deletedTable (apache#4660)
HDDS-8584. Hadoop client write slowly when stream enabled (apache#4703)
HDDS-7732. EC: Verify block deletion from missing EC containers (apache#4705)
HDDS-8581. Avoid random ports in integration tests (apache#4699)
HDDS-8504. ReplicationManager: Pass used and excluded node separately for Under and Mis-Replication (apache#4694)
HDDS-8576. Close RocksDB instance in RDBStore if RDBStore's initialization fails after RocksDB instance creation (apache#4692)
...
@smitajoshi12
smitajoshi12 deleted the HDDS-8339 branch May 22, 2023 10:40
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devmadhuudevmadhuudevmadhuu left review comments
@smengclsmengclsmengcl approved these changes
@dombizitadombizitadombizita approved these changes
@ArafatKhan2198ArafatKhan2198ArafatKhan2198 approved these changes

Assignees

No one assigned

Labels

recon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@smitajoshi12@ArafatKhan2198@kerneltime@dombizita@devmadhuu@smengcl@adoroszlai