Skip to content

HDDS-6705 Add metrics for volume statistics including disk capacity, usage, Reserved - #3430

Merged
jojochuang merged 14 commits into
apache:masterfrom
ArafatKhan2198:HDDS-6687
Jun 1, 2022
Merged

HDDS-6705 Add metrics for volume statistics including disk capacity, usage, Reserved#3430
jojochuang merged 14 commits into
apache:masterfrom
ArafatKhan2198:HDDS-6687

Conversation

@ArafatKhan2198

@ArafatKhan2198ArafatKhan2198 commented May 19, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add metrics for volume statistics including disk capacity, usage, type (SSD/HDD).

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6705

How was this patch tested?

The Metrics have been exposed onto the JMX of the DataNode

 {
"name": "Hadoop:service=HddsDatanode,name=VolumeInfoMetrics-/tmp/hadoop-mohammad.khan/dfs/data",
"modelerType": "VolumeInfoMetrics-/tmp/hadoop-mohammad.khan/dfs/data",
"tag.Context": "ozone",
"tag.VolumeType": "DATA_VOLUME",
"tag.StorageDirectory": "/tmp/hadoop-mohammad.khan/dfs/data/hdds",
"tag.DatanodeUuid": "420d34ce-ac6c-4841-a4ab-86a9836bdb3f",
"tag.StorageType": "DISK",
"tag.Hostname": "Mohammads-MacBook-Pro.local",
"TotalCapacity": 292296314880,
"Used": 8192,
"LayoutVersion": 1,
"Reserved": 0,
"Available": 292296306688,
"Capacity": 292296314880
}

jojochuang
jojochuang reviewed May 19, 2022
View reviewed changes
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
@ArafatKhan2198
ArafatKhan2198 marked this pull request as ready for review May 26, 2022 05:59
@ArafatKhan2198

ArafatKhan2198 commented May 26, 2022

Copy link
Copy Markdown
ContributorAuthor

@adoroszlai@JyotinderSingh@siddhantsangwan@jojochuang@aryangupta1998
Can you please help review this !!

jojochuang
jojochuang requested changes May 26, 2022
View reviewed changes
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
Comment thread...r-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/VolumeInfoStats.java Outdated
jojochuang
jojochuang approved these changes Jun 1, 2022
View reviewed changes
Comment thread...tainer-service/src/main/java/org/apache/hadoop/ozone/container/common/volume/HddsVolume.java
@jojochuang
jojochuang merged commit 24ca66b into apache:masterJun 1, 2022
errose28 added a commit to errose28/ozone that referenced this pull request Jun 7, 2022
@errose28
Merge branch 'master' into HDDS-6760
049d7bc
* master: (87 commits)
HDDS-6686. Do Leadship check before SASL token verification. (apache#3382)
HDDS-4364: [FSO]List FileStatus : startKey can be a non-existed path (apache#3481)
HDDS-6091. Add file checksum to OmKeyInfo (apache#3201)
HDDS-6706. Exposing Volume Information Metrics to the DataNode UI (apache#3478)
HDDS-6759: Add listblock API in MockDatanodeStorage (apache#3452)
HDDS-5821 Container cache management for closing RockDB (apache#3426)
HDDS-6683. Refactor OM server bucket layout configuration usage (apache#3477)
HDDS-6824. Revert changes made in proto.lock by HDDS-6768. (apache#3480)
HDDS-6811. Bucket create message with layout type (apache#3479)
HDDS-6810. Add a optional flag to trigger listStatus as part of listKeys for FSO buckets. (apache#3461)
HDDS-6828. Revert RockDB version pending leak fixes (apache#3475)
HDDS-6764: EC: DN ability to create RECOVERING containers for EC reconstruction. (apache#3458)
HDDS-6795: EC: PipelineStateMap#addPipeline should not have precondition checks post db updates (apache#3453)
HDDS-6823. Intermittent failure in TestOzoneECClient#testExcludeOnDNMixed (apache#3476)
HDDS-6820. Bucket Layout Post-Finalization Validators for ACL Requests. (apache#3472)
HDDS-6819. Add LEGACY to AllowedBucketLayouts in CreateBucketHandler (apache#3473)
HDDS-4859. [FSO]ListKeys: seek all the files/dirs from startKey to keyPrefix (apache#3466)
HDDS-6705 Add metrics for volume statistics including disk capacity, usage, Reserved (apache#3430)
HDDS-6474. Add test to cover the FSO bucket list status with beyond batch boundary and cache. (apache#3379). Contributed by aswinshakil
HDDS-6280. Support Container Balancer HA (apache#3423)
...
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kerneltimekerneltimekerneltime left review comments
@jojochuangjojochuangjojochuang approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ArafatKhan2198@kerneltime@jojochuang