HDDS-13095. Support sorting by most/least used nodes in ozone admin datanode list

[sreejasahithi]

What changes were proposed in this pull request?

This change introduces new CLI options --most-used and --least-used under the ozone admin datanode list command to enable sorting datanodes based on their usage. When these options are used, the output includes additional details such as used space, total capacity, and percentage of space used.
Unit tests for this change are added in TestListInfoSubcommand.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13095

How was this patch tested?

sample outputs when --most-used flag is used:

[ {
"datanodeDetails" : {
"level" : 3,
"cost" : 0,
"id" : {
"uuid" : "6173fced-573b-4850-93b5-d799efc48fbf",
"id" : "6173fced-573b-4850-93b5-d799efc48fbf",
"byteString" : {
"validUtf8" : true,
"empty" : false
}
},
"ipAddress" : "X.X.X.X",
"hostName" : "ozone-datanode-3.ozone_default",
.
.
.
"healthState" : "HEALTHY",
"used" : 400,
"capacity" : 1000,
"percentUsed" : 40.0
"opState" : "IN_SERVICE"
}, {
"datanodeDetails" : {
"level" : 3,
"cost" : 0,
"id" : {
"uuid" : "c2095dc5-d75c-4156-99b2-2da09c7dc32a",
"id" : "c2095dc5-d75c-4156-99b2-2da09c7dc32a",
"byteString" : {
"validUtf8" : true,
"empty" : false
}
},
"ipAddress" : "Y.Y.Y.Y",
"hostName" : "ozone-datanode-2.ozone_default",
.
.
.
"healthState" : "HEALTHY",
"used" : 300,
"capacity" : 1000,
"percentUsed" : 30.0,
"opState" : "IN_SERVICE"
}, {
"datanodeDetails" : {
"level" : 3,
"cost" : 0,
"id" : {
"uuid" : "beb0322f-63e0-4cfc-9c21-72fd48b68440",
"id" : "beb0322f-63e0-4cfc-9c21-72fd48b68440",
"byteString" : {
"validUtf8" : true,
"empty" : false
}
},
"ipAddress" : "Z.Z.Z.Z",
"hostName" : "ozone-datanode-1.ozone_default",
.
.
.
"healthState" : "HEALTHY",
"used" : 200,
"capacity" : 1000,
"percentUsed" : 20.0,
"opState" : "IN_SERVICE"
} ]

When json flag is not given:

Datanode: 6173fced-573b-4850-93b5-d799efc48fbf (/default-rack/X.X.X.X/ozone-datanode-3.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
7aeba473-a20e-4c49-a79e-f0c871222d28/RATIS/THREE/RATIS/OPEN/Follower
21e77351-5200-42e7-8d38-acbc9b4428e9/RATIS/ONE/RATIS/OPEN/Leader
Used: 400
Capacity: 1000
Datanode: c2095dc5-d75c-4156-99b2-2da09c7dc32a (/default-rack/Y.Y.Y.Y/ozone-datanode-2.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
7aeba473-a20e-4c49-a79e-f0c871222d28/RATIS/THREE/RATIS/OPEN/Leader
5c96425b-002f-4ed3-9fae-5433a46eb1ba/RATIS/ONE/RATIS/OPEN/Leader
Used: 300
Capacity: 1000
Datanode: beb0322f-63e0-4cfc-9c21-72fd48b68440 (/default-rack/Z.Z.Z.Z/ozone-datanode-1.ozone_default/2 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Related pipelines:
7aeba473-a20e-4c49-a79e-f0c871222d28/RATIS/THREE/RATIS/OPEN/Follower
225e4b47-fb29-46a9-90ae-5949ef5a0597/RATIS/ONE/RATIS/OPEN/Leader
Used: 200
Capacity: 1000

https://github.com/sreejasahithi/ozone/actions/runs/15297558597

[sreejasahithi]

This might conflict with #8523, let's wait until that one is merged.

[siddhantsangwan]

Trying to understand the use case here. Are we trying to make this command have usage information so users don't need to use two different commands, list and usageinfo?

The jira isn't clear to me.

[sreejasahithi]

Trying to understand the use case here. Are we trying to make this command have usage information so users don't need to use two different commands, list and usageinfo?

The jira isn't clear to me.

CLI options --most-used and --least-used are added to the ozone admin datanode list command, so that a user will be able to get the list of nodes in either descending or ascending order of their usage. The json output of this command when used with the CLI options mentioned earlier includes additional information such as used , capacity and percentused for each node listed and we can use jq's sort_by function for advanced sorting.

We are not trying to remove usageinfo command.

[siddhantsangwan]

CLI options --most-used and --least-used are added to the ozone admin datanode list command, so that a user will be able to get the list of nodes in either descending or ascending order of their usage. The json output of this command when used with the CLI options mentioned earlier includes additional information such as used , capacity and percentused for each node listed and we can use jq's sort_by function for advanced sorting.

We are not trying to remove usageinfo command.

usageinfo can already list out nodes in descending/ascending order. I don't know if it has a json option though.

[sreejasahithi]

usageinfo can already list out nodes in descending/ascending order. I don't know if it has a json option though.

yes your right usageinfo does list descending/ascending order but usageinfo command is used when we want more specific information.
But adding the CLI options --most-used and --least-used to ozone admin datanode list is to just get the datanodes listed in a particular order based on their usage and we are not providing any detailed information as in usageinfo command.

@errose28 should we keep CLI options --most-used and --least-used only in list command and remove from usageinfo command or keep it in both since this option is just about ordering?

[errose28]

@siddhantsangwan we are trying to move our datanode CLI to the same list/info command pattern that is used elsewhere in the system, where list is a command that shows a brief summary of all entities, and info shows verbose information about a single entity. I've added a comment on the parent datanode CLI jira to explain this better.

In context of this change, any options that deal with sorting a list should be under the list command. We can leave the sorting options in usageinfo for now since they don't cause any harm. Going forward I do hope to have a dedicated ozone admin datanode info command that will be a superset of what usageinfo provides so we can hide that command from the CLI and tag it as deprecated.

[sreejasahithi]

This PR can be reviewed first since #8523 is under discussion.

[Tejaskriya]

Thanks for the patch @sreejasahithi , overall looks good, just a suggestion below for the tests

[aryangupta1998]

Thanks for the patch, @sreejasahithi. It looks good, I have some minor comments inline.

[aryangupta1998]

Thanks for addressing the comments @sreejasahithi, LGTM!

[Tejaskriya]

Thanks for the patch @sreejasahithi , and the reviews @aryangupta1998@siddhantsangwan@errose28