Skip to content

HDDS-2035 Implement datanode level CLI to reveal pipeline relation. - #348

Merged
ChenSammi merged 2 commits into
apache:HDDS-1564from
timmylicheng:HDDS-2035
Dec 18, 2019
Merged

HDDS-2035 Implement datanode level CLI to reveal pipeline relation.#348
ChenSammi merged 2 commits into
apache:HDDS-1564from
timmylicheng:HDDS-2035

Conversation

@timmylicheng

@timmylichengtimmylicheng commented Dec 12, 2019

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

#HDDS-2035 Implement datanode level CLI to reveal pipeline info

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-2035

(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

Please replace this section with the link to the Apache JIRA)

How was this patch tested?

bash-4.2$ ozone scmcli datanode list
Datanode: 7a68a668-3dbe-4366-9fed-271f221c1db0 (172.20.0.2/ozone_datanode_1.ozone_default).
Related pipelines:
cd060313-b724-46cf-a83a-8764ba066ae2/ONE/RATIS/OPEN

(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

adoroszlai
adoroszlai reviewed Dec 13, 2019
View reviewed changes

@adoroszlaiadoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for working on this.

Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
@timmylicheng
timmylichengforce-pushed the HDDS-2035 branch 3 times, most recently from 501b642 to 340f5e8CompareDecember 17, 2019 03:44
@timmylicheng

timmylicheng commented Dec 17, 2019

Copy link
Copy Markdown
ContributorAuthor

testCloseContainerEventWithRatis is fixed in #291

@ChenSammi

ChenSammi commented Dec 17, 2019

Copy link
Copy Markdown
Contributor

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

adoroszlai
adoroszlai requested changes Dec 17, 2019
View reviewed changes

@adoroszlaiadoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
Comment threadhadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/ListInfoSubcommand.java Outdated
@timmylicheng

timmylicheng commented Dec 17, 2019

Copy link
Copy Markdown
ContributorAuthor

Stream.filter creates a new stream, which needs to be assigned to the existing stream to let the final forEach see the result of filtering.

@adoroszlai I have made the changes you requested.

@timmylicheng

timmylicheng commented Dec 17, 2019•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Command line filter doesn't take effect.

bash-4.2$ ozone scmcli datanode list --id=aaaa

Datanode: a37d1f73-7b4c-459a-a119-cd33c0fd64da (172.18.0.4/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
a1065d21-77a5-4265-80db-d341ac7c3da0/ONE/RATIS/OPEN/Leader

bash-4.2$ ozone scmcli datanode list
Datanode: 63b02976-3fee-48dc-9f21-b42d7b6e4a40 (172.31.0.3/ozone_datanode_1.ozone_default/1 pipelines).
Related pipelines:
c8455063-62cc-4bb2-a63c-50a9d59a3adf/ONE/RATIS/ALLOCATED/Follower

bash-4.2$ ozone scmcli datanode list --ip 1.1.1.1

@ChenSammi Just updated.

adoroszlai
adoroszlai approved these changes Dec 17, 2019
View reviewed changes

@adoroszlaiadoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @timmylicheng for updating the patch. Tried it locally applied to master, works fine.

@ChenSammi

ChenSammi commented Dec 18, 2019

Copy link
Copy Markdown
Contributor

Thanks @timmylicheng for the contribution. Committed to branch.

@ChenSammi
ChenSammi merged commit e7315e9 into apache:HDDS-1564Dec 18, 2019
@timmylicheng
timmylicheng deleted the HDDS-2035 branch December 19, 2019 02:25
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Dec 23, 2019
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
8c07782
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Jan 7, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
3796154
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 10, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
81ed727
…pache#348)
timmylicheng added a commit to timmylicheng/hadoop-ozone that referenced this pull request Feb 12, 2020
@timmylicheng
HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (a…
e720e7a
…pache#348)
anuengineer pushed a commit that referenced this pull request Feb 19, 2020
@timmylicheng@ChenSammi@xiaoyuyao
HDDS-1564Multiraft support (#538)
4ba1932
* HDDS-1577. Add default pipeline placement policy implementation. (#1366)
(cherry picked from commit b640a5f6d53830aee4b9c2a7d17bf57c987962cd)
* HDDS-1571. Create an interface for pipeline placement policy to support network topologies. (#1395)
(cherry picked from commit 753fc6703a39154ed6013e44dbae572391748906)
* HDDS-2089: Add createPipeline CLI. (#1418)
(cherry picked from commit 326b5acd4a63fe46821919322867f5daff30750c)
* HDDS-1569 Support creating multiple pipelines with same datanode. Contributed by Li Cheng. This closes#28
* HDDS-1572 Implement a Pipeline scrubber to clean up non-OPEN pipeline. (#237)
* Rebase Fix
* HDDS-2650 Fix createPipeline CLI. (#340)
* HDDS-2035 Implement datanode level CLI to reveal pipeline relation. (#348)
* Revert "HDDS-2650 Fix createPipeline CLI. (#340)"
This reverts commit 7c71710.
* HDDS-2650 Fix createPipeline CLI and make it message based. (#370)
* HDDS-1574 Average out pipeline allocation on datanodes and add metrcs/test (#291)
* Resolve rebase conflict.
* HDDS-2756. Handle pipeline creation failure in different way when it exceeds pipeline limit
Closes#401
* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI (#375)
* HDDS-2115 Add acceptance test for createPipeline CLI and datanode list CLI.
* HDDS-2772 Better management for pipeline creation limitation. (#410)
* HDDS-2913 Update config names and CLI for multi-raft feature. (#462)
* HDDS-2924. Fix Pipeline#nodeIdsHash collision issue. (#478)
* HDDS-2923 Add fall-back protection for rack awareness in pipeline creation. (#516)
* HDDS-3007 Fix CI test failure for TestSCMNodeManager. (#550)
Co-authored-by: Sammi Chen <sammichen@apache.org>
Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adoroszlaiadoroszlaiadoroszlai approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@timmylicheng@ChenSammi@adoroszlai