Skip to content

HDDS-5666. Add option to createKey via streaming api in Freon - #2574

Merged
szetszwo merged 4 commits into
apache:HDDS-4454from
sadanand48:HDDS-5666
Apr 8, 2022
Merged

HDDS-5666. Add option to createKey via streaming api in Freon#2574
szetszwo merged 4 commits into
apache:HDDS-4454from
sadanand48:HDDS-5666

Conversation

@sadanand48

@sadanand48sadanand48 commented Aug 25, 2021

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add an option to createKey via streaming api in Freon via OzoneClientKeyGenerator.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5666

How was this patch tested?

Tested the command via docker

@sadanand48
sadanand48 changed the base branch from master to HDDS-4454August 25, 2021 07:08
@sadanand48sadanand48 changed the title HDDS-5666. Add option to createKey via streaming api in RandomKeyGeneratorHDDS-5666. Add option to createKey via streaming api via FreonOct 12, 2021
@sadanand48sadanand48 changed the title HDDS-5666. Add option to createKey via streaming api via FreonHDDS-5666. Add option to createKey via streaming api in FreonOct 12, 2021
@szetszwo
szetszwoforce-pushed the HDDS-4454 branch 2 times, most recently from f637b1e to ce1d880CompareFebruary 16, 2022 14:47

@captainzmccaptainzmc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sadanand48 work on this. I left few comments.

@captainzmccaptainzmcFeb 28, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line will conflict with the patch of EC. EC is currently voting on merge to master, so EC will merge into master soon. So can we continue this PR after EC merge?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sadanand48 , let's hardcode replicationConfig in createStreamKey(..) for the moment. We may change it later if there is a need.

 private void createStreamKey(long counter) throws Exception {
final ReplicationConfig replicationConfig = ReplicationConfig
.fromProtoTypeAndFactor(HddsProtos.ReplicationType.RATIS,
HddsProtos.ReplicationFactor.THREE);
final String key = generateObjectName(counter);
timer.time(() -> {
try (OzoneDataStreamOutput stream = bucket
.createStreamKey(key, keySize, replicationConfig, metadata)) {
contentGenerator.write(stream);
}
return null;
});
}

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@szetszwo
szetszwoforce-pushed the HDDS-4454 branch 2 times, most recently from b6f1921 to f826d12CompareMarch 16, 2022 17:30
@szetszwo

Copy link
Copy Markdown
Contributor

@sadanand48 , there are conflicts. Could could you update the change?

(cherry picked from commit f623bec992e80eda82e6155a3b20c476e683a3d6)

@szetszwoszetszwo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.

@szetszwo
szetszwo merged commit 1069b64 into apache:HDDS-4454Apr 8, 2022
szetszwo pushed a commit to szetszwo/ozone that referenced this pull request May 6, 2022
captainzmc pushed a commit to captainzmc/hadoop-ozone that referenced this pull request Jul 4, 2022
szetszwo pushed a commit that referenced this pull request Oct 25, 2022
(cherry picked from commit 51b5395)
(cherry picked from commit a2501c3de3052a6bfc9729a73fbb99453dbfdd72)
szetszwo pushed a commit that referenced this pull request Nov 7, 2022
(cherry picked from commit 51b5395)
(cherry picked from commit a2501c3de3052a6bfc9729a73fbb99453dbfdd72)
(cherry picked from commit 4a5b683)
szetszwo pushed a commit that referenced this pull request Dec 1, 2022
szetszwo pushed a commit that referenced this pull request Dec 16, 2022
nishitpatira pushed a commit to nishitpatira/ozone that referenced this pull request Dec 16, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@sadanand48@szetszwo@captainzmc