Skip to content

HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760

Merged
szetszwo merged 5 commits into
apache:HDDS-4454from
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged

HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from
guohao-rosicky:ozone-client-multipart-stream-key

Conversation

@guohao-rosicky

@guohao-rosickyguohao-rosicky commented Oct 22, 2021•
edited by szetszwo
Loading

Copy link
Copy Markdown
Contributor

https://issues.apache.org/jira/browse/HDDS-5879

OzoneBucket and RpcClient not support multipart upload with Stream.
Add the createMultipartStreamKey method

@captainzmc

captainzmc commented Oct 22, 2021

Copy link
Copy Markdown
Member

Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.

@guohao-rosicky
guohao-rosickyforce-pushed the ozone-client-multipart-stream-key branch from f7ea63a to 46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky

guohao-rosicky commented Oct 22, 2021

Copy link
Copy Markdown
ContributorAuthor

jira url: https://issues.apache.org/jira/browse/HDDS-5879

@guohao-rosicky

guohao-rosicky commented Oct 22, 2021

Copy link
Copy Markdown
ContributorAuthor

Thanks @captainzmc , has been modified.

bharatviswa504
bharatviswa504 reviewed Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO

@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892

@captainzmc
captainzmc self-requested a review October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed the ozone-client-multipart-stream-key branch from 46f4fa4 to dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky

guohao-rosicky commented Oct 25, 2021

Copy link
Copy Markdown
ContributorAuthor

Thanks @captainzmc , already added UT.

szetszwo
szetszwo reviewed Oct 28, 2021
View reviewed changes

@szetszwoszetszwo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.

Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java Outdated

@szetszwoszetszwoOct 28, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?

@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo
I noticed it. Thanks for reminding me. I have revised it.

@guohao-rosicky
guohao-rosickyforce-pushed the ozone-client-multipart-stream-key branch 2 times, most recently from e10fe82 to b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed the ozone-client-multipart-stream-key branch from b9c5d4c to c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo reviewed Nov 16, 2021
View reviewed changes

@szetszwoszetszwo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.

The new testMultiPartUploadWithStream() failed. Please take a look.

I also have a minor comment inlined.

Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java Outdated
if (feInfo != null) {
// todo: need to support file encrypt,
// https://issues.apache.org/jira/browse/HDDS-5892
return null;

@szetszwoszetszwoNov 16, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.

 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo

szetszwo commented Nov 17, 2021

Copy link
Copy Markdown
Contributor

@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.

@guohao-rosicky

guohao-rosicky commented Nov 19, 2021

Copy link
Copy Markdown
ContributorAuthor

@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.

Thanks @szetszwo , please TBR.

szetszwo
szetszwo approved these changes Nov 19, 2021
View reviewed changes

@szetszwoszetszwo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.

@szetszwo
szetszwo merged commit a8a25b8 into apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit that referenced this pull request Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit that referenced this pull request Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit that referenced this pull request Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit to szetszwo/ozone that referenced this pull request May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit that referenced this pull request May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit that referenced this pull request Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit to captainzmc/hadoop-ozone that referenced this pull request Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit that referenced this pull request Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit that referenced this pull request Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit that referenced this pull request Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit that referenced this pull request Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit to nishitpatira/ozone that referenced this pull request Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method by guohao-rosicky · Pull Request #2760 · apache/ozone · GitHub
Skip to content
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method - #2760
Merged
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Nov 19, 2021
Merged
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method#2760
szetszwo merged 5 commits into
apache:HDDS-4454from 
guohao-rosicky:ozone-client-multipart-stream-key
Conversation
@guohao-rosicky
@guohao-rosickyguohao-rosicky
commented
Oct 22, 2021•

edited by szetszwo

Loading

Copy link


Copy Markdown

Contributor
https://issues.apache.org/jira/browse/HDDS-5879
OzoneBucket and RpcClient not support multipart upload with Stream.

Add the createMultipartStreamKey method
@captainzmc
captainzmc
commented
Oct 22, 2021

Copy link


Copy Markdown

Member
Thanks @guohao-rosicky for your contribution. Could you update the URL of jira in comment? Also, the current code check style still has some problems, and it is better to add some UT to test the new interface.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
f7ea63a to
46f4fa4CompareOctober 22, 2021 10:18
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
jira url: https://issues.apache.org/jira/browse/HDDS-5879
@guohao-rosicky
guohao-rosicky
commented
Oct 22, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , has been modified.
bharatviswa504
bharatviswa504
reviewed
Oct 22, 2021
View reviewed changes
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

@bharatviswa504bharatviswa504Oct 22, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a Jira for this TODO
@guohao-rosickyguohao-rosickyOct 24, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bharatviswa504 , new jira url: https://issues.apache.org/jira/browse/HDDS-5892
@captainzmc
captainzmc
self-requested a review
October 25, 2021 02:50
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
46f4fa4 to
dffcd34CompareOctober 25, 2021 03:42
@guohao-rosicky
guohao-rosicky
commented
Oct 25, 2021

Copy link


Copy Markdown

ContributorAuthor
Thanks @captainzmc , already added UT.
szetszwo
szetszwo
reviewed
Oct 28, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Just a question inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java
Outdated

@szetszwoszetszwoOct 28, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we pass replicationConfig but not use openKey.getKeyInfo().getReplicationConfig()?
@guohao-rosickyguohao-rosickyNov 15, 2021

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szetszwo

I noticed it. Thanks for reminding me. I have revised it.
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
2 times, most recently
from
e10fe82 to
b9c5d4cCompareNovember 16, 2021 05:33
@guohao-rosicky
guohao-rosickyforce-pushed
the
ozone-client-multipart-stream-key
branch
from
b9c5d4c to
c47548aCompareNovember 16, 2021 08:53
szetszwo
szetszwo
reviewed
Nov 16, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guohao-rosicky , thanks for the update.
The new testMultiPartUploadWithStream() failed. Please take a look.
I also have a minor comment inlined.
Comment threadhadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Outdated

 if (feInfo != null) {
 // todo: need to support file encrypt,
 // https://issues.apache.org/jira/browse/HDDS-5892
 return null;
@szetszwoszetszwoNov 16, 2021

Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw a UnsupportedOperationException.
 throw new UnsupportedOperationException(
"FileEncryptionInfo is not yet supported in createMultipartStreamKey");

@szetszwo
szetszwo
commented
Nov 17, 2021

Copy link


Copy Markdown

Contributor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
@guohao-rosicky
guohao-rosicky
commented
Nov 19, 2021

Copy link


Copy Markdown

ContributorAuthor
@guohao-rosicky , TestOzoneRpcClientAbstract.testMultiPartUploadWithStream failed again in the last build. Please take a look.
Thanks @szetszwo , please TBR.
szetszwo
szetszwo
approved these changes
Nov 19, 2021
View reviewed changes
@szetszwoszetszwo
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.
@szetszwo
szetszwo
merged commit a8a25b8
into
apache:HDDS-4454Nov 19, 2021
szetszwo pushed a commit
that referenced
this pull request
Nov 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
f400954
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Dec 20, 2021
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
26877bd
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Dec 30, 2021
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
bd13493
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jan 20, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b8d31f2
…mKey method (#2760)
captainzmc pushed a commit
that referenced
this pull request
Feb 9, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
27c395a
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Feb 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a65c5d1
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 15, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
b6f5244
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Mar 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
a8d7669
…mKey method (#2760)
szetszwo pushed a commit
to szetszwo/ozone
that referenced
this pull request
May 6, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
22d0553
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
May 13, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be48e28
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
May 24, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
d159210
…mKey method (#2760)
szetszwo pushed a commit
that referenced
this pull request
Jun 9, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
085bbf1
…mKey method (#2760)
captainzmc pushed a commit
to captainzmc/hadoop-ozone
that referenced
this pull request
Jul 4, 2022
@guohao-rosicky@captainzmc
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
be9641f
…mKey method (apache#2760)
szetszwo pushed a commit
that referenced
this pull request
Oct 25, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
8de50af
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
szetszwo pushed a commit
that referenced
this pull request
Nov 7, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
2054fe4
…mKey method (#2760)
(cherry picked from commit be9641f)
(cherry picked from commit 9a16e41e1f29373e0e0f2a186237ded0dd0b53b4)
(cherry picked from commit 8de50af)
szetszwo pushed a commit
that referenced
this pull request
Dec 1, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
3f1e2a5
…mKey method (#2760)
(cherry picked from commit 2054fe4)
szetszwo pushed a commit
that referenced
this pull request
Dec 16, 2022
@guohao-rosicky@szetszwo
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
237e0af
…mKey method (#2760)
(cherry picked from commit 2054fe4)
nishitpatira pushed a commit
to nishitpatira/ozone
that referenced
this pull request
Dec 16, 2022
@guohao-rosickyNishit Patira
HDDS-5879. [Ozone-Streaming] OzoneBucket add the createMultipartStrea…
4d56f2c
…mKey method (apache#2760)
(cherry picked from commit 2054fe4)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@szetszwoszetszwoszetszwo approved these changes
@captainzmccaptainzmcAwaiting requested review from captainzmc
@bharatviswa504bharatviswa504Awaiting requested review from bharatviswa504

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@guohao-rosicky@captainzmc@szetszwo@bharatviswa504