HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b… by sadanand48 · Pull Request #2946 · apache/ozone

sadanand48 · 2021-12-24T19:52:00Z

…ufferList.

What changes were proposed in this pull request?

This is a follow up to HDDS-5851. Define a limit/window on the bufferList size which is used for retry. Once this limit is hit , client will perform watchForCommit for the first flush size (first putBlock) worth of data.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6138

How was this patch tested?

unit tests

captainzmc · 2022-01-21T02:26:11Z

Hi @sadanand48, can you update this PR?

captainzmc · 2022-01-27T02:30:12Z

Hi @sadanand48, I tried the cherry-pick PR in my local test, but this CI never passed. Can you confirm the problem of CI?

sadanand48 · 2022-02-09T10:17:18Z

@captainzmc I have fixed some tests that were failing. The current CI failure I think is related to HDDS-6239 . We should sync master with this branch to resolve this

captainzmc · 2022-02-09T11:38:43Z

@captainzmc I have fixed some tests that were failing. The current CI failure I think is related to HDDS-6239 . We should sync master with this branch to resolve this

Thanks @sadanand48 for the update. Just now I have synchronized HDDS-4454 with master. Could you rebase the code? The PR looks good overall, just left a minor comment.

captainzmc · 2022-02-10T01:10:32Z

Hi @sadanand48, In #3059 we had update ratis version and resolve dependency check issues. Currently #3059 has been merged, So you can remove the modification of these two parts in this PR.

sadanand48 · 2022-02-10T04:23:57Z

Thanks @captainzmc for taking care of the dependency check. I have rebased the PR now.

captainzmc

+1 The change looks good.

captainzmc · 2022-02-10T09:07:14Z

Merge this PR. Thanks for @sadanand48’s contribution and @bshashikant for the review.

…ufferList. (apache#2946) Co-authored-by: Sadanand Shenoy <sadanand.shenoy@cloudera.com>

…ufferList. (#2946)

…ufferList. (apache#2946)

…ufferList. (#2946)

…ufferList. (apache#2946)

…ufferList. (#2946) (cherry picked from commit 30f8311) (cherry picked from commit 54361e5a4ded763d83c5b3116203deece0303960)

…ufferList. (#2946) (cherry picked from commit 30f8311) (cherry picked from commit 54361e5a4ded763d83c5b3116203deece0303960) (cherry picked from commit 693eea7)

…ufferList. (#2946) (cherry picked from commit 340f959)

…ufferList. (apache#2946) (cherry picked from commit 340f959)

szetszwo force-pushed the HDDS-4454 branch from 7ce4e01 to ab016f3Compare December 30, 2021 07:59

sadanand48 force-pushed the HDDS-6138 branch from 4dad15b to 6fe6ff0Compare January 4, 2022 07:18

bshashikant reviewed Jan 5, 2022
View reviewed changes

Comment threadhadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockDataStreamOutput.java Outdated

bshashikant reviewed Jan 5, 2022
View reviewed changes

Comment threadhadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockDataStreamOutput.java Outdated

sadanand48 marked this pull request as ready for review January 12, 2022 07:51

bshashikant reviewed Jan 13, 2022
View reviewed changes

Comment threadhadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockDataStreamOutput.java Outdated

szetszwo force-pushed the HDDS-4454 branch from 1ceae85 to e8af582Compare January 20, 2022 09:14

sadanand48 force-pushed the HDDS-6138 branch from ad00fb7 to e8374c0Compare January 24, 2022 05:25

sadanand48 requested a review from captainzmc February 9, 2022 10:17

captainzmc force-pushed the HDDS-4454 branch from e8af582 to dba70bbCompare February 9, 2022 11:14

captainzmc reviewed Feb 9, 2022
View reviewed changes

Comment threadhadoop-ozone/dist/src/main/license/jar-report.txt Outdated

rebase HDDS-6138
5a85fdc

sadanand48 force-pushed the HDDS-6138 branch from 581a10e to 5a85fdcCompare February 10, 2022 04:22

captainzmc approved these changes Feb 10, 2022
View reviewed changes

captainzmc merged commit ad7b45e into apache:HDDS-4454Feb 10, 2022

SincereXIA pushed a commit to SincereXIA/ozone that referenced this pull request Feb 11, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
3fe8965
…ufferList. (apache#2946) Co-authored-by: Sadanand Shenoy <sadanand.shenoy@cloudera.com>

szetszwo pushed a commit that referenced this pull request Feb 16, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
bc87050
…ufferList. (#2946)

szetszwo pushed a commit that referenced this pull request Mar 15, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
4f9265e
…ufferList. (#2946)

szetszwo pushed a commit that referenced this pull request Mar 24, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
d290fba
…ufferList. (#2946)

szetszwo pushed a commit to szetszwo/ozone that referenced this pull request May 6, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
1bbc76f
…ufferList. (apache#2946)

szetszwo pushed a commit that referenced this pull request May 13, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
60fedb0
…ufferList. (#2946)

szetszwo pushed a commit that referenced this pull request May 24, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
5dce655
…ufferList. (#2946)

szetszwo pushed a commit that referenced this pull request Jun 9, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
32598ca
…ufferList. (#2946)

captainzmc pushed a commit to captainzmc/hadoop-ozone that referenced this pull request Jul 4, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
30f8311
…ufferList. (apache#2946)

szetszwo pushed a commit that referenced this pull request Oct 25, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
693eea7
…ufferList. (#2946) (cherry picked from commit 30f8311) (cherry picked from commit 54361e5a4ded763d83c5b3116203deece0303960)

szetszwo pushed a commit that referenced this pull request Dec 1, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
3b1d2cf
…ufferList. (#2946) (cherry picked from commit 340f959)

szetszwo pushed a commit that referenced this pull request Dec 16, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
2657f5a
…ufferList. (#2946) (cherry picked from commit 340f959)

nishitpatira pushed a commit to nishitpatira/ozone that referenced this pull request Dec 16, 2022
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
47145dc
…ufferList. (apache#2946) (cherry picked from commit 340f959)

Navigation Menu

Uh oh!

HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b… - #2946

HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…#2946
captainzmc merged 1 commit into
apache:HDDS-4454from
sadanand48:HDDS-6138

sadanand48 commented Dec 24, 2021•
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

captainzmc commented Jan 21, 2022

Uh oh!

captainzmc commented Jan 27, 2022

Uh oh!

sadanand48 commented Feb 9, 2022

Uh oh!

Uh oh!

captainzmc commented Feb 9, 2022

Uh oh!

captainzmc commented Feb 10, 2022

Uh oh!

sadanand48 commented Feb 10, 2022

Uh oh!

captainzmc left a comment

Uh oh!

captainzmc commented Feb 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

sadanand48 commented Dec 24, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

captainzmc commented Jan 21, 2022

Uh oh!

captainzmc commented Jan 27, 2022

Uh oh!

sadanand48 commented Feb 9, 2022

Uh oh!

Uh oh!

captainzmc commented Feb 9, 2022

Uh oh!

captainzmc commented Feb 10, 2022

Uh oh!

sadanand48 commented Feb 10, 2022

Uh oh!

captainzmc left a comment

Choose a reason for hiding this comment

Uh oh!

captainzmc commented Feb 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sadanand48 commented Dec 24, 2021•
edited
Loading