Skip to content

stream: add the pipeline destoryDestOnError option - #34133

Closed
rickyes wants to merge 11 commits into
nodejs:masterfrom
rickyes:add-pipeline-destoryDestOnError
Closed

stream: add the pipeline destoryDestOnError option#34133
rickyes wants to merge 11 commits into
nodejs:masterfrom
rickyes:add-pipeline-destoryDestOnError

Conversation

@rickyes

Copy link
Copy Markdown
Contributor

Fixes: #26311

/cc @mcollina@ronag

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@ronag

Copy link
Copy Markdown
Member

Could we add a new signature to pipeline with something along the lines of:

pipeline(arrayOfStreams,options,callback)

@rickyes

Copy link
Copy Markdown
ContributorAuthor

Could we add a new signature to pipeline with something along the lines of:

pipeline(arrayOfStreams,options,callback)

Yes, I also found the missing documentation for this section, maybe we can create a new PR to document this separately.

@ronag

Copy link
Copy Markdown
Member

I think in readable.pipe the option is called end to not end(). Maybe we should have { end, destroy } as options?

@addaleaxaddaleax added semver-minor PRs that contain new features and should be released in the next minor version. stream Issues and PRs related to the stream subsystem. labels Jun 30, 2020
@rickyes

Copy link
Copy Markdown
ContributorAuthor

I think in readable.pipe the option is called end to not end(). Maybe we should have { end, destroy } as options?

It may be a little different, the destoryDestOnError option takes effect when an error occurs in the stream.

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test with more than 2 streams?

@rickyes

Copy link
Copy Markdown
ContributorAuthor

Can you add a test with more than 2 streams?

done

@ronag

ronag commented Jul 1, 2020

Copy link
Copy Markdown
Member

I don't find this very elegant. Though I guess there is nothing wrong with it per se... though couldn't you easily achieve the same thing with:

constdst=streams.pop()awaitfinished(pipeline(streams,()=>{}).on('error',(err)=>{// ...}).pipe(dst))

@rickyes

Copy link
Copy Markdown
ContributorAuthor

I don't find this very elegant. Though I guess there is nothing wrong with it per se... though couldn't you easily achieve the same thing with:

constdst=streams.pop()awaitfinished(pipeline(streams,()=>{}).on('error',(err)=>{// ...}).pipe(dst))

Come to think of it, maybe the way @ronag offers it would have fewer side effects. The PR implementation, when an error occurs during the transmission of a stream (response stream has been data sent) we send end('error') again which may be incorrect.

@rickyes

Copy link
Copy Markdown
ContributorAuthor

However, the disadvantage is that we need to combine two api together, which is not very convenient.

@ronag

ronag commented Jul 1, 2020

Copy link
Copy Markdown
Member

However, the disadvantage is that we need to combine two api together, which is not very convenient.

This is a bit of an edge case though... So I'm not sure how important the convenience is :). I'm not opposed to this PR, but I'm unsure whether we need it.

@rickyes
rickyes requested a review from mcollinaJuly 2, 2020 13:51
@rickyes
rickyesforce-pushed the add-pipeline-destoryDestOnError branch from 198779e to 75ad480CompareJuly 8, 2020 04:43
@rickyes

Copy link
Copy Markdown
ContributorAuthor

ping @mcollina

@mcollina

Copy link
Copy Markdown
Member

I would really like some feedbacks from @mafintosh

@mcollina
mcollina requested a review from mafintoshJuly 8, 2020 10:59
Comment threaddoc/api/stream.md Outdated
Comment on lines +1660 to +1662
* `destroyDestOnError` {boolean} If value is false, the destination
is not destroyed when a stream error occurs.
**Default:** `true`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*`destroyDestOnError` {boolean} If value is false, the destination
is not destroyed when a stream error occurs.
**Default:**`true`.
*`destroyDestOnError` {boolean} If value is `false`, the destination
is not destroyed when a stream error occurs. **Default:** `true`.

Not particularly in love with name destroyDestOnError but also don't have any better suggestions.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@rickyes

Copy link
Copy Markdown
ContributorAuthor

ping again @mafintosh

@mafintosh

Copy link
Copy Markdown
Member

I don’t really see the use-case, since you’d have to do some error detection as well to make sure that the dest stream hasn’t been partially written, as the error could be lots of things from the pipeline.

For the use case described in the issue I’d wait for the open event instead that @ronag recently standardized.

In general, PR looks fine, -0 on the feature.

@rickyes

Copy link
Copy Markdown
ContributorAuthor

I don’t really see the use-case, since you’d have to do some error detection as well to make sure that the dest stream hasn’t been partially written, as the error could be lots of things from the pipeline.

Yes, that's what I was worried about at first, and I don't have a good way to handle the situation yet, so if people have some suggestions then Would be great.

@rickyes

Copy link
Copy Markdown
ContributorAuthor

Due to lack of sufficient impetus, I will close this PR.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver-minorPRs that contain new features and should be released in the next minor version.streamIssues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream.pipeline destroys writable stream when error is occurred

6 participants

@rickyes@ronag@mcollina@mafintosh@jasnell@addaleax