Skip to content

[x] stream: eof & pipeline compat - #29724

Closed
ronag wants to merge 5 commits into
nodejs:masterfrom
nxtedition:fix-eof-compat
Closed

[x] stream: eof & pipeline compat#29724
ronag wants to merge 5 commits into
nodejs:masterfrom
nxtedition:fix-eof-compat

Conversation

@ronag

@ronagronag commented Sep 26, 2019

Copy link
Copy Markdown
Member

Some broken streams might emit 'close' before 'end' or 'finish'. However, if they have actually been ended or finished, this should not result in a premature close error.

This fixes#29699 while still maintaining what could be considered correct behaviour.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@ronag

Copy link
Copy Markdown
MemberAuthor

ping @mafintosh

@lpincalpinca added the stream Issues and PRs related to the stream subsystem. label Sep 26, 2019
@ronag
ronagforce-pushed the fix-eof-compat branch 4 times, most recently from 617e410 to ec1fb21CompareSeptember 26, 2019 19:00
@ronag
ronagforce-pushed the fix-eof-compat branch 10 times, most recently from b1deba7 to 53e0fe6CompareSeptember 26, 2019 19:32
Comment threaddoc/api/stream.md Outdated
@ronag
ronagforce-pushed the fix-eof-compat branch 5 times, most recently from 92d3138 to b2f52b5CompareSeptember 26, 2019 20:24
@ronagronag changed the title stream: eof compatstream: eof & pipeline compatSep 26, 2019
@ronag
ronagforce-pushed the fix-eof-compat branch 2 times, most recently from bbba57c to 9e9b654CompareSeptember 26, 2019 20:49
Comment threadtest/parallel/test-stream-pipeline.js Outdated
@ronag

Copy link
Copy Markdown
MemberAuthor

Is this still relevant? Been stale for a while...

Happy new year!

@ronag

ronag commented Jan 1, 2020

Copy link
Copy Markdown
MemberAuthor

rebased to fix conflicts

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Comment threadtest/parallel/test-http-client-finished.js Outdated
Comment threadtest/parallel/test-http-client-finished.js Outdated
@ronag

ronag commented Jan 1, 2020

Copy link
Copy Markdown
MemberAuthor

CI

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@ronag

ronag commented Jan 9, 2020

Copy link
Copy Markdown
MemberAuthor

ping @mafintosh

Make stream.finished callback invoked if stream is already
closed/destroyed.
PR-URL: nodejs#29664
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Some broken streams might emit 'close' before 'end' or 'finish'.
However, if they have actually been ended or finished, this
should not result in a premature close error.
Fixes: nodejs#29699
@ronag

Copy link
Copy Markdown
MemberAuthor

rebased to fix conflict

@ronag

Copy link
Copy Markdown
MemberAuthor

Turns out this actually had a little bit of an issue, the stream might still be in a pending state after destroyed so checking that is not enough.

@ronag
ronagforce-pushed the fix-eof-compat branch 3 times, most recently from a858773 to bfca3d1CompareJanuary 18, 2020 19:47
@mafintosh

Copy link
Copy Markdown
Member

This is still quite complex, but tried to do another pass at it. Can give an slight 👍. As mentioned in my comment I think it's worth investing in making a happy-path for when a stream is autoDestroy: true since that's much easier to reason about.

@ronag

ronag commented Jan 21, 2020

Copy link
Copy Markdown
MemberAuthor

@mafintosh

but tried to do another pass at it. Can give an slight 👍.

Thx!

I think it's worth investing in making a happy-path for when a stream is autoDestroy: true since that's much easier to reason about.

I'm not quite sure I understand what you mean by this? Something like the following?

if(autoDestroy&&emitClose){if(opts.error!==false)stream.on('error',onerror);functiononclose(){callback.call(stream);}if(closed){process.nextTick(callback.bind(stream));}else{stream.on('close',onclose);}return()=>{stream.removeListener('close',onclose);stream.removeListener('error',onerror);};}// Rest is compat and edge cases...

@ronag

ronag commented Jan 25, 2020

Copy link
Copy Markdown
MemberAuthor

My general feeling is that landing this safely is uncertain due to the complexity of the changes and uncertainty of reviewers.

There are actually several changes here that do not need to be directly related and could be split up.

I'm closing this in favour of smaller on more specific PR's. Please let me know if there are any objections to this.

@ronagronag closed this Jan 25, 2020
@ronagronag changed the title stream: eof & pipeline compat[x] stream: eof & pipeline compatJan 25, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

streamIssues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream.finished behaviour change

8 participants

@ronag@Trott@nodejs-github-bot@mcollina@mafintosh@jasnell@BridgeAR@lpinca