Skip to content

stream: eos on closed - #28748

Closed
ronag wants to merge 12 commits into
nodejs:masterfrom
nxtedition:stream-eos-on-closed
Closed

stream: eos on closed#28748
ronag wants to merge 12 commits into
nodejs:masterfrom
nxtedition:stream-eos-on-closed

Conversation

@ronag

@ronagronag commented Jul 18, 2019•
edited
Loading

Copy link
Copy Markdown
Member

stream.finished should be invoked if stream is already closed/destroyed. Currently there is a potential deadlock depending on when stream.finished is called.

For writable premature close is close before finish, i.e. it should look for finished instead of ended.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@ronag
ronagforce-pushed the stream-eos-on-closed branch 8 times, most recently from 3c57be4 to c9d1e19CompareJuly 22, 2019 06:42
@Trott

Trott commented Jul 23, 2019

Copy link
Copy Markdown
Member

/ping @nodejs/streams

@nodejs-github-bot

nodejs-github-bot commented Jul 23, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/24624/

mcollina
mcollina requested changes Jul 23, 2019
View reviewed changes

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind to add the equivalent test for Readable as well?

Comment threadlib/internal/streams/end-of-stream.js Outdated
mcollina
mcollina requested changes Jul 23, 2019
View reviewed changes
Comment threadlib/internal/streams/end-of-stream.js Outdated
Comment threadtest/parallel/test-stream-eos.js Outdated
@ronagronag mentioned this pull request Jul 23, 2019
Closed
1 task
@ronag
ronagforce-pushed the stream-eos-on-closed branch from c9d1e19 to 3816f11CompareJuly 23, 2019 07:30
@ronag

ronag commented Jul 23, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina updated

@ronag
ronagforce-pushed the stream-eos-on-closed branch 2 times, most recently from aefe068 to 1588c78CompareJuly 23, 2019 07:37
mcollina
mcollina requested changes Jul 23, 2019
View reviewed changes
Comment threadlib/internal/streams/end-of-stream.js Outdated
@mcollinamcollina added the semver-major PRs that contain breaking changes and should be released in the next major version. label Jul 23, 2019
@ronag
ronagforce-pushed the stream-eos-on-closed branch 3 times, most recently from 40b4c0b to 39ca23aCompareJuly 24, 2019 09:46
@Fishrock123Fishrock123 added the stream Issues and PRs related to the stream subsystem. label Jul 24, 2019
@ronag

ronag commented Aug 2, 2019

Copy link
Copy Markdown
MemberAuthor

@benjamingr ping

mcollina
mcollina approved these changes Aug 2, 2019
View reviewed changes

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mcollina
mcollina requested a review from a teamAugust 3, 2019 16:53
@mcollina

mcollina commented Aug 3, 2019

Copy link
Copy Markdown
Member

@nodejs/tsc this needs another review.

@ronag

ronag commented Aug 6, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina: Should this maybe result in a ERR_STREAM_DESTROYED?

@ronagronag mentioned this pull request Aug 6, 2019
Closed
2 tasks
@ronagronag mentioned this pull request Sep 1, 2019
Closed
4 tasks
@ronag

ronag commented Sep 1, 2019

Copy link
Copy Markdown
MemberAuthor

Yes please.

@mcollina: done

@ronag

ronag commented Sep 1, 2019

Copy link
Copy Markdown
MemberAuthor

@Trott: Updated PR description to describe the included changes from the other PR (which is now closed).

@ronag

ronag commented Sep 1, 2019

Copy link
Copy Markdown
MemberAuthor

This needs another CITGM (once Travis is ok) given the recent changes.

ronag
ronag commented Sep 1, 2019
View reviewed changes
Comment threadlib/internal/streams/end-of-stream.js Outdated

@ronagronagSep 1, 2019•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an added test for this change. We should always be emitting premature close unless we've seen finish or end events. I don't think this actually changes anything in practice.

@nodejs-github-bot

nodejs-github-bot commented Sep 1, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/25315/

@nodejs-github-bot

nodejs-github-bot commented Sep 1, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/25317/

@mcollina

mcollina commented Sep 2, 2019

Copy link
Copy Markdown
Member

This will need a rebase, otherwise it looks good to land.

@ronag
ronagforce-pushed the stream-eos-on-closed branch from db6eb4f to 9b5a84eCompareSeptember 2, 2019 14:52
@ronag

ronag commented Sep 2, 2019

Copy link
Copy Markdown
MemberAuthor

rebased

@nodejs-github-bot

nodejs-github-bot commented Sep 2, 2019

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/25326/

@mcollina

mcollina commented Sep 3, 2019

Copy link
Copy Markdown
Member

Landed in b03845b

@mcollinamcollina closed this Sep 3, 2019
mcollina pushed a commit that referenced this pull request Sep 3, 2019
@ronag@mcollina
stream: make finished call the callback if the stream is closed
b03845b
Make stream.finished callback invoked if stream is already
closed/destroyed.
PR-URL: #28748
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
@BethGriggsBethGriggs mentioned this pull request Sep 9, 2019
Merged
@mafintosh

mafintosh commented Sep 23, 2019

Copy link
Copy Markdown
Member

A little late to the party here, but this includes a big change in behaviour for streams.

Since the finished flag is used instead of the ended flag, this PR changes the behaiviour of eos to the following:

conststream=require('stream')constws=newstream.Writable({write(data,enc,cb){process.nextTick(cb,null)}})finished(ws,function(err){console.log('This used to *not* error:',err)})ws.write('data')ws.end()ws.emit('close')// modules tend to emit close on resource close but in a non error state

I was just bitten by this in a debugging session, so unsure if that was an intentional change?
If so I think it should be clearly documented :)

Streams are hard.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollinamcollinamcollina approved these changes
@jasnelljasnelljasnell approved these changes
@benjamingrbenjamingrbenjamingr approved these changes
@trivikrtrivikrtrivikr approved these changes
+1 more reviewer
@mafintoshmafintoshmafintosh left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

semver-majorPRs that contain breaking changes and should be released in the next major version.streamIssues and PRs related to the stream subsystem.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@ronag@Trott@nodejs-github-bot@mcollina@mafintosh@jasnell@benjamingr@trivikr@Fishrock123