Skip to content

stream: writable buffering - #28978

Closed
ronag wants to merge 1 commit into
nodejs:masterfrom
nxtedition:stream-writable-buffer
Closed

stream: writable buffering#28978
ronag wants to merge 1 commit into
nodejs:masterfrom
nxtedition:stream-writable-buffer

Conversation

@ronag

@ronagronag commented Aug 5, 2019

Copy link
Copy Markdown
Member

Simplify and optimize writable stream buffering.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-botnodejs-github-bot added the stream Issues and PRs related to the stream subsystem. label Aug 5, 2019
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina@Trott I need a little help on how to run and compare benchmarks on this? Haven't really found the instructions in the guidelines...

@mcollina

Copy link
Copy Markdown
Member

Here they are: https://github.com/nodejs/node/blob/master/doc/guides/writing-and-running-benchmarks.md

Be careful with this change, you are touching one of the worst hot paths in Node.js (together with emit and nextTick). The current code is complex for reasons that made sense in old version of Node.js, so it might be a good time to check if those assumptions still hold true.

@mcollina

Copy link
Copy Markdown
Member

The semversiness of this is a bit problematic, as this is likely to break some external users.

@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina: this is definitly semver major.

Is this something that you would even consider? If not please let me know so I don't spend time in vain :).

@ronag
ronagforce-pushed the stream-writable-buffer branch from dc5503a to a092cb8CompareAugust 5, 2019 14:05
@ronagronag mentioned this pull request Aug 5, 2019
4 tasks
@mcollina

Copy link
Copy Markdown
Member

I think it's worthwhile having a look if it could be simplified/improved the performance of it.

@ronag
ronagforce-pushed the stream-writable-buffer branch from a092cb8 to 23ee586CompareAugust 5, 2019 14:20
@lpinca

Copy link
Copy Markdown
Member

This should definitely be benchmarked to see if it's actually faster and how much.

Comment threadlib/_stream_writable.js Outdated
@ronag
ronagforce-pushed the stream-writable-buffer branch from 23ee586 to b6a212fCompareAugust 5, 2019 14:29
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina@lpinca

Before:

writable-manywrites.js: 2,549,442

After:

writable-manywrites.js: 2,652,592

@ronag
ronagforce-pushed the stream-writable-buffer branch from b6a212f to 3e51f4cCompareAugust 5, 2019 14:50
Comment threadlib/_stream_writable.js Outdated
@ronag
ronagforce-pushed the stream-writable-buffer branch 2 times, most recently from a2ce565 to a8ba29cCompareAugust 5, 2019 15:00
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

Got the benchmark comparison working!

streams/writable-manywrites.js n=2000000 0.83 % ±2.70% ±3.60% ±4.71%

Seems like a negligible difference. Though the simplification part is still probably worth it.

@ronag
ronagforce-pushed the stream-writable-buffer branch 12 times, most recently from c0088f0 to a5607bdCompareAugust 5, 2019 17:20
@ronag
ronagforce-pushed the stream-writable-buffer branch 3 times, most recently from d99f8da to 85b51a2CompareAugust 5, 2019 17:36
Comment threadlib/_stream_writable.js Outdated
@ronag
ronagforce-pushed the stream-writable-buffer branch from 85b51a2 to 30cb4e6CompareAugust 5, 2019 18:02
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina nothing I do here seems to make much of a difference... maybe not a hot path or maybe better benchmarks are needed? Or I'm doing something wrong...

The current benchmark mostly test sync writing... which I don't think is a good real-world example.

@ronag
ronagforce-pushed the stream-writable-buffer branch 7 times, most recently from bfc1416 to cd6006cCompareAugust 5, 2019 18:16
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

Note this also changes the behaviour so that 'finish' is never emitted after 'error'.

@ronag
ronagforce-pushed the stream-writable-buffer branch 2 times, most recently from 53164df to df77306CompareAugust 5, 2019 18:33
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

@mcollina can you please look into this one #28979 before further reviewing this PR.

@ronag
ronagforce-pushed the stream-writable-buffer branch from df77306 to 69a6436CompareAugust 5, 2019 18:59
@ronag
ronagforce-pushed the stream-writable-buffer branch from 69a6436 to 9033d42CompareAugust 5, 2019 19:00
@ronag

ronag commented Aug 5, 2019

Copy link
Copy Markdown
MemberAuthor

this is WIP

@ronagronag closed this Aug 5, 2019
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

streamIssues and PRs related to the stream subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@ronag@mcollina@lpinca@mscdex@nodejs-github-bot