Skip to content

http: avoid retaining unneeded memory - #11926

Closed
lpinca wants to merge 1 commit into
nodejs:masterfrom
lpinca:free/unneeded-memory
Closed

http: avoid retaining unneeded memory#11926
lpinca wants to merge 1 commit into
nodejs:masterfrom
lpinca:free/unneeded-memory

Conversation

@lpinca

Copy link
Copy Markdown
Member

Prevents the events listeners of the sockets obtained with the HTTP upgrade mechanism from retaining unneeded objects like state.

Reduces memory usage by ~20%.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

@nodejs-github-botnodejs-github-bot added the http Issues or PRs related to the http subsystem. label Mar 19, 2017
Comment threadlib/_http_server.js Outdated

@lpincalpincaMar 19, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is ok but there are 2 listeners for the drain event and both seem to be useless for upgrade requests.
The first checks if _httpMessage is set and this is never true for upgrade requests.
The second checks if the socket is _paused but unless I'm missing something this flag is never set for upgrade requests.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.removeAllListeners() also removes any user-installed 'drain' event listeners so no, probably not a good idea.

@lpincalpincaMar 20, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I get that but can a user add a listener for the drain event before this is called?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, server.on('connection', c => c.on('drain', ondrain)).

@lpinca
lpincaforce-pushed the free/unneeded-memory branch from 4b1709a to 5c2d743CompareMarch 20, 2017 07:33
@jasnell

Copy link
Copy Markdown
Member

/cc @nodejs/http

@cjihrigcjihrig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending CI

@lpinca

Copy link
Copy Markdown
MemberAuthor

Refs: #11868

Comment threadlib/_http_server.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.removeAllListeners() also removes any user-installed 'drain' event listeners so no, probably not a good idea.

@lpinca
lpincaforce-pushed the free/unneeded-memory branch from 5c2d743 to b435591CompareMarch 20, 2017 19:04
@lpinca

Copy link
Copy Markdown
MemberAuthor

@bnoordhuis updated, PTAL.
This listener is no longer removed as I can't find a clean way to selectively remove it without exporting the function.

@bnoordhuis

Copy link
Copy Markdown
Member

@lpinca You could move it to lib/internal/http.js.

@lpinca

Copy link
Copy Markdown
MemberAuthor

Done, thanks.

Comment threadlib/internal/http.js Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a comma, saves one line in the diff next time someone adds a property.

@lpinca
lpincaforce-pushed the free/unneeded-memory branch from 295bd86 to 76b237bCompareMarch 20, 2017 20:17
@jasnell

Copy link
Copy Markdown
Member

@jasnell

Copy link
Copy Markdown
Member

@lpinca ... this is not landing cleanly for some reason (even tho it shows zero conflicts...) can you please squash the commits, rebase, and I'll try again.

Prevent the events listeners of the sockets obtained with the HTTP
upgrade mechanism from retaining unneeded memory.
Refs: nodejs#11868
@lpinca
lpincaforce-pushed the free/unneeded-memory branch from 76b237b to 0100984CompareMarch 22, 2017 17:15
@lpinca

Copy link
Copy Markdown
MemberAuthor

@jasnell done.

jasnell pushed a commit that referenced this pull request Mar 22, 2017
Prevent the events listeners of the sockets obtained with the HTTP
upgrade mechanism from retaining unneeded memory.
Ref: #11868
PR-URL: #11926
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@jasnell

Copy link
Copy Markdown
Member

Landed in e0a9ad1

@jasnelljasnell closed this Mar 22, 2017
@lpinca
lpinca deleted the free/unneeded-memory branch March 22, 2017 17:34
@MylesBorins

Copy link
Copy Markdown
Contributor

This will need to be manually backported to v7.x

lpinca added a commit to lpinca/node that referenced this pull request Mar 29, 2017
Prevent the events listeners of the sockets obtained with the HTTP
upgrade mechanism from retaining unneeded memory.
Ref: nodejs#11868
PR-URL: nodejs#11926
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
@jasnelljasnell mentioned this pull request Apr 4, 2017
@MylesBorins

Copy link
Copy Markdown
Contributor

Looks like this change is breaking spdy. Example breakage from spdy tests.

 6) SPDY Server spdy/3 plain mode should not crash on .writeHead() after socket close:
Uncaught TypeError: Cannot read property 'emit' of null
at Socket.socketOnError (_http_server.js:450:19)
at Stream.onStreamError (lib/spdy/handle.js:110:18)
at node_modules/spdy-transport/lib/spdy-transport/connection.js:749:12
at Array.forEach (native)
at Connection.destroyStreams (node_modules/spdy-transport/lib/spdy-transport/connection.js:745:33)
at Socket.onclose (node_modules/spdy-transport/lib/spdy-transport/connection.js:185:10)
at TCP._handle.close [as _onclose] (net.js:534:12)

@ronkorving

Copy link
Copy Markdown
Contributor

@indutny Any chance this could be solved on the spdy side of things?

@bnoordhuis

Copy link
Copy Markdown
Member

Removing the backport-to-v7.x label and adding dont-land-on labels until this is resolved.

@lpinca

Copy link
Copy Markdown
MemberAuthor

Not sure if the following is the proper way to fix the issue but it seems to work:

diff --git a/lib/spdy/server.js b/lib/spdy/server.js
index 777f682..d5321c2 100644
--- a/lib/spdy/server.js+++ b/lib/spdy/server.js@@ -189,6 +189,8 @@ proto._onStream = function _onStream (stream) {
socket = new net.Socket(socketOptions)
}
+ socket.server = this;+
handle.assignSocket(socket)
// For v0.8

Will send a PR.

@lpinca

Copy link
Copy Markdown
MemberAuthor

That patch seems to not work on Node.js 0.10 and 0.12.

The problem is that node-spdycalls the connectionListener on a new socket for which the server property is null.

lpinca added a commit to lpinca/node that referenced this pull request Jun 9, 2017
Fixes a regression that caused an error to be thrown when trying to
emit the 'timeout' event on the server referenced by `socket.server`.
Fixes: nodejs#13435
Refs: nodejs#11926
addaleax pushed a commit that referenced this pull request Jun 12, 2017
Fixes a regression that caused an error to be thrown when trying to
emit the 'timeout' event on the server referenced by `socket.server`.
Fixes: #13435
Refs: #11926
PR-URL: #13578
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
addaleax pushed a commit that referenced this pull request Jun 12, 2017
Fixes a regression that caused an error to be thrown when trying to
emit the 'timeout' event on the server referenced by `socket.server`.
Fixes: #13435
Refs: #11926
PR-URL: #13578
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

httpIssues or PRs related to the http subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@lpinca@jasnell@bnoordhuis@MylesBorins@ronkorving@cjihrig@nodejs-github-bot