src: do not reuse HTTPParser by Drieger · Pull Request #25094 · nodejs/node

Drieger · 2018-12-17T18:55:53Z

Change resource being used, previously HTTParser was being reused. We are now using IncomingMessage and ClientRequest objects. The goal here is to make the async resource unique for each async operation

Refs: #24330
Refs: nodejs/diagnostics#248
Refs: #21313

Co-authored-by: Matheus Marchini matheus@sthima.com

I continued @mmarchini work on #24330 and finished what was missing. I also added a test based on @AndreasMadsen comment nodejs/diagnostics#248 (comment)

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

mmarchini · 2018-12-17T19:14:33Z

CI: https://ci.nodejs.org/job/node-test-pull-request/19624/ ❌

mmarchini · 2018-12-17T19:14:50Z

cc @nodejs/async_hooks @nodejs/diagnostics

mcollina

LGTM

Drieger · 2018-12-17T19:55:06Z

The problem with linter is that common is being imported but not used, but when I do not import common linter error is that it is mandatory. Any ideas how to solve this one?

richardlau · 2018-12-17T19:57:12Z

The problem with linter is that common is being imported but not used, but when I do not import common linter error is that it is mandatory. Any ideas how to solve this one?

Just write:

require('../common');

or alternatively use common (e.g. common.mustCall).

Flarna · 2018-12-17T20:00:26Z

If I understand this correct the resource type is still HTTPPARSER but depending on incoming/outgoing the resource passed in init differs. I think this is confusing.

Wouldn't it better to have two resource types instead?

mscdex · 2018-12-18T00:23:06Z

Perhaps the commit message could be more clear. 'do not reuse HTTPParser' makes it sound as if parser objects are not being reused, when in fact it's the associated async resource that's no longer being reused (if I understand correctly).

AndreasMadsen

Thanks for spending time on this.

Drieger · 2018-12-19T12:13:19Z

@Flarna maybe we could use IncomingMessage and ClientRequest instead? What do you think?

Flarna · 2018-12-19T12:57:35Z

Not sure here; maybe prefix with Http as these names are quite generic in the global AsyncHooks scope.

Flarna · 2018-12-19T13:03:45Z

Isn't this reusing the IncomingMessage constructor function instead of the concrete instance? If I understand the code correct the IncomingMessage instance is created in parserOnHeadersComplete.
If I'm correct I wonder why the dedicated test-httparser-reuse.js is not detecting this.

@Drieger This one is still open and if I'm correct this is a no go.

You are right @Flarna I'll investigate this more in depth.

@FlarnakIncomingMessage is really a function as you said. I fixed the test to catch this.
I will also create a new object to be the resource for the server. Some other changes will be necessary as well. I will update the PR as soon as I this is working.

Drieger · 2019-01-09T10:38:44Z

Not sure here; maybe prefix with Http as these names are quite generic in the global AsyncHooks scope.

@Flarna I just updated the PR, adding two new providers HTTPINCOMINGMESSAGE and HTTPCLIENTREQUEST, could you take a look if that is what you had in mind?

I'll check why is the CI failing.

Flarna · 2019-01-09T11:01:57Z

Yes, looks ok. I think an alternative would be HTTP_REQUEST/HTTP_RESPONSE like in code which would point to what happens instead naming the resource type used (currently).

I would love if others give also their comments on these names as they are actually public API (even it's just in experimental part).

Is HTTPPARSER still needed?

https://github.com/nodejs/node/blob/master/doc/api/async_hooks.md needs also some update once agreement on the names is reached.

Drieger · 2019-01-09T11:13:12Z

You are right, I think HTTPPARSER is not necessary anymore. I'll update doc as soon as the "final" name is chosen.

Drieger · 2019-01-10T12:31:08Z

Fix test issues!
I'll update commit message after all changes have been approved.

/cc @mmarchini @mcollina @AndreasMadsen @richardlau @mscdex could you check again? What do you think about the naming suggested by @Flarna?

Drieger · 2019-01-14T18:47:00Z

I changed the resource in the _http_server.js file and fixed the test that was not able to capture the previous situation (function was being reused). I think it would be a good idea to run benchmarks if everyone agrees with the current implementation.

Obs.: Documentation and commit message will be updated if everything is ok.

mcollina · 2019-01-15T04:30:38Z

I think we should remove this parameter, or is it used somewhere else?

Removed parameter and renamed the method.

mcollina · 2019-01-17T12:38:06Z

CI: https://ci.nodejs.org/job/node-test-pull-request/20179/
CITGM: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1719/

mcollina · 2019-01-17T12:38:58Z

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/287/

mcollina · 2019-01-19T08:45:15Z

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/288/

(again, the previous one errored).

Drieger · 2019-03-06T20:25:54Z

I think I addressed all previous comments, any other comments on this? @nodejs/diagnostics @nodejs/async_hooks

mcollina

LGTM

mcollina · 2019-03-07T14:35:21Z

I've tagged this semver-major because of the impact it can have on async_hooks users. It's not technically required because async_hooks are experimental, but they are widely used.
I'm fine in being semver-minor with a dont-land-on-8.x and dont-land-on-10.x label as well.

Change resource being used, previously HTTParser was being reused. We are now using IncomingMessage and ClientRequest objects. The goal here is to make the async resource unique for each async operatio Refs: #24330 Refs: nodejs/diagnostics#248 Refs: #21313 Co-authored-by: Matheus Marchini <mat@mmarchini.me> PR-URL: #25094 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benedikt Meurer <benedikt.meurer@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Flarna · 2019-04-22T21:00:46Z

I hoped that this one fixes #26961 but actually it didn't. I think that init hook is not called in case a Parser is reused since this change.

Fix some issues introduced/not fixed via #25094: * Init hook is not emitted for a reused HTTPParser * HTTPParser was still used as resource in init hook * type used in init hook was always HTTPINCOMINGMESSAGE even for client requests * some tests have not been adapted to new resource names With this change the async hooks init event is emitted during a call to Initialize() as the type and resource object is available at this time. As a result Initialize() must be called now which could be seen as breaking change even HTTPParser is not part of documented API. It was needed to put the ClientRequest instance into a wrapper object instead passing it directly as async resource otherwise test-domain-multi fails. I think this is because adding an EventEmitter to a Domain adds a property 'domain' and the presence of this changes the context propagation in domains. Besides that tests still refering to resource HTTPParser have been updated/improved. Fixes: #27467Fixes: #26961 Refs: #25094 PR-URL: #27477 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Avoid that destroy hook is invoked twice - once via `Parser::Free()` and once again via `Parser::Reinitialize()` by clearing the async_id in `EmitDestroy()`. Partial backport of nodejs#27477, a full backport would require also nodejs#25094 which has a dont-land-on-v10.x label on it. Fixes: nodejs#26961

Avoid that destroy hook is invoked twice - once via `Parser::Free()` and once again via `Parser::Reinitialize()` by clearing the async_id in `EmitDestroy()`. Partial backport of #27477, a full backport would require also #25094 which has a dont-land-on-v10.x label on it. Fixes: #26961 Backport-PR-URL: #27986 PR-URL: #27477 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. labels Dec 17, 2018

mmarchini mentioned this pull request Dec 17, 2018
WIP http_parser: don't reuse HTTPParser #24330
Closed
3 tasks

Drieger force-pushed the avoid_reuse_of_HTTPParser branch from 368187e to 5d9eebdCompare December 17, 2018 19:13

mcollina approved these changes Dec 17, 2018
View reviewed changes

Trott reviewed Dec 18, 2018
View reviewed changes

Comment threadtest/async-hooks/test-httparser-reuse.js Outdated

AndreasMadsen reviewed Dec 18, 2018
View reviewed changes

Comment threadsrc/async_wrap.cc Outdated

Flarna reviewed Dec 19, 2018
View reviewed changes

mcollina reviewed Jan 15, 2019
View reviewed changes

mcollina mentioned this pull request Jan 28, 2019
process: improve nextTick performance #25461
Merged
2 tasks

mcollina added the semver-major PRs that contain breaking changes and should be released in the next major version. label Mar 7, 2019

mcollina approved these changes Mar 7, 2019
View reviewed changes

mmarchini closed this Apr 22, 2019

refack mentioned this pull request Apr 23, 2019
benchmark: fix http/bench-parser.js #27360
Closed
3 tasks

Flarna mentioned this pull request Apr 23, 2019
test: replace HTTPPARSER by new resource names #27372
Closed
3 tasks

This was referenced Apr 28, 2019
Asynchook is using destroyed AsyncId for new executions #27467
Closed
async_hooks: fixup do not reuse HTTPParser #27477
Closed

addaleax mentioned this pull request May 3, 2019
Domains async_hook based implementation leaks active domain on resources maintained by internal pools #27550
Closed

Flarna mentioned this pull request May 30, 2019
[v10.x] async_hooks: avoid double-destroy HTTPParser #27986
Closed
2 tasks

basti1302 mentioned this pull request Dec 28, 2019
test: correct isReused parameter in test-http-parser #23412
Closed
2 tasks

Flarna mentioned this pull request Sep 12, 2023
The http parser is not async, why does it hook into the async_hook machinery? nodejs/performance#108
Open

Flarna mentioned this pull request Apr 24, 2025
http: improve performance by removing async_hooks #57938
Draft

addaleax mentioned this pull request Aug 25, 2025
http: remove async tracking from http parser #59621
Open

Uh oh!

Uh oh!

Conversation

Drieger commented Dec 17, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

mmarchini commented Dec 17, 2018• edited by Trott Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmarchini commented Dec 17, 2018

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

Drieger commented Dec 17, 2018

Uh oh!

richardlau commented Dec 17, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Flarna commented Dec 17, 2018

Uh oh!

mscdex commented Dec 18, 2018• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

AndreasMadsen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Drieger commented Dec 19, 2018

Uh oh!

Flarna commented Dec 19, 2018

Uh oh!

FlarnaDec 19, 2018

Choose a reason for hiding this comment

Uh oh!

FlarnaJan 10, 2019

Choose a reason for hiding this comment

Uh oh!

DriegerJan 10, 2019

Choose a reason for hiding this comment

Uh oh!

DriegerJan 11, 2019

Choose a reason for hiding this comment

Uh oh!

Drieger commented Jan 9, 2019

Uh oh!

Flarna commented Jan 9, 2019

Uh oh!

Drieger commented Jan 9, 2019

Uh oh!

Drieger commented Jan 10, 2019

Uh oh!

Drieger commented Jan 14, 2019

Uh oh!

mcollinaJan 15, 2019

Choose a reason for hiding this comment

Uh oh!

DriegerJan 15, 2019

Choose a reason for hiding this comment

Uh oh!

mcollina commented Jan 17, 2019

Uh oh!

mcollina commented Jan 17, 2019

Uh oh!

mcollina commented Jan 19, 2019

Uh oh!

Drieger commented Mar 6, 2019

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

mcollina commented Mar 7, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Flarna commented Apr 22, 2019

Uh oh!

Reviewers

Assignees

Drieger commented Dec 17, 2018•
edited
Loading

mmarchini commented Dec 17, 2018•
edited by Trott
Loading

richardlau commented Dec 17, 2018•
edited
Loading

mscdex commented Dec 18, 2018•
edited
Loading

mcollina commented Mar 7, 2019•
edited
Loading