Skip to content

src: avoid deferred gc/cleanup for Buffer.from - #38337

Closed
jasnell wants to merge 2 commits into
nodejs:masterfrom
jasnell:avoid-deferred-cleanup-for-buffer-fro
Closed

src: avoid deferred gc/cleanup for Buffer.from#38337
jasnell wants to merge 2 commits into
nodejs:masterfrom
jasnell:avoid-deferred-cleanup-for-buffer-fro

Conversation

@jasnell

@jasnelljasnell commented Apr 21, 2021

Copy link
Copy Markdown
Member

/cc @addaleax@Daninet

Previously, the code path would allocated a tracked ArrayBuffer that defers cleanup and deallocation of the underlying data with a native SetImmediate. Avoid the unnecessary deferral by just allocating a BackingStore directly and writing into it.

This should be considered an alternative to #38336

Fixes: #38300
Refs: #38336

@jasnell
src: avoid deferred gc/cleanup for Buffer.from
6805449
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: nodejs#38300
Refs: nodejs#38336
@nodejs-github-botnodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Apr 21, 2021
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@jasnelljasnell mentioned this pull request Apr 21, 2021
Closed
addaleax
addaleax approved these changes Apr 22, 2021
View reviewed changes
Comment threadsrc/node_buffer.cc Outdated
@jasnell@addaleax
[Squash] nit
f805a1e
Co-authored-by: Anna Henningsen <anna@addaleax.net>
@jasnelljasnell mentioned this pull request Apr 22, 2021
Closed
XadillaX
XadillaX approved these changes Apr 22, 2021
View reviewed changes

@XadillaXXadillaX left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After testing:

conststr='x'.repeat(10000);console.log('start',process.memoryUsage());console.time('1');for(leti=0;i<1e6;i++){Buffer.from(str);// memory leak happens hereif(i%1e5===0){// await wait(100);gc();console.log('step',i,'rss',process.memoryUsage().rss);}}console.timeEnd('1');gc();console.log('end',process.memoryUsage());

The new PR #38337 runs about 3.9s in my computer. And 39f42d1a2d (before using BackingStore) runs about 3.7s. I think this performance loss is acceptable.

mcollina
mcollina approved these changes Apr 22, 2021
View reviewed changes

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@jasnelljasnell added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 22, 2021
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot

nodejs-github-bot commented Apr 23, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/37635/

@jasnell

jasnell commented Apr 23, 2021

Copy link
Copy Markdown
MemberAuthor

@nodejs/build ... is there a known issue with the node-test-commit-arm-fanned cluster? It is failing consistently on multiple PRs

@richardlau

richardlau commented Apr 23, 2021

Copy link
Copy Markdown
Member

@nodejs/build ... is there a known issue with the node-test-commit-arm-fanned cluster? It is failing consistently on multiple PRs

I cleaned up leftover node process on three of the pi 2's this morning. AFAIK things should be okay now.

@nodejs-github-bot

nodejs-github-bot commented Apr 23, 2021

Copy link
Copy Markdown
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/37643/

@jasnell

jasnell commented Apr 23, 2021

Copy link
Copy Markdown
MemberAuthor

@richardlau ... no joy it seems. See https://ci.nodejs.org/job/node-test-binary-arm-12+/10459/RUN_SUBSET=1,label=pi2-docker/console for #38331, for instance.

targos pushed a commit that referenced this pull request Apr 24, 2021
@jasnell@targos
src: avoid deferred gc/cleanup for Buffer.from
720fdf2
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: #38300
Refs: #38336
PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targos

targos commented Apr 24, 2021

Copy link
Copy Markdown
Member

Landed in 720fdf2

@targostargos closed this Apr 24, 2021
targos pushed a commit that referenced this pull request Apr 29, 2021
@jasnell@targos
src: avoid deferred gc/cleanup for Buffer.from
3649ec5
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: #38300
Refs: #38336
PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targostargos mentioned this pull request May 3, 2021
Merged
targos pushed a commit that referenced this pull request May 30, 2021
@jasnell@targos
src: avoid deferred gc/cleanup for Buffer.from
c0aa3f0
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: #38300
Refs: #38336
PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos pushed a commit that referenced this pull request Jun 5, 2021
@jasnell@targos
src: avoid deferred gc/cleanup for Buffer.from
ebf79c6
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: #38300
Refs: #38336
PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targostargos mentioned this pull request Jun 6, 2021
Merged
targos pushed a commit that referenced this pull request Jun 11, 2021
@jasnell@targos
src: avoid deferred gc/cleanup for Buffer.from
e279b02
Previously, the code path would allocated a tracked ArrayBuffer
that defers cleanup and deallocation of the underlying data with
a SetImmediate. Avoid the unnecessary deferral by just allocating
a `BackingStore` directly and writing into it.
Fixes: #38300
Refs: #38336
PR-URL: #38337
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcollinamcollinamcollina approved these changes
@addaleaxaddaleaxaddaleax approved these changes
@joyeecheungjoyeecheungjoyeecheung approved these changes
+1 more reviewer
@XadillaXXadillaXXadillaX approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.bufferIssues and PRs related to the buffer subsystem.c++Issues and PRs that require attention from people who are familiar with C++.needs-ciPRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Buffer.from(str) memory leak

8 participants

@jasnell@nodejs-github-bot@richardlau@targos@mcollina@addaleax@XadillaX@joyeecheung