Skip to content

util: improve util.inspect performance - #14492

Closed
BridgeAR wants to merge 6 commits into
nodejs:masterfrom
BridgeAR:fix-util-format
Closed

util: improve util.inspect performance#14492
BridgeAR wants to merge 6 commits into
nodejs:masterfrom
BridgeAR:fix-util-format

Conversation

@BridgeAR

Copy link
Copy Markdown
Member

Fixes#14487

A hole in a sparse array is now in itself a O(1) operation instead of O(n) where n is the number of coherent elements of the hole.

In addition a hasOwnProperty check was replaced with a cheaper one.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

util

This is a huge performance improvement in case of sparse arrays
when using util.inspect as the hole will simple be skipped.
@nodejs-github-botnodejs-github-bot added the util Issues and PRs related to the built-in util module. label Jul 26, 2017
@tniessen

Copy link
Copy Markdown
Member

Comment threadlib/util.js Outdated
index++;
index = i;
visibleLength++;
if (visibleLength === ctx.maxArrayLength)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't these previous two lines be merged into if (++visibleLength === ctx.maxArrayLength) ?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@mscdexmscdex added the performance Issues and PRs related to the performance of Node.js. label Jul 26, 2017
Comment threadlib/util.js
break;
const i = +elem;
if (index !== i) {
if (i > 0 === false)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment on why this is necessary, and different from i <= 0? (I understand it because I saw an earlier version of the PR, but comments are more for future than now anyway.)

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

@BridgeAR

Copy link
Copy Markdown
MemberAuthor

Comments addressed

@aqrln

Copy link
Copy Markdown
Contributor

Thanks for addressing the issue! I made some benchmarks back when it was opened, so feel free to include them in your pull request if you wish: https://gist.github.com/aqrln/195743d38f632b5b9a8d5944f30cb6af

Speaking of which, I observe a performance regression with dense arrays:

 improvement confidence p.value
util/inspect-array.js len=100000 n=1000 -35.07 % *** 1.862234e-35
util/inspect-sparse-array.js len=100000 n=1000 2662.66 % *** 1.516381e-41

Can you reproduce it or is it just me (the machine was a bit loaded with other stuff, so the result might be unreliable)?

Comment threadlib/util.js
}
}
if (!hasOwnProperty(visibleKeys, key)) {
if (visibleKeys[key] === undefined) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be worth to eventually make visibleKeys an ES2015 Set instead of emulating it via an object, but I don't know how it will affect performance with the current V8 (if it will at all).

@BridgeARBridgeARJul 28, 2017

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I looked at the code there were a couple more optimizations possible but I tried to be more surgical here. E.g. for arrays there is no need for Object.keys when using for in and ctx.seen should be a Set instead of an Array.

@BridgeAR

BridgeAR commented Jul 28, 2017

Copy link
Copy Markdown
MemberAuthor

@aqrln thanks for the benchmarks. I saw some minor drop as well and changed it now. With for in there was also a minor regression (a edge case) and I added a test for that.

Current benchmark:

 util/inspect-array.js type="denseArray" len=100000 n=100 1.09 % 1.853220e-01
util/inspect-array.js type="mixedArray" len=100000 n=100 2596.93 % *** 8.927819e-18
util/inspect-array.js type="sparseArray" len=100000 n=100 3455.27 % *** 1.563057e-17

@aqrln

Copy link
Copy Markdown
Contributor

@BridgeAR

Copy link
Copy Markdown
MemberAuthor

PTAL

@refackrefack self-assigned this Aug 13, 2017
@refack

refack commented Aug 13, 2017

Copy link
Copy Markdown
Contributor

Landed in 95bbb68
Extra sanity test of master: https://ci.nodejs.org/job/node-test-commit-linuxone/7938/ ✔️

@refackrefack closed this Aug 13, 2017
refack pushed a commit that referenced this pull request Aug 13, 2017
* improve util.inspect performance
This is a huge performance improvement in case of sparse arrays
when using util.inspect as the hole will simple be skipped.
* use faster visibleKeys property lookup
* add inspect-array benchmark
PR-URL: #14492Fixes: #14487
Reviewed-By: Alexey Orlenko <eaglexrlnk@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
addaleax pushed a commit that referenced this pull request Aug 13, 2017
* improve util.inspect performance
This is a huge performance improvement in case of sparse arrays
when using util.inspect as the hole will simple be skipped.
* use faster visibleKeys property lookup
* add inspect-array benchmark
PR-URL: #14492Fixes: #14487
Reviewed-By: Alexey Orlenko <eaglexrlnk@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
@addaleaxaddaleax mentioned this pull request Aug 13, 2017
@MylesBorinsMylesBorins added the baking-for-lts PRs that need to wait before landing in a LTS release. label Sep 20, 2017
@MylesBorins

Copy link
Copy Markdown
Contributor

I'm not sure if we want to backport this to v6.x, but I've put it under baking

@BridgeAR

Copy link
Copy Markdown
MemberAuthor

@MylesBorins I think it should be backported - especially as there are other commits that should also be backported at some point and they depend on this. But when this is backported #14880 has to be backported as well due to the minor regression introduced here.

@BridgeAR

Copy link
Copy Markdown
MemberAuthor

I think enough time has passed to backport this. From this change itself there was no further complain about any regression. What do you think @MylesBorins ?

@MylesBorinsMylesBorins removed the baking-for-lts PRs that need to wait before landing in a LTS release. label Aug 17, 2018
@refackrefack removed their assignment Oct 12, 2018
@BridgeAR
BridgeAR deleted the fix-util-format branch April 1, 2019 23:36
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performanceIssues and PRs related to the performance of Node.js.utilIssues and PRs related to the built-in util module.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants

@BridgeAR@tniessen@aqrln@refack@MylesBorins@mscdex@TimothyGu@nodejs-github-bot