Uh oh!
There was an error while loading. Please reload this page.
benchmark: add benchmark for coverage - #36972
Closed
bcoe wants to merge 1 commit into
Closed
Conversation
bcoe
commented
Jan 17, 2021
ContributorAuthor
@joyeecheung I was thinking this benchmark would be a really good use of |
Trott
reviewed
Jan 17, 2021
Uh oh!
There was an error while loading. Please reload this page.
Trott
approved these changes
Jan 17, 2021
Trott
commented
Jan 17, 2021
Member
Subsystem in commit message should be |
benjamingr
approved these changes
Jan 18, 2021
bcoeforce-pushed
the
benchmark-coverage
branch
2 times, most recently
from
January 18, 2021 19:03
c5507d4 to
b5eadb0Comparejasnell
approved these changes
Jan 18, 2021
bcoe
commented
Jan 18, 2021
ContributorAuthor
@Trott refactored so that the benchmark is easier to use, it now spawns a subprocess with coverage enabled (_so no need to create a separate bin) 👍 |
nodejs-github-bot
commented
Jan 18, 2021
Collaborator
joyeecheung
approved these changes
Jan 19, 2021
Uh oh!
There was an error while loading. Please reload this page.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
nodejs-github-bot
commented
Jan 22, 2021
Collaborator
Uh oh!
There was an error while loading. Please reload this page.
richardlau
approved these changes
Jan 22, 2021
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Collaborator
bcoe pushed a commit
that referenced
this pull request
Jan 27, 2021
PR-URL: #36972 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>
bcoe
commented
Jan 27, 2021
ContributorAuthor
Landed in 8c9dc4e |
targos pushed a commit
that referenced
this pull request
Feb 2, 2021
PR-URL: #36972 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>
targos pushed a commit
that referenced
this pull request
May 1, 2021
PR-URL: #36972 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com> Reviewed-By: Richard Lau <rlau@redhat.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Working on #36956, I became a bit worried that as we extend code coverage to support more types of branching, performance might start to become unacceptable.
I created this benchmark to experiment:
Node 14.15.4, NODE_V8_COVERAGE, vs., no coverage:
process/coverage.js n=100000 *** -18.94 % ±0.77% ±1.02% ±1.30%Node 15.6.0, NODE_V8_COVERAGE, vs., no coverage:
process/coverage.js n=100000 *** -17.95 % ±1.08% ±1.43% ±1.83%Coverage with #36956, vs., no coverage:
process/coverage.js n=100000 *** -19.37 % ±1.36% ±1.79% ±2.30%To Summarize
In the current release of 14/15, we see that running Node with coverage decreases performance by about 18%.
With the introduction of optional chaining in this benchmark, we see a 1 - 2% additional hit to performance.
Other interesting facts
coverage.jsusing Istanbul, vs., NODE_V8_COVERAGE, it was 37.46% slower than no coverage.0.37 % ±1.14% ±1.50% ±1.92%).Conclusions
CC: @schuay