Skip to content

assert: improve partialDeepStrictEqual performance and add benchmark - #56555

Closed
puskin wants to merge 1 commit into
nodejs:mainfrom
puskin:partial-deep-strict-equal-perf
Closed

assert: improve partialDeepStrictEqual performance and add benchmark#56555
puskin wants to merge 1 commit into
nodejs:mainfrom
puskin:partial-deep-strict-equal-perf

Conversation

@puskin

Copy link
Copy Markdown
Contributor

now that #54630 has landed, I took the liberty to review the code I wrote to look for improvements that would squeeze some more performance out of the comparison mechanism

  • simplified some logic
  • reviewed loops and introduced more breaking points to make them run for less cycles
  • added a benchmark file to test future possible regressions when it comes to partialDeepStrictEqual

Refs: #54630

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance

@nodejs-github-botnodejs-github-bot added assert Issues and PRs related to the assert subsystem. needs-ci PRs that need a full CI run. labels Jan 10, 2025
@codecov

codecovBot commented Jan 10, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 57.74648% with 30 lines in your changes missing coverage. Please review.

Project coverage is 90.23%. Comparing base (6b3937a) to head (4502b76).
Report is 515 commits behind head on main.

Files with missing linesPatch %Lines
lib/assert.js57.74%30 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #56555 +/- ##
==========================================
+ Coverage 89.17% 90.23% +1.05% 
==========================================
Files 662 630 -32 Lines 191672 184968 -6704 Branches 36884 36192 -692 ==========================================
- Hits 170922 166904 -4018 + Misses 13614 11084 -2530 + Partials 7136 6980 -156 
Files with missing linesCoverage Δ
lib/assert.js96.50% <57.74%> (-2.54%)⬇️

... and 236 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

BridgeAR
BridgeAR previously requested changes Jan 13, 2025
Comment threadlib/assert.js Outdated
Comment threadlib/assert.js Outdated
Comment threadlib/assert.js Outdated
Comment threadlib/assert.js Outdated
Comment threadlib/assert.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from ac146c0 to 762399fCompareJanuary 13, 2025 15:05
@puskin
puskin requested a review from BridgeARJanuary 17, 2025 08:44
@puskin

Copy link
Copy Markdown
ContributorAuthor

Is anything else needed here?

@lemire

Copy link
Copy Markdown
Member

@puskin94 Can you share the results of your benchmarking?

@puskin

puskin commented Jan 26, 2025

Copy link
Copy Markdown
ContributorAuthor

@puskin94 Can you share the results of your benchmarking?

of course 😄

➜ node git:(partial-deep-strict-equal-perf) ./node --no-warnings benchmark/assert/partial-deep-strict-equal.js assert/partial-deep-strict-equal.js datasetName="objects" size=1000 n=50: 169.87881572367604
assert/partial-deep-strict-equal.js datasetName="sets" size=1000 n=50: 141.24912248982653
assert/partial-deep-strict-equal.js datasetName="maps" size=1000 n=50: 151.36189759383777
assert/partial-deep-strict-equal.js datasetName="arrayBuffers" size=1000 n=50: 14.16071061277997
assert/partial-deep-strict-equal.js datasetName="dataViewArrayBuffers" size=1000 n=50: 16.398050206238278

Comment threadlib/assert.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from 762399f to d061759CompareJanuary 27, 2025 11:08
Comment threadlib/assert.js Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're looking for performance, we should probably be using a classic for(;;) loop. I'd recommend rolling back this change (i.e. put back the safeExpected assignment, because it has nothing to do with performance) for this PR, and make a new one switching to a for(;;) loop on which we can run benchmarks

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you do this tho you would have to call the ArrayFrom primordial. The performance enhancement was to adjust the "low hanging fruit" stuff I could get, "structure" wise, without influencing readability. IMHO the current implementation is better, but happy to change if you don't agree!

reverted the assignment tho

Comment threadbenchmark/assert/partial-deep-strict-equal.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from d061759 to f1f220cCompareJanuary 27, 2025 12:25
Comment threadlib/assert.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from f1f220c to e932899CompareJanuary 29, 2025 08:00
@puskin

Copy link
Copy Markdown
ContributorAuthor

Is anything else needed here?

@puskin

Copy link
Copy Markdown
ContributorAuthor

is any change required here?

@aduh95aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Feb 21, 2025
@aduh95
aduh95 requested a review from BridgeARFebruary 21, 2025 16:32
@aduh95

Copy link
Copy Markdown
Contributor

@BridgeAR can you clarify whether you're still blocking?

@github-actionsgithub-actionsBot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 21, 2025
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollinamcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@BridgeARBridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late reply. I guess this improves a couple of cases, so that's great! :)
I believe there is more room for improvement by reusing the existing implementation. That could be used for a follow-up PR.
Before landing: could we run the benchmark to see the improvement as percentage? :)

Comment threadlib/assert.js Outdated
Comment threadlib/assert.js Outdated
@BridgeAR
BridgeAR dismissed their stale reviewFebruary 27, 2025 16:20

Just did a new review

@aduh95

aduh95 commented Feb 28, 2025

Copy link
Copy Markdown
Contributor

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1664/

Results
 confidence improvement accuracy (*) (**) (***)
assert/partial-deep-strict-equal.js datasetName='arrayBuffers' size=1000 n=50 *** -0.79 % ±0.38% ±0.51% ±0.67%
assert/partial-deep-strict-equal.js datasetName='dataViewArrayBuffers' size=1000 n=50 ** 0.64 % ±0.45% ±0.60% ±0.78%
assert/partial-deep-strict-equal.js datasetName='maps' size=1000 n=50 *** 0.56 % ±0.26% ±0.35% ±0.46%
assert/partial-deep-strict-equal.js datasetName='objects' size=1000 n=50 *** -0.49 % ±0.28% ±0.37% ±0.48%
assert/partial-deep-strict-equal.js datasetName='sets' size=1000 n=50 * 0.34 % ±0.26% ±0.35% ±0.45%
Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 5 comparisons, you can thus
expect the following amount of false-positive results:
0.25 false positives, when considering a 5% risk acceptance (*, **, ***),
0.05 false positives, when considering a 1% risk acceptance (**, ***),
0.01 false positives, when considering a 0.1% risk acceptance (***)

@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from e932899 to a07cf19CompareFebruary 28, 2025 10:43
@puskin

Copy link
Copy Markdown
ContributorAuthor

I must have messed up something with my local codebase. I just pushed again with new performance improvements, this should make some difference

Comment threadlib/assert.js Outdated
@aduh95

Copy link
Copy Markdown
Contributor

@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from a07cf19 to b53b85cCompareFebruary 28, 2025 11:21
Comment threadlib/assert.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from b53b85c to 989b529CompareFebruary 28, 2025 11:43
@aduh95

Copy link
Copy Markdown
Contributor

@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from 989b529 to 2e4fbdbCompareFebruary 28, 2025 13:27
@BridgeAR

Copy link
Copy Markdown
Member

@BridgeARBridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not mistaken, this mainly improves the performance for sets and maps that have mixed entries (primitives + objects). The current benchmark has no such entry.

I also guess that the primordial usage of the iterator is costly. Could you check if that's the case?

Comment threadlib/assert.js Outdated
@puskin
puskinforce-pushed the partial-deep-strict-equal-perf branch from 2e4fbdb to 4502b76CompareMarch 3, 2025 22:28
@BridgeAR

Copy link
Copy Markdown
Member

Latest benchmark https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1671

(*) (**) (***)
00:25:02 assert/partial-deep-strict-equal.js datasetName='arrayBuffers' size=1000 n=10 *** 0.67 % ±0.37% ±0.49% ±0.64%
00:25:02 assert/partial-deep-strict-equal.js datasetName='arrayBuffers' size=1000 n=200 *** 0.87 % ±0.33% ±0.45% ±0.58%
00:25:02 assert/partial-deep-strict-equal.js datasetName='arrayBuffers' size=1000 n=50 *** 1.01 % ±0.40% ±0.54% ±0.70%
00:25:02 assert/partial-deep-strict-equal.js datasetName='circularRefs' size=1000 n=200 *** -0.75 % ±0.38% ±0.50% ±0.65%
00:25:02 assert/partial-deep-strict-equal.js datasetName='maps' size=1000 n=50 *** -0.48 % ±0.27% ±0.36% ±0.46%
00:25:02 assert/partial-deep-strict-equal.js datasetName='objects' size=1000 n=200 *** 0.48 % ±0.23% ±0.30% ±0.40%
00:25:02 assert/partial-deep-strict-equal.js datasetName='objects' size=1000 n=50 *** 0.73 % ±0.28% ±0.38% ±0.49%
00:25:02 assert/partial-deep-strict-equal.js datasetName='typedArrays' size=1000 n=50 *** 0.55 % ±0.31% ±0.41% ±0.53%

Only *** result listed. Looking at the changes and the benchmark outcome, I guess that most work is spend elsewhere. I suggest to land this as is nevertheless.

@aduh95

Copy link
Copy Markdown
Contributor

This needs a rebase

@aduh95aduh95 removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Mar 13, 2025
@BridgeAR

Copy link
Copy Markdown
Member

This was superseded by the other PR

@puskin

puskin commented Mar 13, 2025

Copy link
Copy Markdown
ContributorAuthor

superseeded by #57370

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

assertIssues and PRs related to the assert subsystem.needs-ciPRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@puskin@nodejs-github-bot@lemire@aduh95@BridgeAR@mcollina