events: improve performance by ~120%

[anonrig]

Object.create(null) is 20x slower on Node 14, and 10x slower on Node 18. This is mainly related to the bug I've opened: https://bugs.chromium.org/p/v8/issues/detail?id=13273#c2

V8 is super optimized for new Cls(), but not for Object.create(null)

events/ee-add-remove.js n=1000000 removeListener=0 newListener=0 *** 18.89 % ±0.88% ±1.17% ±1.52%
events/ee-add-remove.js n=1000000 removeListener=0 newListener=1 *** 13.76 % ±1.23% ±1.64% ±2.15%
events/ee-add-remove.js n=1000000 removeListener=1 newListener=0 *** 13.46 % ±1.08% ±1.44% ±1.89%
events/ee-add-remove.js n=1000000 removeListener=1 newListener=1 *** 3.88 % ±1.05% ±1.40% ±1.84%
events/ee-emit.js listeners=1 argc=0 n=2000000 *** 116.87 % ±2.80% ±3.77% ±4.98%
events/ee-emit.js listeners=1 argc=10 n=2000000 *** 114.85 % ±5.35% ±7.20% ±9.56%
events/ee-emit.js listeners=1 argc=2 n=2000000 *** 114.53 % ±3.65% ±4.92% ±6.52%
events/ee-emit.js listeners=1 argc=4 n=2000000 *** 119.26 % ±1.67% ±2.23% ±2.93%
events/ee-emit.js listeners=10 argc=0 n=2000000 *** 9.36 % ±1.44% ±1.91% ±2.49%
events/ee-emit.js listeners=10 argc=10 n=2000000 *** 8.07 % ±1.19% ±1.59% ±2.07%
events/ee-emit.js listeners=10 argc=2 n=2000000 *** 9.76 % ±1.08% ±1.45% ±1.90%
events/ee-emit.js listeners=10 argc=4 n=2000000 *** 8.57 % ±1.16% ±1.55% ±2.01%
events/ee-emit.js listeners=5 argc=0 n=2000000 *** 14.15 % ±1.51% ±2.02% ±2.63%
events/ee-emit.js listeners=5 argc=10 n=2000000 *** 12.37 % ±1.34% ±1.78% ±2.32%
events/ee-emit.js listeners=5 argc=2 n=2000000 *** 13.79 % ±1.42% ±1.89% ±2.47%
events/ee-emit.js listeners=5 argc=4 n=2000000 *** 11.68 % ±3.91% ±5.25% ±6.93%
events/ee-listener-count-on-prototype.js n=50000000 *** -9.40 % ±1.30% ±1.75% ±2.32%
events/ee-listeners.js raw='false' listeners=5 n=5000000 *** -5.34 % ±2.32% ±3.12% ±4.12%
events/ee-listeners.js raw='false' listeners=50 n=5000000 * 1.49 % ±1.16% ±1.55% ±2.03%
events/ee-listeners.js raw='true' listeners=5 n=5000000 *** -7.73 % ±1.63% ±2.19% ±2.89%
events/ee-listeners.js raw='true' listeners=50 n=5000000 *** -5.43 % ±0.91% ±1.21% ±1.59%
events/ee-once.js argc=0 n=20000000 *** 16.98 % ±1.47% ±1.95% ±2.55%
events/ee-once.js argc=1 n=20000000 *** 16.58 % ±1.34% ±1.79% ±2.34%
events/ee-once.js argc=4 n=20000000 *** 15.86 % ±1.17% ±1.57% ±2.06%
events/ee-once.js argc=5 n=20000000 *** 15.28 % ±1.63% ±2.17% ±2.84%
events/eventtarget.js listeners=1 n=1000000 -0.15 % ±0.46% ±0.62% ±0.80%
events/eventtarget.js listeners=10 n=1000000 0.01 % ±0.50% ±0.66% ±0.87%
events/eventtarget.js listeners=5 n=1000000 -0.45 % ±0.54% ±0.73% ±0.95%
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 28 comparisons, you can thus expect the following amount of false-positive results:
1.40 false positives, when considering a 5% risk acceptance (*, **, ***),
0.28 false positives, when considering a 1% risk acceptance (**, ***),
0.03 false positives, when considering a 0.1% risk acceptance (***)

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/46570/

[ThePrimeagen]

this is extremely interesting! well done on the find

[lpinca]

Duplicate of #39939.

[mcollina]

lgtm

the previous PR seems to be stalled

[mscdex]

I think a key benchmark that should be added and checked here is one that uses a configurable number of unique event handlers to potentially highlight the difference between "fast mode" (best when very few properties added to it) and "slow mode" objects (best when lots of properties added to it), kinda like benchmark/es/map-bench.js but with key names that are not dependent upon the number of loop iterations.

[bmeck]

Beware this causes a shared prototype { proto: null } instead of object.create doesn't and should be benchmarks as it does change behavior however slight

[mscdex]

It should also be noted that we used to use this exact same storage method until #11930.

[ycjcl868]

lgtm

[aduh95]

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1191/

Results

 confidence improvement accuracy (*) (**) (***)
events/ee-add-remove.js n=1000000 removeListener=0 newListener=0 *** 11.10 % ±3.59% ±4.82% ±6.35%
events/ee-add-remove.js n=1000000 removeListener=0 newListener=1 *** 24.50 % ±3.28% ±4.37% ±5.72%
events/ee-add-remove.js n=1000000 removeListener=1 newListener=0 *** 8.60 % ±4.33% ±5.81% ±7.64%
events/ee-add-remove.js n=1000000 removeListener=1 newListener=1 3.22 % ±3.94% ±5.25% ±6.84%
events/ee-emit.js listeners=10 argc=0 n=2000000 *** 6.57 % ±1.54% ±2.05% ±2.66%
events/ee-emit.js listeners=10 argc=10 n=2000000 *** 8.54 % ±1.18% ±1.57% ±2.04%
events/ee-emit.js listeners=10 argc=2 n=2000000 *** 6.27 % ±1.37% ±1.82% ±2.37%
events/ee-emit.js listeners=10 argc=4 n=2000000 *** 9.21 % ±1.36% ±1.81% ±2.37%
events/ee-emit.js listeners=1 argc=0 n=2000000 *** 146.74 % ±7.88% ±10.54% ±13.83%
events/ee-emit.js listeners=1 argc=10 n=2000000 *** 141.56 % ±9.43% ±12.60% ±16.52%
events/ee-emit.js listeners=1 argc=2 n=2000000 *** 155.60 % ±9.92% ±13.31% ±17.55%
events/ee-emit.js listeners=1 argc=4 n=2000000 *** 166.25 % ±21.89% ±29.45% ±39.02%
events/ee-emit.js listeners=5 argc=0 n=2000000 *** 13.69 % ±3.19% ±4.24% ±5.52%
events/ee-emit.js listeners=5 argc=10 n=2000000 *** 6.82 % ±3.51% ±4.67% ±6.09%
events/ee-emit.js listeners=5 argc=2 n=2000000 *** 11.49 % ±3.55% ±4.74% ±6.20%
events/ee-emit.js listeners=5 argc=4 n=2000000 *** 6.43 % ±2.49% ±3.31% ±4.31%
events/ee-listener-count-on-prototype.js n=50000000 *** -7.27 % ±2.11% ±2.81% ±3.66%
events/ee-listeners.js raw='false' listeners=50 n=5000000 -0.75 % ±1.79% ±2.39% ±3.13%
events/ee-listeners.js raw='false' listeners=5 n=5000000 -1.80 % ±3.50% ±4.67% ±6.13%
events/ee-listeners.js raw='true' listeners=50 n=5000000 * -2.14 % ±2.00% ±2.67% ±3.48%
events/ee-listeners.js raw='true' listeners=5 n=5000000 ** -6.56 % ±3.94% ±5.24% ±6.83%
events/ee-once.js argc=0 n=20000000 *** 19.11 % ±2.15% ±2.88% ±3.78%
events/ee-once.js argc=1 n=20000000 *** 19.71 % ±2.23% ±2.97% ±3.88%
events/ee-once.js argc=4 n=20000000 *** 18.59 % ±1.28% ±1.71% ±2.23%
events/ee-once.js argc=5 n=20000000 *** 19.69 % ±1.70% ±2.27% ±2.97%
events/eventtarget.js listeners=10 n=1000000 0.74 % ±5.87% ±7.81% ±10.18%
events/eventtarget.js listeners=1 n=1000000 -4.58 % ±4.94% ±6.62% ±8.71%
events/eventtarget.js listeners=5 n=1000000 -1.30 % ±2.18% ±2.90% ±3.79%
Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 28 comparisons, you can thus
expect the following amount of false-positive results:
1.40 false positives, when considering a 5% risk acceptance (*, **, ***),
0.28 false positives, when considering a 1% risk acceptance (**, ***),
0.03 false positives, when considering a 0.1% risk acceptance (***)

[mscdex]

I created a benchmark as described in #44627 (comment) and the results are basically what I suspected:

 confidence improvement accuracy (*) (**) (***)
events/ee-add-remove-multiple.js n=1000000 events=1 *** 79.91 % ±2.11% ±2.83% ±3.74%
events/ee-add-remove-multiple.js n=1000000 events=2 *** -14.78 % ±0.88% ±1.18% ±1.54%
events/ee-add-remove-multiple.js n=1000000 events=3 *** -10.03 % ±1.25% ±1.66% ±2.17%
events/ee-add-remove-multiple.js n=1000000 events=5 *** -7.24 % ±1.12% ±1.49% ±1.95%

Code

'use strict';constcommon=require('../common.js');const{ EventEmitter }=require('events');constbench=common.createBenchmark(main,{events: [1,2,3,5],n: [1e6],});functionmain({ events, n }){constee=newEventEmitter();constlisteners=[];for(letk=0;k<10;k+=1)listeners.push(()=>{});consteventNames=[];for(letk=0;k<events;++k)eventNames.push(`dummy${k}`);bench.start();for(leti=0;i<n;i+=1){for(consteventNameofeventNames){for(letk=listeners.length;--k>=0;/* empty */){ee.on(eventName,listeners[k]);}}for(consteventNameofeventNames){for(letk=listeners.length;--k>=0;/* empty */){ee.removeListener(eventName,listeners[k]);}}}bench.end(n);}

In this case it only takes two unique events for the regression to show up, which is a bit surprising as I thought it used to take more for V8 to transition objects to slow mode. After seeing these results, I don't think the changes in this PR are worth it, considering most event emitters will see handlers added for multiple events and not just one.

[anonrig]

@mscdex I basically run the same benchmark on my machine (M1 Max, 64 GB, arm64), and added more values to events (10 and 20), and my results are a lot different than yours:

 confidence improvement accuracy (*) (**) (***)
events/ee-listen-unique.js n=1000000 events=1 *** 75.80 % ±1.42% ±1.89% ±2.46%
events/ee-listen-unique.js n=1000000 events=10 * -2.07 % ±1.83% ±2.43% ±3.17%
events/ee-listen-unique.js n=1000000 events=2 *** -4.48 % ±1.03% ±1.37% ±1.80%
events/ee-listen-unique.js n=1000000 events=20 0.56 % ±3.92% ±5.22% ±6.79%
events/ee-listen-unique.js n=1000000 events=3 * -1.91 % ±1.55% ±2.06% ±2.69%
events/ee-listen-unique.js n=1000000 events=5 *** -3.86 % ±1.73% ±2.30% ±2.99%
Be aware that when doing many comparisons the risk of a false-positive result increases.
In this case, there are 6 comparisons, you can thus expect the following amount of false-positive results:
0.30 false positives, when considering a 5% risk acceptance (*, **, ***),
0.06 false positives, when considering a 1% risk acceptance (**, ***),
0.01 false positives, when considering a 0.1% risk acceptance (***)

[mscdex]

I ran my benchmarks on an i7-3770K running Linux 5.4.0 (and compared using benchmark/compare.R if that makes a difference) 🤷

I also ran with larger values for events but the trend is pretty clear anyway.

[anonrig]

I also ran with larger values for events but the trend is pretty clear anyway.

I agree. I will continue working on this. Can you share with me more information about the fast/slow objects in the v8? That would be educational for everybody.

[devsnek]

v8 tries to optimize access to objects based on their "shape". If the shape changes too much, v8 gives up trying to optimize it and switches into a fallback "dictionary" mode, which is basically just a raw hash map. The properties of an object and the types of the values make up the shape (to some degree, check out https://mathiasbynens.be/notes/shapes-ics for an in depth explanation), so adding new properties to an object (let x = {}; x.a = 1;) or deleting properties (let x = {a: 1}; delete x.a;), among other things, will eventually make v8 give up.

_{in bad hacky idea land, we could manually try calling %ToFastProperties on eventemitters to make them "recover" from the slow state, though it would be difficult to know when it was a good time to call it.}

[aduh95]

In any case, we may want to land the benchmark, that seems a good idea to have one to catch those kind of perf regression in the future.

[mcollina]

This should not land due to regressions.

[jasnell]

This change is fairly problematic, I think. At the very least it's semver-major. Unfortunately we have a number of ecosystem modules that touch this._events directly and changing the type of thing that is can definitely break stuff.

[iambumblehead]

I don't want to be rude. I would be grateful for any input from EventEmitter experts here.

I updated the benchmark script attached in this thread to determine if it is reasonable to publish and subscribe thousands of different events on an EventEmitter instance (of course there is a warning when more than 10 event ids are attached).

It looks like this is a terrible idea, but if anyone here would confirm that it is terrible, it would close the case for me once and for all. Thank you for any reply.