EventEmitter: use a Map to store event listeners

[targos]

This is a fix for #728.

I ran the events benchmarks to compare this with master and there seems to be a significant performance drop for some cases (though I'm not sure of how I should compare the runs).

Bench	master	Map
ee-add-remove	1216157	950515 (-22%)
ee-emit-multi-args	5909434	5640342 (-5%)
ee-emit	8485253	7912000 (-7%)
ee-listener-count	432010872	56945905 (-87%)
ee-listeners-many	3558539	3490269 (-2%)
ee-listeners	21596479	18429471 (-15%)

[petkaantonov]

In the results you posted Map is always performing worse?

[mscdex]

I believe all of these changes to readable stream are going to result in the same problem as when I proposed optimizations for eventEmitter.once() (#914), which is that some modules on npm have their readable-stream pinned to specific versions or limited version ranges. So if someone uses an older readable-stream on an io.js with these changes, things will break.

[ChALkeR]

@mscdex That would be a lot of someones.

[mscdex]

Last I checked npm (a few weeks ago) it was only about a dozen or so? I had actually thought about submitting PRs to those projects just so we could get the .once() optimizations.

[ChALkeR]

Btw, there are other modules that are using _events directly. For example:

• https://github.com/mscdex/dicer/blob/master/lib/Dicer.js#L91
• https://github.com/mscdex/busboy/blob/master/lib/types/multipart.js#L183
• https://github.com/unshiftio/ultron/blob/master/index.js#L79

That's in my current node_modules dir, except for the readable-stream.

[mscdex]

@ChALkeR I was referring more generally about changes to readable-stream and how modules are depending on it version-wise, not specific code changes. But yes, there are probably a ton of modules that use _events directly.

[ChALkeR]

Last I checked npm (a few weeks ago) it was only about a dozen or so?

Just in my current node_modules dir I have 19 of those. List:

bufferstreams "readable-stream": "^1.0.33"tar-stream "readable-stream": "~1.0.33",dicer "readable-stream": "1.1.x",duplexer2 "readable-stream": "~1.1.9"through2 "readable-stream": ">=1.0.33-1 <1.1.0-0",are-we-there-yet "readable-stream": "^1.1.13"decompress-zip "readable-stream": "^1.1.8",tar-pack "readable-stream": "~1.0.2",bl "readable-stream": "~1.0.26"readdirp "readable-stream": "~1.0.26-2"busboy "readable-stream": "1.1.x"raw-body "readable-stream": "~1.0.33",gulp-bower/through2 "readable-stream": ">=1.0.28 <1.1.0-0",htmlparser2 "readable-stream": "1.1"concat-stream "readable-stream": "~1.1.9",mongodb "readable-stream": "1.0.31"read-all-stream "readable-stream": "~1.1.13"finalhandler "readable-stream": "~1.0.33",duplexify "readable-stream": "^1.1.13"

[ChALkeR]

If this gets merged, will 1.0.* and 1.1.* branches of readable-stream receive patches?
If yes, it is true that there are not so many packages that specify a fixed version of readable-stream as a dependency.

[ChALkeR]

@petkaantonovMap should be faster than an Object here. I guess that's the time measurements in that table. But it would be better if @targos shared his tests.

[petkaantonov]

The benchmark gives results in op/s (higher is better).

[mscdex]

I'm curious about performance differences with this particular change. Is it faster to keep _eventsCount around and create a new Map()?

[targos]

There is no significant performance difference if I revert ca75072. I tested with this._events.clear() and this._events = new Map()

[ChALkeR]

Ah, that's the bundled benchmark from io.js/benchmark/events, I missed that. Something is not right here then.

[mscdex]

Is this necessary? Isn't there a way to create a Map from C++?

[targos]

Maybe. I couldn't find anything related to maps in the v8 API and I know that the Map constructor is written in JS but there may be a way to call it from C++

[brendanashworth]

I see the NativeWeakMap but I can't find the regular Map either.

[targos]

It will be possible with v8 4.5: v8/v8@395fa8b

[targos]

@petkaantonov is right, I assumed it was the time...
So yes, Map is always performing worse :(

[yosuke-furukawa]

ES6 Map is slower than Object props in V8.
Ref: https://code.google.com/p/v8/issues/detail?id=4014

And EventEmitter is our core module, if event emitter is slower, whole system may be slow on performance.

[ChALkeR]

@yosuke-furukawa This seems to depend on the usecase. I saw usecases where replacing objects with Maps resulted in a 2x speedup.

[yosuke-furukawa]

@ChALkeR Of course, the benchmark result is depend on the usecase. But current result seems to be worse. @petkaantonov is correct. our benchmark test shows ops per secondhttps://github.com/nodejs/io.js/blob/master/benchmark/common.js#L225 .

[ChALkeR]

@yosuke-furukawa, Yes, I know about the bundled benchmarks.
In the original OP post there was a claim that these numbers show a performance speedup, that's why I thought that this is not a bundled benchmark but something of his own that measures the time taken.

[Fishrock123]

Is prefixing events something that we can do without breaking anything? I think that would be a much better alternative. Map simply isn't fast enough yet.

[targos]

@yosuke-furukawa@Fishrock123 I agree that the current perf of Map is blocking this.
We can keep the PR open and re-run the benchmarks when there is a v8 upgrade to see if it's improving

[chrisdickinson]

It's going to be an uphill battle to directly change the format of _events, as @mscdex noted, at minimum making sure that both minor versions of readable-stream get a backwards-compatible patch release, and propagating that through the package ecosystem.

If the perf improves, I'd advise presenting a legacy _events interface via a getter and putting the map-backed interface behind a symbol or weakmap.

On May 24, 2015, at 11:12 AM, Jeremiah Senkpiel notifications@github.com wrote:

Is prefixing events something that we can do without breaking anything? I think that would be a much better alternative. Map simply isn't fast enough yet.

—
Reply to this email directly or view it on GitHub.

[ChALkeR]

Btw, about the _events usage — it looks like a very bad thing to me that modules use that.
It probably means that some of the API methods that should be there are missing.
Maybe the EventEmmiter api should be extended to cover those use-cases?

Samples:

• readable-stream/lib/_stream_readable.js:
  1. if (!dest._events || !dest._events.error)
  2. else if (isArray(dest._events.error))
  3. dest._events.error.unshift(onerror);
  4. dest._events.error = [onerror, dest._events.error];
• dicer/lib/Dicer.js
  1. if (this._events.preamble)
  2. if ((start + i) < end && this._events.trailer)
  3. if (this._events[ev])
• busboy/lib/types/multipart.js
  1. if (!boy._events.file) {
• ultron/index.js:
  1. for (event in this.ee._events) { if (this.ee._events.hasOwnProperty(event)) {

It looks to me that there should be methods in EventEmitter to:

1. Get a list of all events from an EventEmmiter that currently have listeners.
2. (optional) Check if there are any listeners for a specific event. Like EventEmitter.listenerCount but using a prototype, like EventEmitter.prototype.listeners but returning just the count.
3. (optional) To prepend an event. Does the documentation specify the order in which the events are executed, btw?

Also it looks like readable-stream (ah, and iojs/lib/_stream_readable.js) breaks _eventsCount. I was wrong, it's fine.

@mscdex, thoughts?

[mscdex]

@ChALkeR Using EventEmitter.listenerCount() is easy and efficient enough and was basically all I was using direct usage for.

[ChALkeR]

@mscdex That's why I marked that as (optional).
But ultron module iterates over events, and there seems to be no method for that in the public API.

[mscdex]

@ChALkeR Well, for that you could iterate over .listeners()?

[chrisdickinson]

• readable-stream is directly vendored from core's streams. Core has a need for prepending events – userland does not (with the exception of readable-stream, which doesn't seem like a good excuse to add it.)
• dicer seems to be checking for specific events, which would be better done with EE.listenerCount or even ee.listeners('event').
• cloning an event emitter wholesale (like ultron is doing) is a pretty unexpected / surprising use of the EE API, I don't think we should support that.

[sam-github]

But ultron module iterates over events, and there seems to be no method for that in the public API.

Not to defend ultron - whatever it is - but introspecting an EE to see all the events that have been listened on would be very useful for debugging. For example, to check if a misspelled event name is being used. I'd like to see node be more introspectable, so if getting this info causes zero additional overhead, I think it'd be nice to have.

[Qard]

👍 to allowing more introspection of event emitters.

[jasnell]

@ChALkeR@targos ... No rush, just curious... where are things at on this one?

[thefourtheye]

Use camelCase.

[targos]

@jasnell I rebased on master. here are the new benchmark numbers:

Bench	master	Map
ee-add-remove	1100610	693568 (-37%)
ee-emit-multi-args	5566010	5303709 (-5%)
ee-emit	8074416	7592220 (-6%)
ee-listener-count	409444871	44700710 (-90%)
ee-listeners-many	3426115	3345520 (-2%)
ee-listeners	21558724	15990929 (-26%)

[targos]

I guess I can close this PR for now. I don't think it will improve in a near future.

[thefourtheye]

@targos This also fixes another bug in the implementation. As of now, if we use inherited members as event names, it will confuse the events module a little. So I kinda hoped that this would land :(

[jasnell]

@targos ... Given the current status on this and your last comment, I'm inclined to close. Per @thefourtheye's comment tho, there may still be some stuff worth saving in this so perhaps another look is warranted? Particularly with the newer V8 in master?

[targos]

OK, closing for now.

@thefourtheye I think the problem you are talking about can be fixed by using Object.create(null) instead of an empty literal to create the map.

[thefourtheye]

@targos I tried that already in #2350 but looks like Object.create(null) is not preferred because of the performance hit.

[targos]

the performance tests were run in 2012. I would try with current V8 version to be sure it is still a problem.

[thefourtheye]

@targos I wrote this simple script to compare the performance of them

'use strict';consttimes=10e7;console.time('Object.create');for(vari=0;i<times;i+=1){letobj=Object.create(null);}console.timeEnd('Object.create');console.time('Object Literal');for(vari=0;i<times;i+=1){letobj={};}console.timeEnd('Object Literal');

Object.create is 10 times faster, but the speed gain is in nano-seconds.