assert: Add support for Map and Set in deepEqual by josephg · Pull Request #12142 · nodejs/node

josephg · 2017-03-31T04:00:51Z

assert.deepEqual and assert.deepStrictEqual currently return true for
any pair of Maps and Sets regardless of content. This patch adds
support in deepEqual and deepStrictEqual to verify the contents of Maps
and Sets.

Unfortunately because there's no way to pairwise fetch set values or map
values which are equivalent but not reference-equal, this change
currently only supports reference equality checking in set values and
map keys. Equivalence checking could be done, but it would be an
O(n^2) operation, and worse, it would get slower exponentially if maps
and sets were nested.

Update: Based on conversation below, the implementation now incurs an O(n^2) cost if the sets or maps are not equal, because to be correct we need to check all pairs of values in sets, and all pairs of keys in a map. This may become exponentially more expensive with input which does deep equality checking on nested maps. But given these checks are almost always only done during testing, and on small objects its probably the right approach.

Update 2: Based on a clever suggestion by @TimothyGu the O(n^2) cost is now only paid when you're either in non-strict mode or when you're using set values & map keys which have typeof object.

Note that this change breaks compatibility with previous versions of
deepEqual and deepStrictEqual if consumers were depending on all maps
and sets to be seen as equivalent. The old behaviour was never
documented, but nevertheless there are certainly some tests out there somewhere
which depend on it.

Support had been stalled because the assert API was frozen, but assert was recently
unfrozen in CTC#63

Fixes: #2309
Refs: tape-testing/tape#342
Refs: #2315
Refs: nodejs/CTC#63

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

Assert

assert.deepEqual and assert.deepStrictEqual currently return true for any pair of Maps and Sets regardless of content. This patch adds support in deepEqual and deepStrictEqual to verify the contents of Maps and Sets. Unfortunately because there's no way to pairwise fetch set values or map values which are equivalent but not reference-equal, this change currently only supports reference equality checking in set values and map key values. Equivalence checking could be done, but it would be an O(n^2) operation, and worse, it would get slower exponentially if maps and sets were nested. Note that this change breaks compatibility with previous versions of deepEqual and deepStrictEqual if consumers were depending on all maps and sets to be seen as equivalent. The old behaviour was never documented, but nevertheless there are certainly some tests out there which depend on it. Support has stalled because the assert API was frozen, but was recently unfrozen in CTC#63 Fixes: nodejs#2309 Refs: tape-testing/tape#342 Refs: nodejs#2315 Refs: nodejs/CTC#63

Trott · 2017-03-31T04:09:37Z

CI: https://ci.nodejs.org/job/node-test-pull-request/7134/

addaleax · 2017-03-31T04:06:11Z

+
+function isSet(object) {
+ return object.constructor === Set;
+}


Can you use isMap and isSet from process.binding('util') instead? These checks won’t work for subclasses of Map/Set or Maps/Sets from other VM contexts.

addaleax · 2017-03-31T04:09:29Z

+ // it would require scanning each pairwise set of not strict-equal items,
+ // which is O(n^2), and would get exponentially worse if sets are nested. So
+ // for now we simply won't support deep equality checking set items or map
+ // keys.


I think I would actually prefer the awful performance over always using strict equality… @nodejs/collaborators thoughts?

In this case, I think I'd value correct over performant, so yeah, I agree with @addaleax.

For tests, I agree. I'm nervous people might be using assert.deepEqual inside runtime code, although I just checked the node_modules directory of a project I'm working on with ~400 transitive dependancies and only jsprim calls assert.deepEqual outside of a test, and thats a in a very minor way. I'll wait for some more feedback, but unless there's any strong objections I'll change the behaviour to be correct and slow.

... Another strong argument in favor of making it correct > fast is that it'll be more compatible with the current implementation. If someone currently has a test that reads assert.deepStrictEqual(new Set([{x:5}]), new Set([{x:5}])) that test will currently pass because the sets won't be compared. With the PR in its current state, that test will start to fail.

@josephg Maybe just update this PR and see if somebody objects? If we need to go back, the current HEAD is at f051840.

#14258 just landed and reduces the complexity to O(n log n). Therefore the performance should not be of much concern anymore.

Updated PR based on comments Ref: nodejs#12142

Trott

LGTM if CI comes back green and with or without addression @addaleax's comments (but preferably with!)

This change updates the checks for deep equality checking on Map and Set to check all set values / all map keys to see if any of them match the expected result. This change is much slower, but based on the conversation in the pull request its probably the right approach. Ref: nodejs#12142

mcollina

LGTM

josephg · 2017-03-31T08:21:39Z

I've updated the PR to do the quadratic pairwise checking based on the conversation above.

addaleax

Awesome PR. 👍

addaleax · 2017-03-31T08:19:39Z

+ // deepEqual. The implementation currently returns false, which is a simpler
+ // and faster implementation.
+ if (a.size !== b.size)
+ return false;


👍 I think deepEqual should be correct to reject sets of different sizes.

addaleax · 2017-03-31T08:20:32Z

+
+ // This check is not strictly necessary, but its here to improve
+ // performance of the common case when reference-equal keys exist (which
+ // includes all primitive valued keys).


nit: primitive-valued

addaleax · 2017-03-31T08:24:58Z

@nodejs/ctc semver-major or semver-minor or semver-minor but all dont-land labels?

targos · 2017-03-31T08:45:21Z

 return false;
 }

+ // Sets and maps don't have their entries accessable via normal object


s/accessable/accessible/

targos · 2017-03-31T08:49:00Z

I think it's semver-major. This can and probably will break something.

vsemozhetbyt · 2017-03-31T09:43:24Z

@josephg A nit: traditionally, commit title should use lower case imperative verb after subsystem prefix. Buth this can be fixed by anybody who will land the commits.

evanlucas · 2017-03-31T14:42:03Z

+ return false;
+
+ var val1, val2;
+ outer: for (val1 of a) {


Can you use const here?

evanlucas · 2017-03-31T14:42:13Z

+ // hunting for something thats deep-equal to it. Note that this is O(n^2)
+ // complexity, and will get slower if large, very similar sets / maps are
+ // nested inside. Unfortunately there's no real way around this.
+ for (val2 of b) {


Can you use const here?

evanlucas · 2017-03-31T14:43:11Z

+ return false;
+
+ var key1, key2, item1, item2;
+ outer: for ([key1, item1] of a) {


can you use const here as well?

evanlucas · 2017-03-31T14:44:05Z

+
+ // Hunt for keys which are deep-equal to key1 in b. Just like setEquiv
+ // above, this hunt makes this function O(n^2).
+ for ([key2, item2] of b) {


can you use const here as well?

evanlucas · 2017-03-31T14:45:58Z

+ const m2 = new Map();
+ m1.x = 5;
+ assertNotDeepOrStrict(m1, m2);
+}


Maybe add a case where there is a circular reference?

Cleaned up as per comments in issue Ref: nodejs#6416

TimothyGu · 2017-03-31T22:56:45Z

+ // The value doesn't exist in the second set by reference, so we'll go
+ // hunting for something thats deep-equal to it. Note that this is O(n^2)
+ // complexity, and will get slower if large, very similar sets / maps are
+ // nested inside. Unfortunately there's no real way around this.


Is it possible to optimize this by only starting the additional search if !strict || typeof val1 === 'object'?

Yes. Thats really clever, and very obvious now that you've pointed it out. Fixed.

TimothyGu · 2017-03-31T22:57:18Z

+ }
+
+ // Hunt for keys which are deep-equal to key1 in b. Just like setEquiv
+ // above, this hunt makes this function O(n^2).


Based on comments in the PR, this change restricts an O(n^2) to only happen when your set contains object-like objects, your map contains object-like keys or you're not in strict mode. ref: nodejs#12142 (review)

addaleax · 2017-03-31T23:46:25Z

- for (const val2 of b) {
- if (_deepEqual(val1, val2, strict, actualVisitedObjects)) {
- continue outer;
+ if (!strict || typeof val1 === 'object') {


(!strict || (typeof val1 === 'object' && val1 !== null) || typeof val1 === 'function')?

We don't need to search for functions because the only way deepEqual will consider functions equivalent is through reference equality. The null check makes some sense, although I think it'll make a very small difference in practice because putting null in a set and using null as a key are rare. (In my code, null popping up in either of those places is usually an indication of a bug)

Its a minor point, but do you think its still worth putting in the null check? I'm indifferent either way, but happy to add it if you think the marginal speed improvement is worth the marginal extra complexity of that check.

@josephg Yes, I’d say it’s worth it… there might be cases where the difference is not so marginal, and I think it’s okay for Node core to sacrifice a tiny bit of readability for a small performance gain.

ref: https://github.com/nodejs/node/pull/12142/files/8fb6ebf9745badfb2f3f6a72cef211ff8bfbf6ba

TimothyGu · 2017-04-01T00:23:24Z

Not sure how important this is to the discussion. There are also some subtle differences in SameValueZero (the set of comparison semantics .has() uses), abstract equality (==, the operator assert.equal uses), and strict equality (===, the operator assert.strictEqual uses). Concretely, this means code such as the following may yield surprising results:

assert.equal(NaN,NaN);// AssertionError: NaN === NaNassert.strictEqual(NaN,NaN);// AssertionError: NaN === NaNconstmap1=newMap([[NaN,0]]);constmap2=newMap([[NaN,0]]);assert.deepEqual(map1,map2);// No errorassert.strictDeepEqual(map1,map2);// No error

josephg · 2017-04-01T14:27:04Z

@joyeecheung I've cleaned up the documentation. Left the little inline YAML changelog alone for now.

joyeecheung · 2017-04-01T15:34:14Z

 memos.actual.push(actual);
 memos.expected.push(expected);

+


Just noticed..unrelated white space change?

joyeecheung · 2017-04-01T15:47:19Z

@josephg I believe the format for a not-yet-released changelog is...

changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/12142
description: ....

@joyeecheung

- Added changes: entries in assert API documentation - Refactored setEquiv based on @joyeecheung's comments ref: nodejs#12142

joyeecheung · 2017-04-01T15:57:50Z

 // Different type tags
 ```

+An exception is made to the strict equality comparison rule for [`Map`][] keys


This should probably go to the item 1 above...also the NaN case can be demonstrated by linking to the existing caveats section, something like
1. For Sets and Maps, primitive values are compared using the [SameValueZero][] comparison (which means they are free of the [caveats][]). For other objects, primitive values are compared using the [Strict Equality Comparison][] ( `===` )

joyeecheung · 2017-04-01T16:00:36Z

 changes:
+ - version: REPLACEME
+ pr-url: https://github.com/nodejs/node/pull/12142
+ description: Sets and Maps are handled correctly.


"handled correctly" is a bit vague..maybe "elements in Sets and Maps are compared during deep comparison" or something like that?

joyeecheung

minus some nits, I think this can go ahead now...thanks for bearing with me and for the contribution, your hard work is defnitely appreciated :D

ref: nodejs#12142

josephg · 2017-04-01T16:11:11Z

Cheers :)

@joyeecheung

Refactored setEquiv and mapEquiv based on @joyeecheung's stylistic preference to avoid labels.

josephg · 2017-04-01T17:01:36Z

@joyeecheung I've rewritten mapEquiv and setEquiv based on our conversation above. The tight if() statement logic we were talking about turned out to be completely unreadable. As proof that it was a bad idea neither of us was able to produce working code of that form. Even my final attempt had a small mistake and I completely confused myself trying to fix it, so I abandoned that style completely. The current code passed all tests on its first run.

I think its all very readable now - although using ancillary functions makes it more wordy than it was using loop labels. I'm not sure the code is any better for it. But, we're enough opinions deep that I'm getting exhausted from all this churn on such a minor patch. If anyone wants to make more purely stylistic changes I'm happy to add you as a contributor to josephg/node. You're welcome to modify the PR directly.

Otherwise I don't know the process, but I'm happy for it to be merged.

joyeecheung · 2017-04-01T17:53:56Z

@josephg Sorry if my reviews are disheartening and thanks for sticking with this PR!

CI: https://ci.nodejs.org/job/node-test-pull-request/7153/
CITGM since it's semver-major: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/683/

Oudated reviews..

addaleax · 2017-04-03T08:19:53Z

Landed in 6481c93, and again thanks for the PR!

assert.deepEqual and assert.deepStrictEqual currently return true for any pair of Maps and Sets regardless of content. This patch adds support in deepEqual and deepStrictEqual to verify the contents of Maps and Sets. Deeo equivalence checking is currently an O(n^2) operation, and worse, it gets slower exponentially if maps and sets were nested. Note that this change breaks compatibility with previous versions of deepEqual and deepStrictEqual if consumers were depending on all maps and sets to be seen as equivalent. The old behaviour was never documented, but nevertheless there are certainly some tests out there which depend on it. Support has stalled because the assert API was frozen, but was recently unfrozen in CTC#63. --- Later squashed in: This change updates the checks for deep equality checking on Map and Set to check all set values / all map keys to see if any of them match the expected result. This change is much slower, but based on the conversation in the pull request its probably the right approach. Fixes: #2309 Refs: tape-testing/tape#342 Refs: #2315 Refs: nodejs/CTC#63 PR-URL: #12142 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>

josephg · 2017-04-04T11:20:58Z

Yay! Thanks for helping me get my first PR in node everyone!

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. Fixes: nodejs#13347 Refs: nodejs#12142

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. PR-URL: nodejs#13426Fixes: nodejs#13347 Refs: nodejs#12142 Reviewed-By: Refael Ackermann <refack@gmail.com>

This fixes a bug where deepEqual and deepStrictEqual would have incorrect behaviour in sets and maps containing multiple equivalent keys. PR-URL: #13426Fixes: #13347 Refs: #12142 Reviewed-By: Refael Ackermann <refack@gmail.com>

nodejs-github-bot added the assert Issues and PRs related to the assert subsystem. label Mar 31, 2017

Trott added the semver-minor PRs that contain new features and should be released in the next minor version. label Mar 31, 2017

addaleax approved these changes Mar 31, 2017
View reviewed changes

addaleax requested review from Trott, evanlucas and joyeecheung March 31, 2017 04:10

assert: Use isSet and isMap from process.binding
f051840
Updated PR based on comments Ref: nodejs#12142

Trott approved these changes Mar 31, 2017
View reviewed changes

mcollina approved these changes Mar 31, 2017
View reviewed changes

addaleax approved these changes Mar 31, 2017
View reviewed changes

assert: Fixed nit in code review
800ae46

targos reviewed Mar 31, 2017
View reviewed changes

assert: Fixed typo in comment
031f6f3

evanlucas previously requested changes Mar 31, 2017
View reviewed changes

assert: var -> const and added tests
d6baaee
Cleaned up as per comments in issue Ref: nodejs#6416

TimothyGu reviewed Mar 31, 2017
View reviewed changes

assert: restrict O(n^2) search
8fb6ebf
Based on comments in the PR, this change restricts an O(n^2) to only happen when your set contains object-like objects, your map contains object-like keys or you're not in strict mode. ref: nodejs#12142 (review)

addaleax reviewed Mar 31, 2017
View reviewed changes

addaleax added semver-major PRs that contain breaking changes and should be released in the next major version. and removed semver-minor PRs that contain new features and should be released in the next minor version. labels Mar 31, 2017

addaleax added this to the 8.0.0 milestone Mar 31, 2017

assert: added null check in set/map short circuit
acef701
ref: https://github.com/nodejs/node/pull/12142/files/8fb6ebf9745badfb2f3f6a72cef211ff8bfbf6ba

joyeecheung reviewed Apr 1, 2017
View reviewed changes

assert: Updated based on PR
7bc29b0
- Added changes: entries in assert API documentation - Refactored setEquiv based on @joyeecheung's comments ref: nodejs#12142

joyeecheung reviewed Apr 1, 2017
View reviewed changes

joyeecheung approved these changes Apr 1, 2017
View reviewed changes

josephg added 2 commits April 2, 2017 02:07

assert: updated docs based on PR
6bdfcaf
ref: nodejs#12142

assert: fixed docs # link
fc5196a

assert: refactored based on PR
7f9d4d8
Refactored setEquiv and mapEquiv based on @joyeecheung's stylistic preference to avoid labels.

addaleax closed this Apr 3, 2017

jasnell mentioned this pull request Apr 4, 2017
8.0.0 Release Proposal #12220
Closed

Trott mentioned this pull request Apr 11, 2017
test: complete coverage for lib/assert.js #12239
Closed
3 tasks

TimothyGu mentioned this pull request May 5, 2017
assert: improve deepEqual perf for large input #12849
Closed
4 tasks

vsemozhetbyt mentioned this pull request May 31, 2017
assert: deepEqual of two Sets with different content passes #13347
Closed

josephg deleted the assert-sets-and-maps branch June 2, 2017 23:53

josephg mentioned this pull request Jun 3, 2017
assert: fix deepEqual similar sets and maps bug #13426
Closed
3 tasks

Uh oh!

Uh oh!

Conversation

josephg commented Mar 31, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Affected core subsystem(s)

Uh oh!

Trott commented Mar 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephgMar 31, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephgMar 31, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Trott left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

josephg commented Mar 31, 2017

Uh oh!

addaleax left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

addaleax commented Mar 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

targos commented Mar 31, 2017

Uh oh!

vsemozhetbyt commented Mar 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TimothyGuMar 31, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephgMar 31, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

josephg commented Mar 31, 2017•
edited
Loading

josephgMar 31, 2017•
edited
Loading

josephgMar 31, 2017•
edited
Loading

Trott left a comment •
edited
Loading

TimothyGuMar 31, 2017•
edited
Loading

josephgMar 31, 2017•
edited
Loading

josephgApr 1, 2017•
edited
Loading

TimothyGu commented Apr 1, 2017•
edited
Loading

josephg commented Apr 1, 2017•
edited
Loading

joyeecheungApr 1, 2017•
edited
Loading

josephg commented Apr 1, 2017•
edited
Loading