test: added `common.mustCallAtLeast` by refack · Pull Request #12935 · nodejs/node

refack · 2017-05-09T21:24:40Z

added common.mustCallAtLeast for calling more than minimum times.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

test

addaleax · 2017-05-09T21:27:37Z

typo: then → than

refack · 2017-05-09T21:28:08Z

CI: https://ci.nodejs.org/job/node-test-commit/9765/
/cc @nodejs/testing

cjihrig · 2017-05-09T21:28:06Z

Can you undo this block of changes.

the JSDocs?
It's usefull for people with IDEs that can parse these (VSCode / Webstorm)

Other than lib/punycode.js which is vendored, there are only 9 @param matches in all of lib, src, and test. It's more consistent to drop them.

cjihrig · 2017-05-09T21:28:28Z

This code should live in test/parallel/test-common.js I think.

didn't see that one...

cjihrig · 2017-05-09T21:29:05Z

I think this needs a documentation update.

refack · 2017-05-09T21:43:32Z

Docs added, nit's addressed. @cjihrig I'd rather keep the JSDocs, unless you have a strong objection.

cjihrig · 2017-05-09T21:45:07Z

I wouldn't say strong objection, but definitely -1.

refack · 2017-05-09T21:47:22Z

I wouldn't say strong objection, but definitely -1.

Gone.
I'll do a PR with JSDocs for the whole file once 😉

Fishrock123 · 2017-05-09T21:51:59Z

Could you point to somewhere that this would be better suited for rather than the explicit number of calls?

Fishrock123 · 2017-05-09T21:52:37Z

I generally think this is a bad idea because if something is being called more times than you expect you've probably got a bug.

refack · 2017-05-09T21:52:45Z

Could you point to somewhere that this would be better suited for rather than the explicit number of calls?

#12930 (comment)

refack · 2017-05-09T21:55:49Z

I generally think this is a bad idea because if something is being called more times than you expect you've probably got a bug.

I tend to agree, but some things are either OS dependent or non deterministic. In those cases we tend to skip the mustCall all together which IMHO is worse.
I agree this should be used sparingly, and usage should receive justification in a comment.

jasnell

I've been wanting this for a while. Once @cjihrig is happy with it, I'm LGTM

mscdex · 2017-05-09T22:32:06Z

-1 I don't think it is a good idea to reuse the same parameter (or even the same function) for this.

jasnell · 2017-05-09T22:33:14Z

@Fishrock123 ... this came up with regards to adding mustCall() around a data event handler, which could be reasonably called any number of times.

@mscdex ... the other option I had in mind was a common.mustCallAtLeast(n) method that would allow a minimum number to be specified.

mscdex · 2017-05-09T22:39:09Z

@jasnell I would much prefer a separate method so that the intention is clear.

refack · 2017-05-09T22:57:44Z

@jasnell I would much prefer a separate method so that the intention is clear.

@mscdex I was thinking about that.
Will do.

Trott

-0 from me. I dislike magic values. I prefer that things in common have simple, self-explanatory, and intuitively obvious interfaces. This adds more cognitive overhead, especially for newcomers.

Trott · 2017-05-09T23:08:46Z

Nit: This documentation change results in a confusing run-on sentence:
If the returned function has not been called exactly expected number of times, or at least once if
expected === '+', when the test is complete, then the test will fail.
Would be better as something like:
If the returned function has not been called exactly expected number of times when the test is complete, then the test will fail. If expected is '+'`, then the test must run at least once.

gibfahn · 2017-05-09T23:13:31Z

the other option I had in mind was a common.mustCallAtLeast(n)

+1 to that

-0 from me. I dislike magic values

@Trott thoughts on @jasnell's proposed common.mustCallAtLeast(2)?

Trott · 2017-05-09T23:19:07Z

@Trott thoughts on @jasnell's proposed common.mustCallAtLeast(2)?

Eh, still pretty +/-0 on it. I think there's enough API surface area in common and we quickly reach a point of diminishing returns.

Moreover, I'm not really sure this is solving a very common problem. I imagine it's mostly for data callbacks in tests that are called once but could theoretically be called more than once. However, those almost never need to actually be wrapped to make sure they're called because they usually have something like data += chunk and then the value of data is checked somewhere, so you don't really need to check that the callback was called. The test will fail if it wasn't. (Will there be a small number of empty callbacks for data that we want to check? Sure. Is it worth adding a whole other function to the common monolith for it? I doubt it.)

But if others feel this has big value, I won't stop it.

refack · 2017-05-09T23:28:32Z

Replace magic '+' with mustCallAtLeast PTAL

refack · 2017-05-09T23:38:33Z

P.S. why don't we adopt something like sinon.
I'm a big believer that better tooling begets better tests.

Trott · 2017-05-10T04:16:03Z

P.S. why don't we adopt something like sinon.

Biggest reason is probably because no one's bothered to do it yet.

That said, a PR adding sinon very well might not get accepted. Speaking only for myself, I have grown wary of things that raise the barrier to entry for people working on tests. So to the extent that we avoid extra tools and unneeded abstractions in the common module, I'm happy. But that said, if the benefit would be considerable and obvious, I'd be for it.

Others in @nodejs/testing might have other opinions.

refack · 2017-05-19T17:40:04Z

Pre land CI: https://ci.nodejs.org/job/node-test-commit/10013/

refack · 2017-05-19T19:24:35Z

landed in fccc0bf

MylesBorins · 2017-07-17T14:51:25Z

Should this land on v6.x?

refack · 2017-07-17T15:49:34Z

Should this land on v6.x?

Sure. It's a semver-minor change in the test harness.

refack · 2017-07-17T18:09:40Z

#14327 depends on this

PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

nodejs-github-bot added the test Issues and PRs related to the tests. label May 9, 2017

refack self-assigned this May 9, 2017

refack requested review from addaleax and jasnell May 9, 2017 21:25

addaleax approved these changes May 9, 2017
View reviewed changes

Comment threadtest/common/index.js Outdated
addaleaxMay 9, 2017
Copy link
Copy Markdown
Member
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: then → than

cjihrig requested changes May 9, 2017
View reviewed changes

refack mentioned this pull request May 9, 2017
test: improve numerous http tests #12930
Closed
3 tasks

jasnell approved these changes May 9, 2017
View reviewed changes

Trott reviewed May 9, 2017
View reviewed changes

refack force-pushed the mustCall+ branch from 95517cc to a6bb177Compare May 9, 2017 23:30

refack changed the title ~~test: allow mustCall '+' as expected~~test: added common.mustCallAtLeastMay 9, 2017

refack mentioned this pull request May 10, 2017
create public nodejs/collaborators repo nodejs/TSC#243
Closed

refack force-pushed the mustCall+ branch from 141dc1f to a5b2396Compare May 19, 2017 17:39

refack closed this May 19, 2017

refack force-pushed the mustCall+ branch from a5b2396 to fccc0bfCompare May 19, 2017 19:24

refack deleted the mustCall+ branch May 19, 2017 19:24

refack merged commit fccc0bf into nodejs:masterMay 19, 2017

jasnell mentioned this pull request May 28, 2017
8.0.0 Release Proposal #12220
Closed

refack mentioned this pull request Jun 3, 2017
test: add hijackStdout and hijackStderr #13439
Closed
4 tasks

refack removed their assignment Jun 12, 2017

gibfahn mentioned this pull request Jun 15, 2017
Auditing for 6.11.1 nodejs/Release#230
Closed
3 tasks

MylesBorins added the lts-watch-v6.x label Jul 17, 2017

MylesBorins added the semver-minor PRs that contain new features and should be released in the next minor version. label Jul 17, 2017

refack mentioned this pull request Jul 17, 2017
[v6.x backport] test: harden test-dgram-bind-shared-ports #14327
Closed
3 tasks

MylesBorins added land-on-v6.x and removed lts-watch-v6.x semver-minor PRs that contain new features and should be released in the next minor version. labels Aug 14, 2017

MylesBorins pushed a commit that referenced this pull request Aug 14, 2017
test: add mustCallAtLeast
ce86b8e
PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

MylesBorins pushed a commit that referenced this pull request Aug 16, 2017
test: add mustCallAtLeast
f578c9b
PR-URL: #12935 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

MylesBorins mentioned this pull request Aug 16, 2017
v6.11.3-proposal #14852
Merged

Uh oh!

Uh oh!

Conversation

refack commented May 9, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Affected core subsystem(s)

Uh oh!

addaleaxMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

refack commented May 9, 2017

Uh oh!

cjihrigMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

refackMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

cjihrigMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

cjihrigMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

refackMay 9, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cjihrig commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

cjihrig commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

Fishrock123 commented May 9, 2017

Uh oh!

Fishrock123 commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

jasnell left a comment

Choose a reason for hiding this comment

Uh oh!

mscdex commented May 9, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jasnell commented May 9, 2017

Uh oh!

mscdex commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

Trott left a comment

Choose a reason for hiding this comment

Uh oh!

TrottMay 9, 2017

Choose a reason for hiding this comment

Uh oh!

gibfahn commented May 9, 2017

Uh oh!

Trott commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

refack commented May 9, 2017

Uh oh!

Trott commented May 10, 2017

Uh oh!

refack commented May 19, 2017

Uh oh!

refack commented May 19, 2017

Uh oh!

MylesBorins commented Jul 17, 2017

Uh oh!

refack commented Jul 17, 2017

refack commented May 9, 2017•
edited
Loading

refackMay 9, 2017•
edited
Loading

mscdex commented May 9, 2017•
edited
Loading