assert: ensure .rejects() disallows sync throws

[not-an-aardvark]

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• commit message follows commit guidelines

This PR has two commits; the first commit fixes an issue with the assert tests, and the second commit fixes#19646.

First commit:

test: ensure failed assertions cause build to fail
This updates the test in `test/parallel/test-assert-async.js` to add an
assertion that the Promises used in the test end up fulfilled.
Previously, if an assertion failure occurred, the Promises would have
rejected and a warning would have been logged, but the test would still
have exit code 0.

Second commit:

assert: ensure .rejects() disallows sync throws
This updates `assert.rejects()` to disallow any errors that are thrown
synchronously from the given function. Previously, throwing an error
would cause the same behavior as returning a rejected Promise.
Fixes: https://github.com/nodejs/node/issues/19646

Note that the second commit would be semver-major, but it modifies an API (introduced in #18023) that has not been backported to a release branch yet. As a result, the second commit could be backported as long as it's applied at the same time as #18023.

[BridgeAR]

LGTM

[BridgeAR]

Suggestion: use catch(common.mustCall(err) => { assert.strictEqual(err, THROWN_ERROR); })

[BridgeAR]

Nit: this test should now be redundant if I am not mistaken.

[not-an-aardvark]

I'm assuming you mean that this test is redundant with the other test here. However, this test is a bit different from the other one because it throws asynchronously, and so the promise returned by assert.rejects fulfills (whereas in the other test, the function throws synchronously and the promise returned by assert.rejects rejects and needs to be caught separately).

[not-an-aardvark]

Actually, I just realized what you meant: this test is now redundant with this other test. I agree, I'll remove the other one.

edit: Removed.

[BridgeAR]

Nit: use common.crashOnUnhandledRejection() at the top of the file instead of adding then(common.mustCall()) everywhere.

[targos]

@BridgeAR about .then(common.mustCall()):

It is necessary to make sure the promise eventually fulfills (does not stay forever in a pending state).

I think the best would be to combine common.crashOnUnhandledRejection() at the top and an async IIFE:

common.crashOnUnhandledRejection();(async()=>{{// test 1awaitassert.rejects(something);}{// test 2awaitassert.rejects(somethingElse);}})().then(common.mustCall());

We should maybe add something like common.runAsync(async () => {}) that does both?

[BridgeAR]

@targos fair point. But should the test not time out in case the promise does not fulfill?

[TimothyGu]

@BridgeAR Here's a case where the .then(common.mustCall()) is absolutely necessary:

// Never fulfills.constprom=newPromise(()=>{});(async()=>{awaitprom;console.log('Never printed');})();

The process does not stay alive since promise states are not part of the event loop.

[not-an-aardvark]

Updated to follow the suggestions in #19650 (comment) and #19650 (comment).

[not-an-aardvark]

CI: https://ci.nodejs.org/job/node-test-pull-request/13906/

[BridgeAR]

@TimothyGu thanks for the heads up!

@not-an-aardvark to have feature parity, it might make sense to check in assert.doesNotReject that the passed in function will actually return a promise. From that point on the function has some legitimacy. Right now I do not see any legit use case. What do you think?

[targos]

Just thinking about it now: wouldn't it be better if assert.rejects expected a Promise instead of a function?

[BridgeAR]

@targos I agree. That solves that dilemma. Since we did not yet publish the functionality, we can still change it. [update] Thinking about it again: It might still be better to only handle functions:

• We use functions in assert.throws and assert.doesNotThrow
• It has the benefit to test a specific functionality to really return a promise

I am going to give it another thought when I have some more time later on.

[targos]

In favor of changing it: existing assertion libraries that I know (Should.js, Chai as Promised, jest) work directly with promises.

[not-an-aardvark]

Arguably, this is different from assert.throws because assert.throws makes an assertion about how a function behaves, whereas assert.rejects only makes an assertion about the function's return value. So it would make sense for it to just accept a Promise rather than a function.

On the other hand, one potential argument for the current assert.rejects behavior is that it could be simpler for users who don't know how Promises work. They might have a mental model of "always use await when calling an async function to avoid weird behavior". With that mental model, assert.rejects(someAsyncFunction) would make sense, but assert.rejects(someAsyncFunction()) would not make sense. They might try to use assert.rejects(await someAsyncFunction()) instead, but this would cause other problems. The current behavior of assert.rejects would allow for a mental model of "this is just like assert.throws, but for async functions."

[not-an-aardvark]

@BridgeAR@targos What would you recommend doing with this PR? If the behavior of assert.rejects is going to be changed to accept Promises rather than functions, then I'm fine with closing this. However, if we don't do anything, then it seems likely that the current behavior will end up landing in a release, after which it will be much harder to change. (I haven't landed this PR yet because I'm not sure if you wanted to change the API signature instead.)

[BridgeAR]

I am actually considering supporting to accept both: passing in a function that returns a promise or directly passing in a promise. That would ease the user experience. I do not really see a downside accepting both.

[not-an-aardvark]

I'm going to land this since we seem to be in agreement that it's an improvement over the current behavior, but I'd be fine with the behavior being changed in a future PR.

[not-an-aardvark]

Landed in 8995125 and fdb35d8