test: run the addon tests last by seppevs · Pull Request #12062 · nodejs/node

seppevs · 2017-03-27T09:32:13Z

Running the addon tests before the parallel, sequential, etc. tests
can be a problem if there is a bug in node that prevents the
addon tests from running properly. When the addon tests fail for any
reason, then none of the other tests (e.g. parallel, etc.)
are executed.

Running the addon tests last fixes this.

See #12031

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows [commit guidelines][]

Affected core subsystem(s)

test

aqrln · 2017-03-27T10:27:39Z

I think that "See ..." line should be formatted as a proper "Refs:" or "Fixes:" metadata field. Also, there's probably a typo in the commit message (s/lasts/last ?).

seppevs · 2017-03-27T10:43:01Z

Thanks for the tips, I've changed the commit message.

vsemozhetbyt · 2017-03-27T12:54:33Z

CI: https://ci.nodejs.org/job/node-test-pull-request/7054/

vsemozhetbyt · 2017-03-27T12:56:57Z

Does this affect Windows builds? Should the vcbuild.bat also be fixed?

vsemozhetbyt · 2017-03-27T13:34:27Z

Some tests failed, trying another CI: https://ci.nodejs.org/job/node-test-pull-request/7055/

aqrln · 2017-03-27T13:37:21Z

@seppevs regarding @vsemozhetbyt's comment, please see https://github.com/nodejs/node/blob/master/vcbuild.bat#L62-L63.

aqrln · 2017-03-27T13:44:49Z

@vsemozhetbyt those tests are unrelated to this PR and I don't think another CI run will help, tbh :)
Failed test on Windows is already marked as flaky in #12054, and I've seen parallel/test-async-wrap-check-providers as well as other tests failing with timeout on CentOS 5 for a while too (just open any of the recent PRs).

seppevs · 2017-03-27T13:47:04Z

@aqrln Thanks for the tip. I have modified the vcbuild.bat now. Can you review it, I'm not completely sure I did it correctly (and I have no Windows machine to verify).

aqrln · 2017-03-27T13:50:14Z

@seppevs heh, neither do I, so I hope my suggestion was correct :)

/cc @nodejs/build @nodejs/platform-windows

EDIT: whoops, only members can mention teams. @vsemozhetbyt can you do that, please?

vsemozhetbyt · 2017-03-27T13:56:26Z

/cc @nodejs/build @nodejs/platform-windows

Fishrock123

Yes please.

cc @bnoordhuis probably?

aqrln · 2017-03-27T14:30:32Z

One more thing. CI runs make test-ci, not make test, so you need to change line 311 too.

Thanks, I've changed the order on line 311 as well.

jasnell

Note that failures in cctest also suffer this problem.

refack · 2017-03-29T13:38:20Z

I have modified the vcbuild.bat now. Can you review it, I'm not completely sure I did it correctly (and I have no Windows machine to verify).

So on windows, it's only a half solution since build-addons is run before the tests, and so it the build fails no tests are run 😞

aqrln · 2017-03-29T13:42:02Z

@refack hmm, not only only on Windows. The same goes for Unix.

refack · 2017-03-29T13:42:16Z

maybe split this to "all other test" -> build -> "test addons"... 🤔

refack · 2017-03-29T13:44:08Z

Is you decide to split the test, you'll need it here too...

refack

Too much hassle to refactor vcbuild.bat
Current LGTM (works on Windows)

jasnell · 2017-04-04T17:34:59Z

This needs a rebase before it can be landed.

Running the addon tests before the parallel, sequential, etc. tests can be a problem if there is a bug in node that prevents the addon tests from running properly. When the addon tests fail for any reason, then none of the other tests (e.g. parallel, etc.) are executed. Running the addon tests last fixes this. Refs: nodejs#12031

seppevs · 2017-04-04T20:38:22Z

@jasnell rebase done

mscdex · 2017-04-05T22:22:55Z

CI again: https://ci.nodejs.org/job/node-test-pull-request/7241/

aqrln · 2017-04-12T18:35:17Z

@seppevs can you please rebase it again?

jasnell · 2017-04-13T22:08:11Z

~~This needs to be rebased before it can land! :-)~~

ha! posted then saw @aqrln's comment. guess I should read all the way through before posting ;-)

refack · 2017-04-13T22:16:24Z

Poor guy already rebased one. I'm taking care of this.

Running the addon tests before the parallel, sequential, etc. tests can be a problem if there is a bug in node that prevents the addon tests from running properly. When the addon tests fail for any reason, then none of the other tests (e.g. parallel, etc.) are executed. Running the addon tests last fixes this. Refs: nodejs#12031 PR-URL: nodejs#12062 Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>

refack · 2017-04-13T22:35:57Z

Landed in 3d2181c

Running the addon tests before the parallel, sequential, etc. tests can be a problem if there is a bug in node that prevents the addon tests from running properly. When the addon tests fail for any reason, then none of the other tests (e.g. parallel, etc.) are executed. Running the addon tests last fixes this. Refs: #12031 PR-URL: #12062 Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>

nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Mar 27, 2017

seppevs force-pushed the test_run_addon_tests_last branch from 9eeb691 to cb369d2Compare March 27, 2017 10:42

seppevs changed the title ~~test: run the addon tests lasts~~test: run the addon tests lastMar 27, 2017

seppevs force-pushed the test_run_addon_tests_last branch from cb369d2 to dc3a770Compare March 27, 2017 13:45

Fishrock123 approved these changes Mar 27, 2017
View reviewed changes

aqrln reviewed Mar 27, 2017
View reviewed changes

seppevs force-pushed the test_run_addon_tests_last branch from dc3a770 to 3196f56Compare March 27, 2017 14:37

bnoordhuis approved these changes Mar 27, 2017
View reviewed changes

jasnell approved these changes Mar 27, 2017
View reviewed changes

mscdex added addons Issues and PRs related to native addons. test Issues and PRs related to the tests. labels Mar 27, 2017

richardlau approved these changes Mar 27, 2017
View reviewed changes

cjihrig approved these changes Mar 27, 2017
View reviewed changes

refack suggested changes Mar 29, 2017
View reviewed changes

refack approved these changes Mar 29, 2017
View reviewed changes

seppevs force-pushed the test_run_addon_tests_last branch from 3196f56 to 141a6d9Compare April 4, 2017 10:23

seppevs force-pushed the test_run_addon_tests_last branch from 141a6d9 to 9149610Compare April 4, 2017 20:38

refack closed this Apr 13, 2017

jasnell mentioned this pull request May 11, 2017
8.0.0 Release Proposal #12220
Closed

gibfahn mentioned this pull request Jun 15, 2017
Auditing for 6.11.1 nodejs/Release#230
Closed
3 tasks

gibfahn added the land-on-v6.x label Jun 18, 2017

MylesBorins mentioned this pull request Jul 18, 2017
v6.11.2 proposal #14356
Merged

Uh oh!

Uh oh!

Conversation

seppevs commented Mar 27, 2017

Checklist

Affected core subsystem(s)

Uh oh!

aqrln commented Mar 27, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seppevs commented Mar 27, 2017

Uh oh!

vsemozhetbyt commented Mar 27, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vsemozhetbyt commented Mar 27, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vsemozhetbyt commented Mar 27, 2017

Uh oh!

aqrln commented Mar 27, 2017

Uh oh!

aqrln commented Mar 27, 2017

Uh oh!

seppevs commented Mar 27, 2017

Uh oh!

aqrln commented Mar 27, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vsemozhetbyt commented Mar 27, 2017

Uh oh!

Fishrock123 left a comment

Choose a reason for hiding this comment

Uh oh!

aqrlnMar 27, 2017

Choose a reason for hiding this comment

Uh oh!

seppevsMar 27, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jasnell left a comment

Choose a reason for hiding this comment

Uh oh!

refack commented Mar 29, 2017

Uh oh!

aqrln commented Mar 29, 2017

Uh oh!

refackMar 29, 2017

Choose a reason for hiding this comment

Uh oh!

refackMar 29, 2017

Choose a reason for hiding this comment

Uh oh!

refack left a comment

Choose a reason for hiding this comment

Uh oh!

jasnell commented Apr 4, 2017

Uh oh!

seppevs commented Apr 4, 2017

Uh oh!

mscdex commented Apr 5, 2017

Uh oh!

aqrln commented Apr 12, 2017

Uh oh!

jasnell commented Apr 13, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

refack commented Apr 13, 2017

Uh oh!

refack commented Apr 13, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

aqrln commented Mar 27, 2017•
edited
Loading

vsemozhetbyt commented Mar 27, 2017•
edited
Loading

vsemozhetbyt commented Mar 27, 2017•
edited
Loading

aqrln commented Mar 27, 2017•
edited
Loading

seppevsMar 27, 2017•
edited
Loading

jasnell commented Apr 13, 2017•
edited
Loading