build: run cctests as part of test-ci target

[bnoordhuis]

For discussion.

CI: https://ci.nodejs.org/job/node-test-pull-request/3591/

[bnoordhuis]

Different take that moves merging of the TAP files out of tools/test.py and into a standalone script: https://ci.nodejs.org/job/node-test-pull-request/3592/

[richardlau]

Will this work without renumbering the individual tests in the files being merged?
As it stands I think we'll end up with multiple ok 1 ....

[jbergstroem]

cc @nodejs/build

[joaocgreis]

On Windows, vcbuild test-ci already runs cctest but ignores the results. Here is the error from a Debug build:

Running main() from gtest_main.cc
[==========] Running 22 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 4 tests from UtilTest
[ RUN ] UtilTest.ListHead
[ OK ] UtilTest.ListHead (0 ms)
[ RUN ] UtilTest.StringEqualNoCase
[ OK ] UtilTest.StringEqualNoCase (0 ms)
[ RUN ] UtilTest.StringEqualNoCaseN
[ OK ] UtilTest.StringEqualNoCaseN (0 ms)
[ RUN ] UtilTest.ToLower
[ OK ] UtilTest.ToLower (0 ms)
[----------] 4 tests from UtilTest (0 ms total)
[----------] 18 tests from InspectorSocketTest
[ RUN ] InspectorSocketTest.ReadsAndWritesInspectorMessage
Assertion failed: backlog > 0, file src\win\tcp.c, line 561

[bnoordhuis]

Removed the script again, made changes to vcbuild.bat and added a -Wformat fix for the inspector cctest because I didn't feel like filing a separate PR for that. :-)

New CI: https://ci.nodejs.org/job/node-test-pull-request/3609/

@ofrobots@eugeneo Running the cctests seems to have unearthed some issues. Several of the buildbots fail like this (from https://ci.nodejs.org/job/node-test-commit-linux/4529/nodes=debian8-x86/console):

# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1
not ok 9 - InspectorSocketTest.ExtraLettersBeforeRequest
---
duration_ms: 0.001
...
# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1
not ok 10 - InspectorSocketTest.RequestWithoutKey
---
duration_ms: 0
...
# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1
ok 11 - InspectorSocketTest.KillsConnectionOnProtocolViolation

[bnoordhuis]

Rebased, let's see if the test failures have been resolved: https://ci.nodejs.org/job/node-test-pull-request/3809/

[eugeneo]

I'm working on it - failures are still there...

On Tue, Aug 23, 2016 at 12:10 PM Ben Noordhuis notifications@github.com
wrote:

Rebased, let's see if the test failures have been resolved:
https://ci.nodejs.org/job/node-test-pull-request/3809/

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#8034 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AARkrScGJo88I6TjfrW1w9jXvQZkVG6Fks5qi0WggaJpZM4JgM5v
.

[eugeneo]

FYI - the test on Windows hit #8155, I'll create a PR once I fix it.

[bnoordhuis]

Rebased. New CI run: https://ci.nodejs.org/job/node-test-pull-request/4007/

[bnoordhuis]

@eugeneo cctest failures still appear to be unfixed:

not ok 8 - InspectorSocketTest.ExtraTextBeforeRequest
---
duration_ms: 0
...
# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1
not ok 9 - InspectorSocketTest.ExtraLettersBeforeRequest
---
duration_ms: 0
...
# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1
not ok 10 - InspectorSocketTest.RequestWithoutKey
---
duration_ms: 0
...
# Value of: 0
# Expected: uv_is_active(reinterpret_cast<uv_handle_t*>(&socket))
# Which is: 1

[eugeneo]

cctest works for me on Windows, Mac and Linux after I apply #8528

[bnoordhuis]

CI with fixes from #8505 and #8528 incorporated: https://ci.nodejs.org/job/node-test-pull-request/4044/

EDIT: Green, except for an infrastructure failure on one of the ARM buildbots.

[bnoordhuis]

Rebased. New CI: https://ci.nodejs.org/job/node-test-pull-request/4243/

[bnoordhuis]

Quite a few flakes but none related to this PR. Can I get a LGTM?

[bnoordhuis]

Back-porters, commit b3d283a (the last one) should be omitted when back-porting to v4.x (but not v6.x.)

[mscdex]

@bnoordhuis Why did the default result printer switch from pretty print to TAP? I get TAP output from cctest when running make test locally now.

[MylesBorins]

waiting to backport this until we have consensus on the tap output... has this been decided on master yet?

[mscdex]

@thealphanerd The default reporter format change was reverted in master.

[MylesBorins]

@mscdex can you give me the PR that happened in?

[mscdex]

@thealphanerd#8948

[MylesBorins]

@bnoordhuis or @mscdex would one of you be willing to backport this with the commit from #8948?

[jbergstroem]

and on top, #9262.

[mscdex]

@thealphanerd#9682