Add two child-process stdio flush tests by petrosagg · Pull Request #7257 · nodejs/node

petrosagg · 2016-06-10T08:25:54Z

Checklist

make -j4 test (UNIX) or vcbuild test nosign (Windows) passes
a test and/or benchmark is included
the commit message follows commit guidelines

Affected core subsystem(s)

child-process

Description of change

Fixed a previous testcase that wasn't actually triggering the condition it tried to test. Added a new test that needs #7160 to be merged for it to pass.

While commit 12274a5 fixed data loss when a readable event handler is attached, the problem still exists when the stdio stream has a piped consumer that doesn't read fast enough. Signed-off-by: Petros Angelatos <petrosagg@gmail.com>

In order for the testcase to fail, the calls to read() must be done after 'exit' is emitted and after flushStdio has been called. With this change, the testcase will catch any regressions on this issue. Signed-off-by: Petros Angelatos <petrosagg@gmail.com>

addaleax · 2016-06-10T08:43:36Z

@petrosagg Thanks, this is appreciated a lot! It would be great if we could replace spawning seq with another command or e.g. add a seq.js to the test/fixtures directory that implements seq in a cross-platform way. Otherwise, it would probably be best to skip that part of the test on Windows for now.

The subject line of the first commit should also probably start with test: (even though it affects only tests for a specific subsystem).

addaleax · 2016-06-10T08:45:24Z

@mscdex The second commit here (the setImmediate change) changes the test case from #5036 so that it fails when executed with node binaries from before that change, which it currently does not afaict, would be great if you could take a look at that.

mscdex · 2016-06-11T15:28:12Z

@addaleax I'm not sure what you mean. The existing test (without the setImmediate() addition) definitely does fail on node before the merge of #5036 (for example with node v5.5.0). Adding the setImmediate() does not seem to change things, but if that is the case, why does it need to be added?

addaleax · 2016-06-11T18:15:55Z

@mscdex Okay, sorry, I probably got something mixed up – thing is, reverting the lib/ change from #5036 on top of the current master leaves the test working (at least for me locally on Linux). I’ll see if I can figure out what changed in the meantime.

addaleax · 2016-06-11T19:50:04Z

@mscdex Okay, so, apparently the order of some child-process-related I/O events changed in libuv/libuv#611, making the test work even without your fix from #5036… would that make sense to you?

mscdex · 2016-06-11T20:18:56Z

@addaleax if that's the case, then I'm fine with that change

cjihrig · 2016-06-13T14:30:13Z

+function spawnWithPipe() {
+ const buffer = [];
+ const through = new stream.PassThrough();
+ const p = cp.spawn('seq', [ '36000' ]);


Can you remove the spaces inside the square braces.

addaleax · 2016-06-14T11:36:21Z

Initial CI (rebased against master to include #7160): https://ci.nodejs.org/job/node-test-commit/3741/

addaleax · 2016-06-15T18:58:38Z

CI is red with timeouts in parallel/test-child-process-flush-stdio on a couple of platforms – not what I would have expected.

addaleax · 2016-06-23T02:13:11Z

ping @petrosagg

The test timeouts seem to stem from seq filling up the pipe buffer while the PassThrough is not consuming it, making seq stall and never actually exit (You can test it on Linux by increasing the counter to e.g. 360000).

I’m not sure how that’s best addressed. Killing the child process after a short timeout seems a bit crude, but it’s the most reliable thing that comes to my mind right now.

addaleax · 2016-07-04T19:23:21Z

@petrosagg Still interested in pursuing this? I’d definitely like to get these tests in in some way, and if you’re not feeling like it, I’d like to offer to take over.

jasnell · 2017-03-01T01:33:48Z

ping @addaleax ... still interested in pursuing this?

addaleax · 2017-03-01T15:20:13Z

yeah let’s close this out, the underlying issues in the streams implementation have been fixed & have tests

petrosagg added 2 commits June 10, 2016 00:59

child-process: add stdio flush testcase with pipes
3685d04
While commit 12274a5 fixed data loss when a readable event handler is attached, the problem still exists when the stdio stream has a piped consumer that doesn't read fast enough. Signed-off-by: Petros Angelatos <petrosagg@gmail.com>

nodejs-github-bot added the test Issues and PRs related to the tests. label Jun 10, 2016

petrosagg mentioned this pull request Jun 10, 2016
Fix child process stdio data loss with slow piped consumers #7185
Closed
4 tasks

addaleax added child_process Issues and PRs related to the child_process subsystem. lts-watch-v4.x labels Jun 10, 2016

cjihrig reviewed Jun 13, 2016
View reviewed changes

addaleax mentioned this pull request Jun 13, 2016
Streams Regression in v4.x nodejs/Release#112
Closed

addaleax self-assigned this Jun 23, 2016

addaleax mentioned this pull request Jul 17, 2016
test: fix child_process testcase to actually detect regression #7774
Closed

Trott force-pushed the master branch from b0df363 to c5ce7f4Compare September 21, 2016 00:09

rvagg force-pushed the master branch 2 times, most recently from c133999 to 83c7a88Compare October 18, 2016 17:02

MylesBorins force-pushed the master branch from 8df7ee0 to 54fef67Compare February 1, 2017 01:00

jasnell added the stalled Issues and PRs that are stalled. label Mar 1, 2017

addaleax closed this Mar 1, 2017

Uh oh!

Uh oh!

Conversation

petrosagg commented Jun 10, 2016

Checklist

Affected core subsystem(s)

Description of change

Uh oh!

addaleax commented Jun 10, 2016

Uh oh!

addaleax commented Jun 10, 2016

Uh oh!

mscdex commented Jun 11, 2016

Uh oh!

addaleax commented Jun 11, 2016

Uh oh!

addaleax commented Jun 11, 2016

Uh oh!

mscdex commented Jun 11, 2016

Uh oh!

cjihrigJun 13, 2016

Choose a reason for hiding this comment

Uh oh!

addaleax commented Jun 14, 2016

Uh oh!

addaleax commented Jun 15, 2016

Uh oh!

addaleax commented Jun 23, 2016

Uh oh!

addaleax commented Jul 4, 2016

Uh oh!

jasnell commented Mar 1, 2017

Uh oh!

addaleax commented Mar 1, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants