tty: set the handle to blocking mode

[Fishrock123]

Checklist

• tests and code linting passes
• a test and/or benchmark is included
• documentation is changed or added
• the commit message follows commit guidelines

Affected core subsystem(s)

tty, process

Description of change

See discussion at:

• ArrayBuffer <-> Buffer interop? #177 (comment)
• stdio buffered writes (chunked) issues & process.exit() truncation #6456 (comment)
• process: flush stdout/stderr upon process.exit() #6773

cc @nodejs/ctc @isaacs@ksc@saghul etc

Note: we still have more investigation to do but this is looking like it is probably the correct thing to do.

[indutny]

I have a deja vu here... Is it like 3rd time we re-enable it?

[indutny]

Am I right that running node script.js | some-prog-that-does-not-read-stdin will make script.js eventually hang on writes?

[addaleax]

@indutny This should only affect ttys, so, no, I don’t think so.

[indutny]

@addaleax oh, right! Thank you. LGTM then, I have no issues with this.

[mscdex]

Shouldn't there at least be an official way to change the behavior (in case someone needs/wants async)?

[Fishrock123]

I have a deja vu here... Is it like 3rd time we re-enable it?

Haha, not quite. ;)

@mscdex Official? Idk. But you can still do tty._handle.setBlocking(false) afaik.

That being said, I can't think of a reason you'd need async in a TTY? Maybe if only one stream is to a TTY?

[mscdex]

@Fishrock123 I meant a way without accessing an underscored property.

[saghul]

LGTM. (fingers crossed so we don't open a can of worms).

/cc @bnoordhuis I believe this whole thing make some sense in your brain, I'd love to see your yay or nay here.

[jasnell]

@mscdex ... like this? jasnell@4982550

[jasnell]

This PR should include a documentation update that (a) describes the default behavior of tty.ReadStream and (b) demonstrates a supported way of making it unblocking. I'm not comfortable with this landing until at least (a) is added.

[Fishrock123]

@jasnell So, there are a couple issues with your API proposal.

The biggest is that actually creating TTY streams isn't particularly supported:

In normal circumstances, process.stdout will be the only tty.WriteStream instance ever created (and only when isatty(1) is true).

In fact, I've tried to make tty streams before and found it's virtually impossible. You can only make one if the fd is a tty and it is not already in use, which is basically never. I actually don't know of a case when you could ever make one manually without severe hackery.

However, exposing a setBlocking method publicly also causes some concerns:

1. The existing problem that stdio streams do not have consistent APIs depending on the stream type is yet again amplified (very difficult to fix)
2. It doesn't work for pipes, but putting this functionality into net.Socket publicly is harmful because of TCP connections.
3. I'm not sure unblocking is ever valuable for this, in any manor that isn't strictly theoretical.
   • Same goes for pipes, where blocking only has theoretical niceties.

describes the default behavior of tty.ReadStream

I doubt that is useful due to the above, or the correct place to put it, since no-one will go looking for the TTY module from console.log() or process.std{in|out|err}.

I will add a note to where the existing note is for file piping though: https://nodejs.org/dist/latest-v6.x/docs/api/process.html#process_process_stderr

[Fishrock123]

Updated with docs note.

[addaleax]

nit: It’s usually stylized as Windows in the docs.

[addaleax]

LGTM

[mscdex]

@jasnell Something like that would be ideal of course, but even if there was a way to simply change the behavior once on startup. I'm not sure what that would look like or if enough people would be interested in supporting something like that though ...

[jasnell]

@mscdex ... I'm definitely for it (either API or command line switch) but given the discussion I doubt we'd get consensus to add it so I'm going to just drop it.

[jasnell]

I would say that this looks like a semver-major change but I know there will be disagreement with that.

[addaleax]

@jasnell Could you give an example of what you imagine might break because of this change?

[Fishrock123]

I can imagine a lot of things that are already breaking without this change. ;)

[Fishrock123]

@jasnell@mscdex I'd be open to it if there is actually a reason to do it, but no reasons are appearing...?

[jasnell]

@addaleax ... not off hand but it is a change in the established default behavior, which would technically qualify it as a semver-major. Note that I didn't add the label ;-) ... I'm quite certain that I'm in the minority.

[Fishrock123]

but it is a change in the established default behavior,

Sorta. It gets tricky here because on OS X it appears to have previously actually blocked when it wasn't supposed to so the majority of the CLI users never noticed this because it never changed on their platform sooner. So... the previous behavior is actually more established there, and due to previous versions of node.js, also across the board I think. Additionally, I'm unsure if console.log() blocks in the browser but I would not be particularly surprised if it did.

[mscdex]

I'm more inclined to say semver-major too since the non-blocking behavior in general has existed for quite a long time now in node, so I don't view this change as a "oops we just introduced a bug, let's fix it quickly" kind of situation. shrug

[addaleax]

As far as I can tell, the only visible difference would be in timing, no? And unlike with pipes, for ttys there is (in 99.99 % of practically occurring situations) an active consumer on the other end, so writing won’t block indefinitely because of this.

[Trott]

Nit: comma after 1

Nit: backticks around stdout and stderr

Nit: async->asynchronous

[Trott]

Left some nits on the documentation prose, but nothing that I think must happen before landing. (In other words: Don't interpret my nits as me objecting to this landing.)

[MylesBorins]

@Fishrock123 is this something that should be included in v4.5.0?

[saghul]

@kzc A note to remember: when stdio was made blocking on macOS that was meant to restore the behavior aas in v4.x On other Unix systems it never blocked. We are changing that now. Is it a good or a bad thing? I no longer know.

[kzc]

@saghul I know the history well. The blocking on Mac was initially inadvertently introduced as result of a bug pre-node 4.x when trying to re-open the tty - ironically to not block stdio - and somehow this blocking stdio behavior on Mac became grandfathered as the status quo.

Whether this PR is good or bad, it just addresses that inconsistency on UNIX platforms. #6816 (comment) sums up my view. Blocking stdio tty alone is not enough to fix the many long standing problems with node stdio. Something like #6773 is still needed to resolve truncated piped stdout/stderr and some similar fix is still required to flush piped stdio in the event of an uncaught exception. Once node is in the process of an abrupt exit and the event loop is no longer running (although not destroyed) there's zero downside to flushing stdout and stderr as users would reasonably expect.

As I don't watch these node issue/PR threads any longer please use my @ handle if you want to get any other feedback.

[saghul]

@kzc Sure, I know we are currently not in a very good place. Thanks for your efforts so far.

[Fishrock123]

@thealphanerd Give it's potentially debatable if this is a major, I'm going to say no.

[Fishrock123]

I consider it a non-breaking bugfix given that:

• There are no shortage of complaints about the current behavior
• There is no evidence this could cause problems

[Fishrock123]

New CI: https://ci.nodejs.org/job/node-test-pull-request/3324/

@jasnell Could you take a look at this again and let us know if you are comfortable with it?

[thefourtheye]

We don't need the additional square brackets for links I guess.

[Fishrock123]

Huh? These should use links defined at the bottom of the file...?

[thefourtheye]

If we have defined them at the bottom the empty square brackets are not necessary I think

[jasnell]

LGTM but we'll need to message this properly. While I believe there is consensus around landing this as a bug fix, it is technically a fairly significant change in behavior that should technically be tagged as a semver-major. There should be a release candidate cycle for any release that this goes out in just to be on the safe side.

[Fishrock123]

If it must land in a semver-major, it could.

I do not think it is reasonable to see this as an API change however, so I think it may be worth our while to get this in before v6 goes LTS.

[addaleax]

This still LGTM, and I’m still +1 on not seeing this as a semver-major because afaict, this doesn’t necessarily represent a visible behavioural change.

[Fishrock123]

Guess we're clear to land... or as clear as it will get.

@jasnell how do you propose we message it?

[jasnell]

Twitter PSA at the very least (I can do that in just a minute). Even tho this will land as a patch, we need to make sure it's called out explicitly in the release notes. Beyond that, I'm not sure what else.

[jasnell]

I would definitely recommend that we do a release candidate for whatever release this lands in.

[addaleax]

I’ll be landing this later today if there are no objections and nobody else does before me, this has been lying around long enough.

edit: One more CI: https://ci.nodejs.org/job/node-test-commit/4501/

[addaleax]

Landed in ab3306a