Stdout is not completely flushed on process exit

[tschaub]

The following fails with Node 4.1 (but passes on 0.12 and 0.10) on OS 10.10.5:

var assert = require('assert');
var spawn = require('child_process').spawn;

var out = new Array(Math.pow(2, 13)).join('.') + '!!';

if (process.argv[2] === '--child') {
  process.stdout.write(out);
  process.exit(0);
} else {
  var child = spawn(process.argv[0], [process.argv[1], '--child']);

  var got = '';
  child.stdout.on('data', function(data) {
    got += String(data);
  });

  child.on('exit', function() {
    assert.equal(got, out);
  });
}

The child writes 8193 bytes to stdout. The parent only gets 8192 before the exit.

See also nodejs/node-v0.x-archive#8329.

[mscdex]

Duplicate of #784, #1771, #1741, #2148.

[tschaub]

It's not clear to me from the above issues that they refer to the same regression between 0.12 and 4.0.

In #2148, you mention the same problem occurs with 0.12 (this is not the case here). #1741 and #2148 are also about writing to stdout/err in a loop.

The test case above writes a single string to stdout. It fails when the string is 8193 bytes. It passes at 8192. And this only occurs with Node 4 (not 0.10 or 0.12).

[bnoordhuis]

The issue is that process.stdout.write() is asynchronous but doesn't get time to run to completion due to the process.exit() call immediately afterwards. It's a variation of (at least some of) the issues @mscdex linked to.

[bnoordhuis]

Forgot to mention, in your example you can fix the issue by dropping the immediate process.exit() call and rewriting the process.stdout.write() call to process.stdout.write(out, process.exit).

[tschaub]

Forgot to mention, in your example you can fix the issue by dropping the immediate process.exit() call and rewriting the process.stdout.write() call to process.stdout.write(out, process.exit).

Thanks for the extra detail @bnoordhuis. Still curious what exactly changed since 0.12 that causes the test above to fail at 8192 bytes. Any chance this size is configurable?

In my case, I cannot change where process.exit() is called in the child process.

[bnoordhuis]

Still curious what exactly changed since 0.12 that causes the test above to fail at 8192 bytes. Any chance this size is configurable?

Stdio is fully asynchronous now (and that's not configurable.) It used to be synchronous, mostly, but with so many caveats that even core team members often didn't know when it would or wouldn't be.

[tschaub]

Stdio is fully asynchronous now (and that's not configurable.) It used to be synchronous, mostly, but with so many caveats that even core team members often didn't know when it would or wouldn't be.

Thanks @bnoordhuis. It looks like the code from 20176a9 is still there, and the docs were recently changed to say that process.stdout.write() always blocks (see 892bf65). So process.stdout.write() blocks but is asynchronous?

It sounds like the only way to ensure that process.stdout.write() results in a write to stdout is to never call process.exit() (assuming you may not control all the call sites of process.stdout.write() or call it in a loop etc.).

[Fishrock123]

Duplicate of #784, #1771, #1741, #2148.

Oh dear, this again.

It sounds like the only way to ensure that process.stdout.write() results in a write to stdout is to never call process.exit() (assuming you may not control all the call sites of process.stdout.write() or call it in a loop etc.).

Yes. process.exit() terminates the process as soon as possible. It's not "safe" per-say.

Still curious what exactly changed since 0.12 that causes the test above to fail at 8192 bytes. Any chance this size is configurable?

That's when something at OS(?)-level starts breaking it into chunks.

[Fishrock123]

(Closing as a dupe, feel free to continue discussing)

[Fishrock123]

Also, updated the breaking change documentation on this: https://github.com/nodejs/node/wiki/API-changes-between-v0.10-and-v4#process

[bnoordhuis]

For people coming here from Google, this:

Stdio is fully asynchronous now

Is correct at the time of writing, except when stdio is redirected to a file. That includes special files like /dev/null and /dev/zero but excludes /dev/tty, /dev/stdout, etc. and bash's /dev/tcp pseudo-files.

See also #3170, where I was very careful to use the word 'may' in 'may block'. :-)

[mikemaccana]

See #6456 for the latest.