child_process: close pipe ends that are re-piped #21209
Conversation
nodejs-github-bot
added
the
child_process
|
Can we add regression tests here? |
|
@addaleax - sure. |
|
@addaleax - done, please have a look! |
|
@evanlucas @cjihrig are you ok with this landing? |
|
In theory this LGTM, but the test introduced by this PR is failing in the CI. |
|
my test logic is very straight forward:
in windows (with vs2017) this seems to produce 1272. could it be related to different semantics for vs2017? I will debug. |
|
@gireeshpunathil -- what's the status of this? (I tried to see the failure, but looks like the build results expired.) |
|
@elibarzilay - it is stuck on (some flavors of) windows failure of its own test case. I am running very low on bandwidth these weeks so kept it in the back burner. If you have some spare time to look at, please see why it fails on windows with VS 2017. |
|
@gireeshpunathil -- is there a way to re-run a build so I can see the failure? If not, then I'll try doing a Windows build, but that will obviously take more time... |
|
sure. @Trott or anyone from build team - can you please help us with running Windows CI against this PR (even better, if this single test is run in Windows) ? I can run full CI but doesn't look like the best way to use machines. |
|
@gireeshpunathil https://ci.nodejs.org/job/node-stress-single-test/1933/nodes=win2016-1p-vs2017/ If I didn't mess up the parameters, that should build from this PR and run the test 10 times. |
|
(Although due to the cross-compiling or whatever is used to speed up build in node-test-commit-windows-fanned, it might still be faster to run that. So here's one of those two: https://ci.nodejs.org/job/node-test-commit-windows-fanned/19162/ Again, hopefully I didn't mess up any of the parameters...) |
|
Ping... any updates on this one? |
|
@jasnell - here is where it is stuck: I have developed a test case for this PR, that is consistently failing in some windows flavors (I wasn't successful in recreating it in my local machines). I am unable to correlate the failure with my test logic, as explained in I should admit I left it unattended for few weeks now. I shall work towards resolving this soon. |
|
The original test case posed challenge in windows: when the spawned child (cmd.exe) echo text, the So in the new version I made it simple, and platform independent - just write an MB of data in a file, with lines of one KB. |
gireeshpunathil
added
the
author ready
when t0 and t1 are spawned with t0's outputstream [1, 2] is piped into t1's input, a new pipe is created which uses a copy of the t0's fd. This leaves the original copy in Node parent, unattended. Net result is that when t0 produces data, it gets bifurcated into both the copies Detect the passed handle to be of 'wrap' type and close after the native spawn invocation by which time piping would have been over. Fixes: nodejs#9413 Fixes: nodejs#18016 PR-URL: nodejs#21209 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
gireeshpunathil
removed
the
author ready
gireeshpunathil
force-pushed
the
close-bifurcated-stdio
branch
from
17258ca to
b1f82e4
Compare
|
landed as b1f82e4 |
when t0 and t1 are spawned with t0's outputstream [1, 2] is piped into t1's input, a new pipe is created which uses a copy of the t0's fd. This leaves the original copy in Node parent, unattended. Net result is that when t0 produces data, it gets bifurcated into both the copies Detect the passed handle to be of 'wrap' type and close after the native spawn invocation by which time piping would have been over. Fixes: #9413 Fixes: #18016 PR-URL: #21209 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
|
FYI: #25988 test is flaky on windows |
|
sure @refack , will have a look. |
Closing the underlying resource completely has the unwanted side effect that the stream can no longer be used at all, including passing it to other child processes. What we want to avoid is accidentally reading from the stream; accordingly, it should be sufficient to stop its readable side manually, and otherwise leave the underlying resource intact. Fixes: nodejs#27097 Refs: nodejs#21209
Closing the underlying resource completely has the unwanted side effect that the stream can no longer be used at all, including passing it to other child processes. What we want to avoid is accidentally reading from the stream; accordingly, it should be sufficient to stop its readable side manually, and otherwise leave the underlying resource intact. Fixes: #27097 Refs: #21209 PR-URL: #27373 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]>
Closing the underlying resource completely has the unwanted side effect that the stream can no longer be used at all, including passing it to other child processes. What we want to avoid is accidentally reading from the stream; accordingly, it should be sufficient to stop its readable side manually, and otherwise leave the underlying resource intact. Fixes: #27097 Refs: #21209 PR-URL: #27373 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]>
when t0 and t1 are spawned with t0's outputstream [1, 2] is piped into
t1's input, a new pipe is created which uses a copy of the t0's fd.
This leaves the original copy in Node parent, unattended. Net result is
that when t0 produces data, it gets bifurcated into both the copies
Detect the passed handle to be of 'wrap' type and close after the
native spawn invocation by which time piping would have been over.
Fixes: #9413
Fixes: #18016
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes