parallel/test-tls-server-verify is awfully slow on Windows #1461

jbergstroem · 2015-04-18T05:43:02Z

Running this test on one of our jenkins windows bots consistently takes more than 35 seconds, while taking 0.6s on my (os x) desktop.

mathiask88 · 2015-04-30T09:48:03Z

I think the cause for the long runtime is in OpenSSL. The s_client on windows generates extra entropy by loading screen memory and this takes ~1-2s on each spawn. I don't get why they are not using something like windows CryptGenRandom() function but I'm no openssl or even crypto expert so correct me if I am wrong.

jbergstroem · 2015-05-12T03:00:53Z

@mathiask88 thanks for digging that up. Seems to have been around for a while.

trevnorris · 2015-05-12T04:14:45Z

/cc @indutny @piscisaureus on using CryptGetRandom()

piscisaureus · 2015-05-12T04:17:37Z

@mathiask88 The test has always been slow for me, but I never got any further than figuring out that the openssl command-line client is really slow. Thanks for digging this up.

mathiask88 · 2015-05-13T10:43:21Z

@jbergstroem Yes and I think it is not necessary. The client runs app_RAND_load_file() that returns on windows something like C:\.rnd and tries to read that entropy file. If this fails RAND_status() gets called and if this is called for the first time openssl gets initialized with RAND_poll() and that function calls CryptGetRandom() on Windows NT+ or Windows CE 3.0+ anyway. So I think RAND_screen() is a relict where OpenSSL didn't use an OS-based seed initialization.

But I tested a bit more and it seems that the screen loading is not the bottleneck here. It doesn't take ~1s as I thought. If I run a server like in the test and then a *.bat file with echo QUIT | openssl-cli.exe s_client -connect 127.0.0.1:12346 the openssl-cli takes about 2s to exit after _(un)authed.

orangemocha · 2015-05-14T11:58:09Z

The test is slow on Windows because of two 1 second delays in the Windows version of openssl-cli.exe. We have reported the issue to openssl: http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=3849

Now working on a workaround, modifying the test to let all those test cases run in parallel instead of serialized, so that it finishes quickly in spite of the openssl-cli delay.

We will submit the change to io.js as well as Node.

Fishrock123 · 2015-05-14T14:27:21Z

We will submit the change to io.js as well as Node.

Awesome, thanks!

orangemocha · 2015-05-22T16:19:31Z

Do you folks mind reviewing the change here: nodejs/node-v0.x-archive#25368, to make sure we have a consistent +1? I can then cherry pick the commits to io.js in another PR. Thanks!

rvagg · 2015-05-23T00:42:25Z

/cc @nodejs/crypto to @orangemocha's comment above, sounds positive to me but could do with a word of encouragement from someone in the crypto team

shigeki · 2015-05-27T05:45:36Z

I looked at this and found that we can have one more improvement to use child.kill() not to wait for child processes to be ended. shigeki@ad45f88

RAND_screen() on my Windows takes more than one second. I think we need not to have a good client randomness in this test. Adding a -no-rand-screen option for openssl s_client on Windows is one of ideas to remove this overhead. shigeki@12be7ec and shigeki@dae36fb

The benchmark results are

Current
TotalSeconds      : 86.2152129

joyent/node#25368 
TotalSeconds      : 41.0233476

joyent/node#25368 + child.kill()
TotalSeconds      : 33.9199469

joyent/node#25368 + child.kill() + -no-rand-screen
TotalSeconds      : 9.5351881

9.5 seconds are still slow compared to Unix but it seems to be enough good for CI.

jbergstroem · 2015-05-27T05:49:04Z

I'm all for --no-rand-screen; this openssl binary isn't used for anything other than verifying our test suite.

orangemocha · 2015-05-27T13:36:36Z

Regarding using child.kill(), I wonder if this changes the test in way that could limit its effectiveness in catching bugs. But I will add shigeki@ad45f88 to the PR and let people who are more experienced in this area provide feedback.

Regarding, --no-rand-screen it sounds like a wonderful feature. But it would be a floating patch on a dependency, and I wonder if that would be appropriate here. In node we usually tend to limit those floating patches to changes that have already been made in the upstream project, or are high priority and at least have a clear path for being accepted upstream.

orangemocha · 2015-05-27T14:19:15Z

Updated nodejs/node-v0.x-archive#25368

shigeki · 2015-05-28T02:35:55Z

@orangemocha The patch of -no-rand-screen would not be accepted to the upstream because it can not be applied in a general use. This fix was made carefully to be as small as minimum and limited to have changes of only openssl-cli.exe binary so that libopenssl.a linked to node is nothing to do with it.

There are private patches which are already applied in deps/openssl. For openssl-1.0.2a case, these are
https://github.com/nodejs/io.js/blob/master/deps/openssl/doc/UPGRADING.md#2-apply-private-patches
But I agree that those private patches should be kept in minimum.

orangemocha · 2015-05-28T16:02:13Z

@shigeki : added all your commits to nodejs/node-v0.x-archive#25368 to get feedback there.

The PR at nodejs/node-v0.x-archive#25368 is ready for review. cc @nodejs/crypto

shigeki · 2015-05-29T02:02:27Z

@indutny @bnoordhuis Could you give a comment on shigeki@12be7ec if we can add it into deps/openssl? It surely saves our time to wait for CI results on Windows.

bnoordhuis · 2015-05-29T10:55:19Z

@orangemocha @shigeki If you want your changes to get landed in io.js, can you file them as a PR here?

orangemocha · 2015-05-29T11:43:26Z

sure @bnoordhuis , against v1.x?

bnoordhuis · 2015-05-29T11:45:48Z

@orangemocha master (or next if you want it to go into v3.x.) If it needs to get back-ported to a v1.x. release, please mention that in the pull request.

orangemocha · 2015-05-29T17:12:28Z

io.js PR is here: #1836

OpenSSL s_client introduces some delay on Windows. With all clients running sequentially, this delay is big enough to break CI. This fix runs the clients in parallel (unless the test includes renegotiation), reducing the total run time. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

Different servers must use different ports. Since we can count only on common.PORT and common.PORT+1, run only 2 servers in parallel. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

When running in parallel, it is not easy to identify what server and client failed when the test fails. This adds identifiers to all lines of console output. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

For better performance of the test, the parent kills child processes so as not to wait them to be ended. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Backport-PR-URL: #19638 Fixes: #1461 PR-URL: #1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: #1461 PR-URL: #1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: nodejs#1461 PR-URL: nodejs#1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: #1461 Backport-PR-URL: #28230 PR-URL: #1836 Reviewed-By: Ben Noordhuis <[email protected]>

In openssl s_client on Windows, RAND_screen() is invoked to initialize random state but it takes several seconds in each connection. This added -no_rand_screen to openssl s_client on Windows to skip RAND_screen() and gets a better performance in the unit test of test-tls-server-verify. Do not enable this except to use in the unit test. Fixes: nodejs/node#1461 PR-URL: nodejs/node#1836 Reviewed-By: Ben Noordhuis <[email protected]>

jbergstroem added windows Issues and PRs related to the Windows platform. test Issues and PRs related to the tests. labels Apr 18, 2015

jbergstroem mentioned this issue May 12, 2015

test/simple/test-tls-server-verify.js times out on our Windows 2012 Jenkins agent nodejs/node-v0.x-archive#25279

Closed

orangemocha mentioned this issue May 12, 2015

test,win: disable test-tls-server-verify for CI nodejs/node-v0.x-archive#25284

Closed

orangemocha mentioned this issue May 22, 2015

test,win: speedup tls-server-verify nodejs/node-v0.x-archive#25368

Closed

orangemocha mentioned this issue May 29, 2015

test,win: speedup tls-server-verify #1836

Closed

adamlaska mentioned this issue Jun 23, 2023

[Snyk] Fix for 1 vulnerabilities adamlaska/node#110

Open

aliscco mentioned this issue Jun 24, 2023

[Snyk] Fix for 1 vulnerabilities aliscco/alisco-node#576

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

parallel/test-tls-server-verify is awfully slow on Windows #1461

parallel/test-tls-server-verify is awfully slow on Windows #1461

jbergstroem commented Apr 18, 2015

mathiask88 commented Apr 30, 2015

jbergstroem commented May 12, 2015

trevnorris commented May 12, 2015

piscisaureus commented May 12, 2015

mathiask88 commented May 13, 2015

orangemocha commented May 14, 2015

Fishrock123 commented May 14, 2015

orangemocha commented May 22, 2015

rvagg commented May 23, 2015

shigeki commented May 27, 2015

jbergstroem commented May 27, 2015

orangemocha commented May 27, 2015

orangemocha commented May 27, 2015

shigeki commented May 28, 2015

orangemocha commented May 28, 2015

shigeki commented May 29, 2015

bnoordhuis commented May 29, 2015

orangemocha commented May 29, 2015

bnoordhuis commented May 29, 2015

orangemocha commented May 29, 2015

parallel/test-tls-server-verify is awfully slow on Windows #1461

parallel/test-tls-server-verify is awfully slow on Windows #1461

Comments

jbergstroem commented Apr 18, 2015

mathiask88 commented Apr 30, 2015

jbergstroem commented May 12, 2015

trevnorris commented May 12, 2015

piscisaureus commented May 12, 2015

mathiask88 commented May 13, 2015

orangemocha commented May 14, 2015

Fishrock123 commented May 14, 2015

orangemocha commented May 22, 2015

rvagg commented May 23, 2015

shigeki commented May 27, 2015

jbergstroem commented May 27, 2015

orangemocha commented May 27, 2015

orangemocha commented May 27, 2015

shigeki commented May 28, 2015

orangemocha commented May 28, 2015

shigeki commented May 29, 2015

bnoordhuis commented May 29, 2015

orangemocha commented May 29, 2015

bnoordhuis commented May 29, 2015

orangemocha commented May 29, 2015