Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use stderr instead of file #365

Merged
merged 1 commit into from
Sep 11, 2018
Merged

Use stderr instead of file #365

merged 1 commit into from
Sep 11, 2018

Conversation

waghanza
Copy link
Collaborator

Hi,

Benchmark results (wrk results) are written in file, and this one is parsed as CSV.

To simplify process, it will be convenient to use stderr (memory) to write this CSV.

Regards,

@waghanza
Copy link
Collaborator Author

This could impact latency metrics. As an example on the same environment, I have (for a concurrency of 1000)

  • master
Language (Runtime) Framework (Middleware) Average 50% percentile 90% percentile 99% percentile 99.9% percentile Standard deviation
ruby roda 2.95 ms 2.04 ms 6.29 ms 15.79 ms 65.76 ms 3219.67
ruby rack-routing 3.80 ms 2.33 ms 8.53 ms 24.83 ms 113.65 ms 5220.33
ruby hanami 6.24 ms 2.82 ms 15.89 ms 47.66 ms 144.62 ms 9571.00
ruby flame 6.65 ms 3.74 ms 16.38 ms 37.75 ms 123.35 ms 8038.67
ruby sinatra 8.23 ms 4.24 ms 19.48 ms 64.23 ms 185.05 ms 12433.00
ruby rails 30.17 ms 7.44 ms 96.35 ms 184.99 ms 356.68 ms 44082.33
  • here
Language (Runtime) Framework (Middleware) Average 50% percentile 90% percentile 99% percentile 99.9% percentile Standard deviation
ruby sinatra 9.27 ms 1.93 ms 27.10 ms 85.32 ms 226.08 ms 17126.67
ruby roda 3.10 ms 2.02 ms 6.51 ms 18.96 ms 128.36 ms 4143.33
ruby rack-routing 3.79 ms 2.44 ms 8.45 ms 21.99 ms 106.48 ms 4616.67
ruby hanami 6.31 ms 2.83 ms 16.29 ms 46.90 ms 138.03 ms 9486.00
ruby flame 7.12 ms 3.66 ms 17.96 ms 43.42 ms 127.91 ms 9103.67
ruby rails 29.45 ms 7.34 ms 94.01 ms 179.29 ms 340.13 ms 42854.33

@waghanza waghanza merged commit c4c66c5 into the-benchmarker:master Sep 11, 2018
@waghanza waghanza deleted the io_result branch September 11, 2018 13:26
waghanza added a commit that referenced this pull request Sep 12, 2018
* use io (stderr chanel) instead of file on benchmark (#365)

* [Japronto] Support multicore (#366)

* support multicore

* Support multicore

* fix typo

* update koa to 2.5.3 (#367)

* setup dependencies.io

* specify python version on dependencies.yml

* specify python version under x.y.z format for deps
waghanza added a commit to qti3e/web-frameworks that referenced this pull request Oct 5, 2018
waghanza added a commit to qti3e/web-frameworks that referenced this pull request Oct 5, 2018
* use io (stderr chanel) instead of file on benchmark (the-benchmarker#365)

* [Japronto] Support multicore (the-benchmarker#366)

* support multicore

* Support multicore

* fix typo

* update koa to 2.5.3 (the-benchmarker#367)

* setup dependencies.io

* specify python version on dependencies.yml

* specify python version under x.y.z format for deps
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant