Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add simple allocation benchmarks #291

Merged
merged 7 commits into from
Jan 26, 2022
Merged

Conversation

vilterp
Copy link
Contributor

@vilterp vilterp commented Jan 20, 2022

cc @vchuravy @NHDaly

A simple-as-possible benchmark to help us see the impact of JuliaLang/julia#42768, JuliaLang/julia#43868 etc.

Was also thinking of using @timed and asserting against gc_stats.poolalloc to make sure we're actually measuring pool allocations.

Here it is running on JuliaLang/julia#43868:

julia> res_pr["strings"]
BenchmarkTools.Trial: 266 samples with 1 evaluation.
 Range (min … max):  16.121 ms … 42.947 ms  ┊ GC (min … max): 0.00% … 16.64%
 Time  (median):     16.783 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   18.576 ms ±  3.231 ms  ┊ GC (mean ± σ):  8.38% ± 10.70%

   ▅█▃                                                         
  ▆████▇▄▄▃▁▂▂▁▁▂▂▁▁▁▁▂▁▁▁▁▁▂▁▁▁▁▁▂▁▃▁▄▅▄▅▆▄▃▃▃▃▃▁▁▁▁▂▁▂▂▁▁▁▂ ▃
  16.1 ms         Histogram: frequency by time        25.3 ms <

 Memory estimate: 25.17 MiB, allocs estimate: 549000.

julia> res_pr["arrays"]
BenchmarkTools.Trial: 1504 samples with 1 evaluation.
 Range (min … max):  2.628 ms … 35.518 ms  ┊ GC (min … max): 0.00% … 76.78%
 Time  (median):     2.842 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   3.282 ms ±  1.582 ms  ┊ GC (mean ± σ):  9.75% ± 14.90%

  █▇▇▅▄▃▂▂ ▁                                     ▁            
  ████████████▇▆▅▅▅▆▁▄▅▄▅▅▄▁▁▁▁▁▄▄▁▄▁▄▁▄▁▄▁▁▁▁▅▇███▆▆▄▅▅▅▅▄▆ █
  2.63 ms      Histogram: log(frequency) by time     8.66 ms <

 Memory estimate: 5.35 MiB, allocs estimate: 100100.

julia> res_pr["structs"]
BenchmarkTools.Trial: 5105 samples with 1 evaluation.
 Range (min … max):  557.607 μs … 16.754 ms  ┊ GC (min … max):  0.00% … 93.99%
 Time  (median):     683.703 μs              ┊ GC (median):     0.00%
 Time  (mean ± σ):   963.428 μs ±  1.185 ms  ┊ GC (mean ± σ):  24.99% ± 17.89%

  █▇▆▄▂                                                        ▁
  ██████▇▇▅▃▄▅▃▁▃▄▃▅▁▁▃▁▁▁▁▃▁▁▁▁▁▃▁▁▁▁▁▁▁▃▁▃▁▆▇▇▆▇▆▆▆▆▇▇▇▇▆▆▇▅ █
  558 μs        Histogram: log(frequency) by time       6.8 ms <

 Memory estimate: 3.83 MiB, allocs estimate: 100100.

julia> res_pr["grow_array"]
BenchmarkTools.Trial: 1831 samples with 1 evaluation.
 Range (min … max):  2.166 ms … 19.977 ms  ┊ GC (min … max):  0.00% … 86.57%
 Time  (median):     2.353 ms              ┊ GC (median):     0.00%
 Time  (mean ± σ):   2.707 ms ±  1.839 ms  ┊ GC (mean ± σ):  11.24% ± 13.48%

  █▇▄▂                                                        
  ████▆▆▅▆▃▁▃▃▁▁▁▁▁▁▁▁▁▃▁▁▁▃▁▁▁▁▁▁▁▃▁▅▁▄▁▁▁▃▁▃▄▃▅▃▁▄▅▅▄▆▃▁▃▄ █
  2.17 ms      Histogram: log(frequency) by time     13.6 ms <

 Memory estimate: 2.57 MiB, allocs estimate: 48911.

src/alloc/AllocBenchmarks.jl Outdated Show resolved Hide resolved
@vtjnash
Copy link
Member

vtjnash commented Jan 20, 2022

Can we add a GC.safepoint() call and Core.Intrinsics.atomic_fence() (if available) also, so that the compiler is not permitted to defer the assignment to the loop footer and then skip the loop

@vilterp
Copy link
Contributor Author

vilterp commented Jan 25, 2022

here's a gist of the results, running on JuliaLang/julia#43868 vs master. Shows no regressions

https://gist.github.com/vilterp/2444a133822427238e46eb23f5b60925

(though honestly running on my laptop is not consistent)

@vilterp
Copy link
Contributor Author

vilterp commented Jan 25, 2022

Moved some work to inner loops to offset the impact of calling Threads.atomic_fence()

Copy link
Contributor

@NHDaly NHDaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. Thanks for setting it up! :) 👍 👍

@vilterp vilterp requested review from vchuravy and vtjnash January 25, 2022 22:40
@vtjnash vtjnash merged commit bcec214 into JuliaCI:master Jan 26, 2022
@NHDaly
Copy link
Contributor

NHDaly commented Jan 29, 2022

Excellent, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants