From cd14d275e7120ecffe20a37dd981790fab5a1504 Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Thu, 22 Aug 2024 22:42:15 +0100 Subject: [PATCH 1/5] start docs section about loading/precomp/ttfx time --- doc/src/manual/performance-tips.md | 110 +++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/doc/src/manual/performance-tips.md b/doc/src/manual/performance-tips.md index 38e27476f0af8..af72964fdae44 100644 --- a/doc/src/manual/performance-tips.md +++ b/doc/src/manual/performance-tips.md @@ -1394,6 +1394,116 @@ Prominent examples include [MKL.jl](https://github.com/JuliaLinearAlgebra/MKL.jl These are external packages, so we will not discuss them in detail here. Please refer to their respective documentations (especially because they have different behaviors than OpenBLAS with respect to multithreading). +## Execution latency, loading and precompiling time + +### Reducing time to first plot etc. + +The first time a julia function is called it will be compiled. The [`@time`](@ref) macro family illustrates this. + +``` +julia> foo() = rand(2,2) * rand(2,2) +foo (generic function with 1 method) + +julia> @time @eval foo(); + 0.252395 seconds (1.12 M allocations: 56.178 MiB, 2.93% gc time, 98.12% compilation time) + +julia> @time @eval foo(); + 0.000156 seconds (63 allocations: 2.453 KiB) +``` + +Note that `@time @eval ...` form is explained in the [`@time`](@ref) documentation. + +When developing a package, it can be helpful to cache this compilation in the package caches that are "precompiled" +during the `Pkg` installation process or before package load. To precompile package code effectively, it's recommended +to use [`PrecompileTools.jl`](https://julialang.github.io/PrecompileTools.jl/stable/) to run a "precompile workload" during +precompilation time that is representative of typical package usage, which will cache the native compiled code into the +package `pkgimage` cache, greatly reducing "time to first ..." (often referred to as TTFX) for such usage. + + +### Reducing package loading time + +When keeping the time taken to load the package down is usually helpful. +General good practice for package developers includes: + +1. Balancing the need to depending on packages for functionality vs. their load times, including stdlibs which on newer + julia versions may not be loaded already as stdlibs are no longer always included in the default sysimage. +2. Avoiding adding `__init__` functions to package modules unless unavoidable, especially those which might trigger a + lot of compilation, or just take a long time to execute. +3. Watch out for invalidations in the dependency tree during load that could be causing significant recompilation downstream. + +The `InteractiveUtils` tool [`@time_imports`](@ref) can be useful in the REPL to review the above factors. + +```julia-repl +julia> @time @time_imports using Plots + 0.5 ms Printf + 16.4 ms Dates + 0.7 ms Statistics + ┌ 23.8 ms SuiteSparse_jll.__init__() 86.11% compilation time (100% recompilation) + 90.1 ms SuiteSparse_jll 91.57% compilation time (82% recompilation) + 0.9 ms Serialization + ┌ 39.8 ms SparseArrays.CHOLMOD.__init__() 99.47% compilation time (100% recompilation) + 166.9 ms SparseArrays 23.74% compilation time (100% recompilation) + 0.4 ms SparseArraysExt + 0.5 ms TOML + 8.0 ms Preferences + 0.3 ms PrecompileTools + 0.2 ms Reexport +... many deps omitted for example ... + 1.4 ms Tar + ┌ 73.8 ms p7zip_jll.__init__() 99.93% compilation time (100% recompilation) + 79.4 ms p7zip_jll 92.91% compilation time (100% recompilation) + ┌ 27.7 ms GR.GRPreferences.__init__() 99.77% compilation time (100% recompilation) + 43.0 ms GR 64.26% compilation time (100% recompilation) + ┌ 2.1 ms Plots.__init__() 91.80% compilation time (100% recompilation) + 300.9 ms Plots 0.65% compilation time (100% recompilation) + 1.795602 seconds (3.33 M allocations: 190.153 MiB, 7.91% gc time, 39.45% compilation time: 97% of which was recompilation) + +``` + +Notice that in this example there are multiple packages loaded, some with `__init__()` functions, some of which cause +compilation of which some is recompilation. + +This report gives a good opportunity to review whether the cost of dependency load time is worth the functionality it brings. +Also the `Pkg` utility `why` can be used to report why a an indirect dependency exists. + +``` +(CustomPackage) pkg> why FFMPEG_jll + Plots → FFMPEG → FFMPEG_jll + Plots → GR → GR_jll → FFMPEG_jll +``` + +or to see the indirect dependencies that a package brings in, you can `pkg> rm` the package, see the deps that are removed +from the manifest, then revert the change with `pkg> undo`. + +If loading time is dominated by slow `__init__()` methods having compilation, one verbose way to identify what is being +compiled is to use the julia args `--trace-compile=stderr --trace-compile-timing` which will report a [`precompile`](@ref) +statement each time a method is compiled, along with how long compilation took. For instance, the full setup would be: + +``` +$ julia --startup-file=no --trace-compile=stderr --trace-compile-timing +julia> @time @time_imports using CustomPackage +... +``` + +Note the `--startup-file=no` which helps isolate the test from packages you may have in your `startup.jl`. + +More analysis of the reasons for recompilation can be achieved with the +[`SnoopCompile`](https://github.com/timholy/SnoopCompile.jl) package. + +### Reducing precompilation time + +If package precompilation is taking a long time, one option is to set the following internal and then precompile. +``` +julia> Base.PRECOMPILE_TRACE_COMPILE[] = "stderr" + +pkg> precompile +``` + +This has the effect of setting `--trace-compile=stderr --trace-compile-timing` in the precompilation processes themselves, +so will show which methods are precompiled and how long they took to precompile. + +There are also profiling options such as [using the external profiler Tracy to profile the precompilation process](@ref Profiling-package-precompilation-with-Tracy). + ## Miscellaneous From 9468810ef632726b107a0ce90ddf3deb106f028f Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Mon, 9 Sep 2024 20:44:22 -0400 Subject: [PATCH 2/5] add most of the review suggestions (wip) --- doc/src/manual/performance-tips.md | 46 +++++++++++++++++++----------- 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/doc/src/manual/performance-tips.md b/doc/src/manual/performance-tips.md index af72964fdae44..013000d56558a 100644 --- a/doc/src/manual/performance-tips.md +++ b/doc/src/manual/performance-tips.md @@ -1394,11 +1394,12 @@ Prominent examples include [MKL.jl](https://github.com/JuliaLinearAlgebra/MKL.jl These are external packages, so we will not discuss them in detail here. Please refer to their respective documentations (especially because they have different behaviors than OpenBLAS with respect to multithreading). -## Execution latency, loading and precompiling time +## Execution latency, package loading and package precompiling time ### Reducing time to first plot etc. -The first time a julia function is called it will be compiled. The [`@time`](@ref) macro family illustrates this. +The first time a julia method is called it (and any methods it calls, or ones that can be statically determined) will be +compiled. The [`@time`](@ref) macro family illustrates this. ``` julia> foo() = rand(2,2) * rand(2,2) @@ -1411,27 +1412,34 @@ julia> @time @eval foo(); 0.000156 seconds (63 allocations: 2.453 KiB) ``` -Note that `@time @eval ...` form is explained in the [`@time`](@ref) documentation. +Note that `@time @eval` is better for measuring compilation time because without [@eval](@ref), some compilation may +already be done before timing starts. -When developing a package, it can be helpful to cache this compilation in the package caches that are "precompiled" -during the `Pkg` installation process or before package load. To precompile package code effectively, it's recommended -to use [`PrecompileTools.jl`](https://julialang.github.io/PrecompileTools.jl/stable/) to run a "precompile workload" during -precompilation time that is representative of typical package usage, which will cache the native compiled code into the -package `pkgimage` cache, greatly reducing "time to first ..." (often referred to as TTFX) for such usage. +When developing a package, you may be able to improve the experience of your users with *precompilation*, +so that when they use the package the code they use is already compiled. To precompile package code effectively, it's +recommended to use [`PrecompileTools.jl`](https://julialang.github.io/PrecompileTools.jl/stable/) to run a +"precompile workload" during precompilation time that is representative of typical package usage, which will cache the +native compiled code into the package `pkgimage` cache, greatly reducing "time to first execution" (often referred to as +TTFX) for such usage. +Note that [`PrecompileTools.jl`](https://julialang.github.io/PrecompileTools.jl/stable/) workloads can be +disabled and sometimes configured via Preferences if you do not want to spend the extra time precompiling, which +may be the case during development of a package. ### Reducing package loading time -When keeping the time taken to load the package down is usually helpful. +Keeping the time taken to load the package down is usually helpful. General good practice for package developers includes: -1. Balancing the need to depending on packages for functionality vs. their load times, including stdlibs which on newer - julia versions may not be loaded already as stdlibs are no longer always included in the default sysimage. -2. Avoiding adding `__init__` functions to package modules unless unavoidable, especially those which might trigger a - lot of compilation, or just take a long time to execute. -3. Watch out for invalidations in the dependency tree during load that could be causing significant recompilation downstream. +1. Reduce your dependencies to those you really need. +2. Consider organizing any extended package functionality that is dependent on particular dependencies into + [package extensions](@ref) that can serve optional functionality depending on whether the user's environment (or the + package depending on yours) has those particular dependencies. +3. Avoid use of [`__init__()`](@ref) functions unless there is no alternative, especially those which might trigger a lot + of compilation, or just take a long time to execute. +4. Where possible, fix [invalidations](@ref) among your dependencies and from your package code. -The `InteractiveUtils` tool [`@time_imports`](@ref) can be useful in the REPL to review the above factors. +The tool [`@time_imports`](@ref) can be useful in the REPL to review the above factors. ```julia-repl julia> @time @time_imports using Plots @@ -1443,7 +1451,7 @@ julia> @time @time_imports using Plots 0.9 ms Serialization ┌ 39.8 ms SparseArrays.CHOLMOD.__init__() 99.47% compilation time (100% recompilation) 166.9 ms SparseArrays 23.74% compilation time (100% recompilation) - 0.4 ms SparseArraysExt + 0.4 ms Statistics → SparseArraysExt 0.5 ms TOML 8.0 ms Preferences 0.3 ms PrecompileTools @@ -1461,7 +1469,11 @@ julia> @time @time_imports using Plots ``` Notice that in this example there are multiple packages loaded, some with `__init__()` functions, some of which cause -compilation of which some is recompilation. +compilation of which some is recompilation. Recompilation is caused by earlier packages invalidating methods, then in +these cases when the following packages run their `__init__()` function some hit recompilation before the code can be run. + +Further, note the `Statistics` extension `SparseArraysExt` has been activated because `SparseArrays` is in the dependency +tree. i.e. see `0.4 ms Statistics → SparseArraysExt`. This report gives a good opportunity to review whether the cost of dependency load time is worth the functionality it brings. Also the `Pkg` utility `why` can be used to report why a an indirect dependency exists. From f59945a83ceb8825dd51b2fbcf2a05465cae69ed Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Thu, 12 Sep 2024 16:18:29 -0400 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Christian Guinard <28689358+christiangnrd@users.noreply.github.com> Co-authored-by: Tim Holy --- doc/src/manual/performance-tips.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/doc/src/manual/performance-tips.md b/doc/src/manual/performance-tips.md index 013000d56558a..8af693b86185c 100644 --- a/doc/src/manual/performance-tips.md +++ b/doc/src/manual/performance-tips.md @@ -1415,8 +1415,8 @@ julia> @time @eval foo(); Note that `@time @eval` is better for measuring compilation time because without [@eval](@ref), some compilation may already be done before timing starts. -When developing a package, you may be able to improve the experience of your users with *precompilation*, -so that when they use the package the code they use is already compiled. To precompile package code effectively, it's +When developing a package, you may be able to improve the experience of your users with *precompilation* +so that when they use the package, the code they use is already compiled. To precompile package code effectively, it's recommended to use [`PrecompileTools.jl`](https://julialang.github.io/PrecompileTools.jl/stable/) to run a "precompile workload" during precompilation time that is representative of typical package usage, which will cache the native compiled code into the package `pkgimage` cache, greatly reducing "time to first execution" (often referred to as @@ -1431,10 +1431,7 @@ may be the case during development of a package. Keeping the time taken to load the package down is usually helpful. General good practice for package developers includes: -1. Reduce your dependencies to those you really need. -2. Consider organizing any extended package functionality that is dependent on particular dependencies into - [package extensions](@ref) that can serve optional functionality depending on whether the user's environment (or the - package depending on yours) has those particular dependencies. +1. Reduce your dependencies to those you really need. Consider using [package extensions](@ref) to support interoperability with other packages without bloating your essential dependencies. 3. Avoid use of [`__init__()`](@ref) functions unless there is no alternative, especially those which might trigger a lot of compilation, or just take a long time to execute. 4. Where possible, fix [invalidations](@ref) among your dependencies and from your package code. From d08f800ae9f83df05b6dd1410f41068c2ef66207 Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Fri, 13 Sep 2024 13:02:02 -0400 Subject: [PATCH 4/5] fix broken links --- doc/src/manual/performance-tips.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/src/manual/performance-tips.md b/doc/src/manual/performance-tips.md index 8af693b86185c..826cdde4d6bba 100644 --- a/doc/src/manual/performance-tips.md +++ b/doc/src/manual/performance-tips.md @@ -1412,7 +1412,7 @@ julia> @time @eval foo(); 0.000156 seconds (63 allocations: 2.453 KiB) ``` -Note that `@time @eval` is better for measuring compilation time because without [@eval](@ref), some compilation may +Note that `@time @eval` is better for measuring compilation time because without [`@eval`](@ref), some compilation may already be done before timing starts. When developing a package, you may be able to improve the experience of your users with *precompilation* @@ -1434,7 +1434,7 @@ General good practice for package developers includes: 1. Reduce your dependencies to those you really need. Consider using [package extensions](@ref) to support interoperability with other packages without bloating your essential dependencies. 3. Avoid use of [`__init__()`](@ref) functions unless there is no alternative, especially those which might trigger a lot of compilation, or just take a long time to execute. -4. Where possible, fix [invalidations](@ref) among your dependencies and from your package code. +4. Where possible, fix invalidations among your dependencies and from your package code. The tool [`@time_imports`](@ref) can be useful in the REPL to review the above factors. From 5ba4edfb6ced5f6e6254eb62e551c1e13056c996 Mon Sep 17 00:00:00 2001 From: Ian Butterworth Date: Fri, 13 Sep 2024 15:36:28 -0400 Subject: [PATCH 5/5] link to invalidations blog post --- doc/src/manual/performance-tips.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/manual/performance-tips.md b/doc/src/manual/performance-tips.md index 826cdde4d6bba..436d58f54754a 100644 --- a/doc/src/manual/performance-tips.md +++ b/doc/src/manual/performance-tips.md @@ -1434,7 +1434,7 @@ General good practice for package developers includes: 1. Reduce your dependencies to those you really need. Consider using [package extensions](@ref) to support interoperability with other packages without bloating your essential dependencies. 3. Avoid use of [`__init__()`](@ref) functions unless there is no alternative, especially those which might trigger a lot of compilation, or just take a long time to execute. -4. Where possible, fix invalidations among your dependencies and from your package code. +4. Where possible, fix [invalidations](https://julialang.org/blog/2020/08/invalidations/) among your dependencies and from your package code. The tool [`@time_imports`](@ref) can be useful in the REPL to review the above factors.