Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show more info on why package precompilation was needed #52619

Merged

Conversation

IanButterworth
Copy link
Member

@IanButterworth IanButterworth commented Dec 23, 2023

Expands the precompiling log from e.g.

[ Info: Precompiling JET [c3a54625-cd67-489e-a8e7-0a5a0ff4e31b] 

to

[ Info: Precompiling JET [c3a54625-cd67-489e-a8e7-0a5a0ff4e31b] (cache misses: invalid header (10), wrong dep loaded (2), wrong julia version (6))

Hitting code load precompilation can happen for various unexpected reasons, and while JULIA_DEBUG=loading has nice verbose info to diagnose the issue, the issue is usually resolved by the precompilation job and so is hard to reproduce with debug on.

This PR is a rough idea of how we could expand on the standard messaging to give a little more insight without being overly verbose, but catch the one-off issues. I think including a bit more info in this log is ok because it's now a fall-back for when parallel precompilation didn't work.

Using the same example from #44329 (comment) which demonstrates issue #50070 (cc @topolarity)

% ./julia                    
┌ Info: Precompiling Revise [295af30f-e4ad-537b-8983-00126c2a3abe] (cache misses: wrong julia version (1))
└ @ Base loading.jl:2634

(@v1.11) pkg> activate --temp
  Activating new project at `/var/folders/1z/jf841bdj73bdj3vk7kc7f_3w0000gn/T/jl_pPDv9H`

(jl_pPDv9H) pkg> add JET [email protected]
...
...
Precompiling project...
  ✓ JuliaInterpreter
  6 dependencies successfully precompiled in 28 seconds. 32 already precompiled.
  1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version

julia> using JET
[ Info: Precompiling JET [c3a54625-cd67-489e-a8e7-0a5a0ff4e31b] (cache misses: invalid header (10), wrong dep loaded (2), wrong julia version (6))

julia> 

@IanButterworth IanButterworth force-pushed the ib/precompile_show_reasons branch from 13698d9 to 66b6a2b Compare December 23, 2023 23:36
@IanButterworth IanButterworth changed the title RFC: show more info on why package precompilation was needed Show more info on why package precompilation was needed Dec 28, 2023
@IanButterworth IanButterworth marked this pull request as ready for review December 28, 2023 07:49
@IanButterworth IanButterworth merged commit 26d0460 into JuliaLang:master Dec 28, 2023
9 checks passed
@IanButterworth IanButterworth deleted the ib/precompile_show_reasons branch December 28, 2023 07:52
jishnub pushed a commit that referenced this pull request Oct 19, 2024
Falling back to the older serial precompilation process is basically a
bug (except for if a manifest hasn't been resolved) so
#52619 added more info on why
it's been hit so we have a chance of fixing issues that are otherwise
very difficult to recreate.

However "invalid header" which usually just means it was made by a
different julia version appears to sound too alarming to users.
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445

So soften it there and in error messages, given it seems a better
description.

Suggested by @giordano in
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4?u=ianshmean
giordano pushed a commit that referenced this pull request Oct 19, 2024
Falling back to the older serial precompilation process is basically a
bug (except for if a manifest hasn't been resolved) so
#52619 added more info on why
it's been hit so we have a chance of fixing issues that are otherwise
very difficult to recreate.

However "invalid header" which usually just means it was made by a
different julia version appears to sound too alarming to users.
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445

So soften it there and in error messages, given it seems a better
description.

Suggested by @giordano in
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4?u=ianshmean

(cherry picked from commit aa51abe)
KristofferC pushed a commit that referenced this pull request Oct 21, 2024
Falling back to the older serial precompilation process is basically a
bug (except for if a manifest hasn't been resolved) so
#52619 added more info on why
it's been hit so we have a chance of fixing issues that are otherwise
very difficult to recreate.

However "invalid header" which usually just means it was made by a
different julia version appears to sound too alarming to users.
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445

So soften it there and in error messages, given it seems a better
description.

Suggested by @giordano in
https://discourse.julialang.org/t/cache-misses-when-using-packages-since-upgrading-to-1-11/121445/4?u=ianshmean
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant