Failed to precompile CSV

[mthelm85]

using CSV in a Pluto notebook causes:

Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to C:\Users\mthel\.julia\compiled\v1.5\CSV\HHBkp_tAWrN.ji.

error(::String)@error.jl:33
compilecache(::Base.PkgId, ::String)@loading.jl:1305
_require(::Base.PkgId)@loading.jl:1030
require(::Base.PkgId)@loading.jl:928
require(::Module, ::Symbol)@loading.jl:923
top-level scope@Local: 1

REPL output:

      From worker 2:    ERROR: LoadError: UndefVarError: invrefpool not defined
      From worker 2:    Stacktrace:
      From worker 2:     [1] getproperty(::Module, ::Symbol) at .\Base.jl:26
      From worker 2:     [2] top-level scope at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162  
      From worker 2:     [3] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [4] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [5] top-level scope at none:2
      From worker 2:     [6] eval at .\boot.jl:331 [inlined]
      From worker 2:     [7] eval(::Expr) at .\client.jl:467
      From worker 2:     [8] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\PooledArrays\4I2sD\src\PooledArrays.jl:162
      From worker 2:    ERROR: LoadError: Failed to precompile PooledArrays [2dfb63ee-cc39-5dd5-95bd-886bf059d720] to C:\Users\mthel\.julia\compiled\v1.5\PooledArrays\vi11X_tAWrN.ji.
      From worker 2:    Stacktrace:
      From worker 2:     [1] error(::String) at .\error.jl:33
      From worker 2:     [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1305
      From worker 2:     [3] _require(::Base.PkgId) at .\loading.jl:1030
      From worker 2:     [4] require(::Base.PkgId) at .\loading.jl:928
      From worker 2:     [5] require(::Module, ::Symbol) at .\loading.jl:923
      From worker 2:     [6] include(::Function, ::Module, ::String) at .\Base.jl:380
      From worker 2:     [7] include(::Module, ::String) at .\Base.jl:368
      From worker 2:     [8] top-level scope at none:2
      From worker 2:     [9] eval at .\boot.jl:331 [inlined]
      From worker 2:     [10] eval(::Expr) at .\client.jl:467
      From worker 2:     [11] top-level scope at .\none:3
      From worker 2:    in expression starting at C:\Users\mthel\.julia\packages\CSV\YUbbG\src\CSV.jl:15

Out put of versioninfo():

Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 9 3950X 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, znver2)
Environment:
  JULIA_EDITOR = "C:\Users\mthel\AppData\Local\Programs\Microsoft VS Code\Code.exe"
  JULIA_NUM_THREADS = 32

CSV version is [336ed68f] CSV v0.8.3 and Pluto is [c3e4b0f8] Pluto v0.12.20.

Other users are experiencing this error as well: https://discourse.julialang.org/t/failed-to-precompile-csv-in-pluto/54633/4

[mthelm85]

A Discourse user @thatcher was able to get everything working by using the following package versions:

(@v1.5) pkg> rm CSV
(@v1.5) pkg> rm DataFrames
(@v1.5) pkg> rm PooledArrays
(@v1.5) pkg>  rm CategoricalArrays
(@v1.5) pkg>  add [email protected]
(@v1.5) pkg>  add [email protected]
(@v1.5) pkg>  add [email protected]
(@v1.5) pkg>  add [email protected]

[RobvanW]

Just to confirm from my side:

Funny thing is CSV (0.8.3), DataFramesMeta (0.60) and DataFrames (0.18.3) load & precompile nicely when run from the REPL include("pluto_notebook_file.jl") but all fail to precompile in the Pluto notebook environment.

Since it doesn't happen to any other packages, well at the moment at least, I wonder if the bug is in Pluto or in these packages (and other packages might follow suit).

[j-fu]

See the discussion under JuliaData/CategoricalArrays.jl#325 :

It seems that it is somehow linked to the way Pkg and Pluto interact.

Somehow under Pluto, an older version 1.4.x of DataAPI.jl was drawn in, though both DataFrames.jl (via PooledArrays.jl) and CategoricalArrays.jl have requirements for 1.5 in their Project.toml .

I was able to fix my situation by removing .julia/packages and .julia/environments, and re-adding/re-instantiating.

[bkamins]

Somehow under Pluto, an older version 1.4.x of DataAPI.jl was drawn in, though both DataFrames.jl (via PooledArrays.jl) and CategoricalArrays.jl have requirements for 1.5 in their Project.toml .

It might be a problem on package server side - hopefully now it is resolved (see https://discourse.julialang.org/t/pkg-protocol-delay-in-updating-generalregistry/54521 for a discussion of similar problems).

[fonsp]

A possible cause is that Pluto loads Tables.jl on the notebook process before running your code, so the loaded version of Tables could be different from the one resolved by Pkg.

Can you use this inside the notebook to check which version of Tables was loaded? It should be >=1.1

[juliohm]

I am having the same issue on Manjaro Linux. The same error with PooledArrays.jl inside a Pluto session.

[fonsp]

It was a problem on the Pluto side (#895 (comment) ), and I fixed it on master 🌈

We will backport to a patch release for 0.12 soon

[filchristou]

I get the same problem for the ReinforcementLearning package. (works with REPL)
I am using Pluto v0.14.8.

[filchristou]

Okey. After deleting, removing, removing, installing, removing deleting, installing it worked. sorry for the hasty comment but again it looks like something is not working 100% .. Also I observed that the compilation time is much more that REPL's. idk if this is normal.

[adomasbaliuka]

I have the same error as OP again, on Windows. The problem is NOT solved by deleting the .julia folder or reinstalling everything. The CSV package works fine outside of Pluto.

Failed to precompile CSV [336ed68f-0bac-5ca0-87d4-7b16caf5d00b] to C:\Users\MyName\.julia\compiled\v1.7\CSV\jl_BB08.tmp.

error(::String)@error.jl:33
compilecache(::Base.PkgId, ::String, ::IO, ::IO, ::Bool)@loading.jl:1466
compilecache(::Base.PkgId, ::String)@loading.jl:1410
_require(::Base.PkgId)@loading.jl:1120
require(::Base.PkgId)@loading.jl:1013
require(::Module, ::Symbol)@loading.jl:997
top-level scope@[Local: 9](http://localhost:1234/edit?id=c5df97f0-c49c-11ec-23db-83e8fcd277e9#)

Further stacktrace info and versions of stuff here: error_details.txt.

[fonsp]

@adomasbaliuka Can you share a notebook file that does not work?

[adomasbaliuka]

Thanks for the quick response and sorry, it seems I didn't test this carefully enough. The problem is with my notebook reproduce_CSV_compile_bug.jl.txt, though I don't know what the problem is exactly. Maybe some package versions contained in the internal manifest are inconsistent.

If I make a new notebook, even with the same list of packages to use, it actually works fine.

Is there a way to "order a given Pluto notebook to regenerate its internal manifest" or something?

[fonsp]

Hey @adomasbaliuka ! In that case, it looks like your notebook was still using a broken version of CSV.jl.

You can update package by clicking on the checkmark next to a package name, and then the update button. More info here: https://github.com/fonsp/Pluto.jl/wiki/%F0%9F%8E%81-Package-management