ERROR: ArgumentError: NULL library handle

[hddmaelle]

I use Julia1.6.1, macOS Catalina 10.15.7, Pkg.test("CUTEst") passes all test. Thank you.

`julia> using CUTEst

julia> probname = "BROWNDEN"
"BROWNDEN"

julia> nlp = CUTEstModel(probname)
ERROR: ArgumentError: NULL library handle
Stacktrace:
[1] #dlsym#1
@ ./libdl.jl:54 [inlined]
[2] dlsym(hnd::Ptr{Nothing}, s::Symbol)
@ Base.Libc.Libdl ./libdl.jl:54
[3] (::CUTEst.var"#24#25"{Bool, Bool, Tuple{}, Vector{Int32}, String, String})()
@ CUTEst ~/.julia/packages/CUTEst/ww8Jh/src/CUTEst.jl:215
[4] cd(f::CUTEst.var"#24#25"{Bool, Bool, Tuple{}, Vector{Int32}, String, String}, dir::String)
@ Base.Filesystem ./file.jl:106
[5] CUTEstModel(::String; decode::Bool, verbose::Bool, efirst::Bool, lfirst::Bool, lvfirst::Bool)
@ CUTEst ~/.julia/packages/CUTEst/ww8Jh/src/CUTEst.jl:205
[6] CUTEstModel(::String)
@ CUTEst ~/.julia/packages/CUTEst/ww8Jh/src/CUTEst.jl:193
[7] top-level scope
@ none:1

(debug) pkg> status
Status ~/Desktop/julia/debug/Project.toml
[1b53aba6] CUTEst v0.11.0`

[dpo]

Hi @hddmaelle. Thanks for the report. Are you saying that CUTEst.jl passes all tests but you get a null library handle? Are you able to run sifdecoder(probname)?

[hddmaelle]

Hello @dpo
Thank you for your response.
When I run sifdecoder(probname) nothing is displayed: no error nor result.

[dpo]

Hmmm... It works for me out of the box on the same version of Catalina.

Here's what I do:

(@v1.6) pkg> activate test-cutest
  Activating new environment at `/private/tmp/test-cutest/Project.toml`

(test-cutest) pkg> add CUTEst

(test-cutest) pkg> test CUTEst  # tests pass

julia> using CUTEst
┌ Info: using problem repository
└   ENV["MASTSIF"] = "/Users/dpo/.julia/artifacts/a7ea0d0aaf29a39ca0fe75588fc077cdd5b5ed54/optrove-sif-99c5b38e7d03"

julia> sifdecoder("ROSENBR")  # no output

julia> readdir(CUTEst.cutest_problems_path)  # you should see the decoded problem here

julia> "libcutest_double.dylib" in readdir(CUTEst.libpath)
true

Does the above work for you?

[hddmaelle]

Here is what I get when I make the same commands as yours:

julia> using CUTEst
┌ Info: using problem repository
└   ENV["MASTSIF"] = "/Users/maelle/.julia/artifacts/a7ea0d0aaf29a39ca0fe75588fc077cdd5b5ed54/optrove-sif-99c5b38e7d03"

julia> sifdecoder("ROSENBR")

julia> readdir(CUTEst.cutest_problems_path)
138-element Vector{String}:
 "AUTOMAT_AKIVA.SIF.d"
 "AUTOMAT_BEALE.SIF.d"
 "AUTOMAT_BENNETT5LS.SIF.d"
 "AUTOMAT_BIGGS6.SIF.d"
 "AUTOMAT_BOX3.SIF.d"
 ⋮
 "libSNAKE.dylib"
 "libTWOBARS.dylib"
 "libWAYSEA2.dylib"
 "libZECEVIC2.dylib"

julia> "libcutest_double.dylib" in readdir(CUTEst.libpath)
true

[dpo]

Ok. Now let's continue:

julia> sifdecoder("ROSENBR", verbose=true)

 Problem name: ROSENBR

 Double precision version will be formed

 The objective function uses 1 linear group
 The objective function uses 1 nonlinear group

 There are 2 free variables


 File successfully decoded

julia> "libROSENBR.dylib" in readdir(CUTEst.cutest_problems_path)
true

julia> librosenbr = Libdl.dlopen(joinpath(CUTEst.cutest_problems_path, "libROSENBR.dylib"))
Ptr{Nothing} @0x00007fa45849e700

[hddmaelle]

Everything works like you until:
julia> librosenbr = Libdl.dlopen(joinpath(CUTEst.cutest_problems_path, "libROSENBR.dylib")) ERROR: UndefVarError: Libdl not defined Stacktrace: [1] top-level scope @ none:1

[dpo]

Sorry, you have to do using Libdl (it's a standard module).

[hddmaelle]

Sorry for the response time and thanks again for helping me.
I did add the using Libdl and I get the same thing as you after that!
`julia> using Libdl

julia> "libROSENBR.dylib" in readdir(CUTEst.cutest_problems_path)
true

julia> librosenbr = Libdl.dlopen(joinpath(CUTEst.cutest_problems_path, "libROSENBR.dylib"))
Ptr{Nothing} @0x00007f9dba2dc470`

[dpo]

Ok, so you can decode a model with sifdecoder and you can open the resulting shared library by hand. Now we have to figure out why things happen differently when you call CUTEstModel(...). What is the output of

julia> using CUTEst
[ Info: Precompiling CUTEst [1b53aba6-35b6-5f92-a507-53c67d53f819]
┌ Info: using problem repository
└   ENV["MASTSIF"] = "/Users/dpo/.julia/artifacts/a7ea0d0aaf29a39ca0fe75588fc077cdd5b5ed54/optrove-sif-99c5b38e7d03"

julia> sifdecoder("ROSENBR")

julia> CUTEst.cutest_lib
Ptr{Nothing} @0x00007faea3ee5bb0

julia> model = CUTEstModel("ROSENBR", decode=false)
  Problem name: ROSENBR
   All variables: ████████████████████ 2      All constraints: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
            free: ████████████████████ 2                 free: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
         low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0              low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
           fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0                fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
          infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0               infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
            nnzh: (  0.00% sparsity)   3               linear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
                                                    nonlinear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
                                                         nnzj: (------% sparsity)

[hddmaelle]

Here is what I get. Only, I don't have the line corresponding to Precompiling CUTEst, and the last call did not work. Maybe I'm missing something...
`julia> using CUTEst
┌ Info: using problem repository
└ ENV["MASTSIF"] = "/Users/maelle/.julia/artifacts/a7ea0d0aaf29a39ca0fe75588fc077cdd5b5ed54/optrove-sif-99c5b38e7d03"

julia> sifdecoder("ROSENBR")

julia> CUTEst.cutest_lib
Ptr{Nothing} @0x00007f9d5f7e3110

julia> model = CUTEstModel("ROSENBR", decode=false)
ERROR: CUTEst: no decoded problem found
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] (::CUTEst.var"#24#25"{Bool, Bool, Tuple{}, Vector{Int32}, String, String})()
@ CUTEst ~/.julia/packages/CUTEst/ww8Jh/src/CUTEst.jl:207
[3] cd(f::CUTEst.var"#24#25"{Bool, Bool, Tuple{}, Vector{Int32}, String, String}, dir::String)
@ Base.Filesystem ./file.jl:106
[4] CUTEstModel(::String; decode::Bool, verbose::Bool, efirst::Bool, lfirst::Bool, lvfirst::Bool)
@ CUTEst ~/.julia/packages/CUTEst/ww8Jh/src/CUTEst.jl:205
[5] top-level scope
@ REPL[4]:1`

[dpo]

Could you check that you have write permissions to the folder CUTEst.cutest_problems_path?

Also, what does this say:

julia> filter(x -> contains(x, "ROSENBR"), readdir(CUTEst.cutest_problems_path))
5-element Vector{String}:
 "AUTOMAT_ROSENBR.SIF.d"
 "AUTOMAT_ROSENBR.d"
 "OUTSDIF_ROSENBR.SIF.d"
 "OUTSDIF_ROSENBR.d"
 "libROSENBR.dylib"

[hddmaelle]

This is all I have :
`julia> CUTEst.cutest_problems_path
"/Users/maelle/.julia/packages/CUTEst/ww8Jh/src/../deps/files"

julia> filter(x -> contains(x, "ROSENBR"), readdir(CUTEst.cutest_problems_path))
3-element Vector{String}:
"AUTOMAT_ROSENBR.d"
"OUTSDIF_ROSENBR.d"
"libROSENBR.dylib"`

[dpo]

@hddmaelle It turns out that we had made changes that might fix your issue, but did not release a new version. We just issued a new release. Could you update your version of CUTEst?

[hddmaelle]

So I updated the CUTEst package and it seems that my problem is indeed fixed:
`julia> using CUTEst
┌ Info: using problem repository
└ ENV["MASTSIF"] = "/Users/maelle/.julia/artifacts/a7ea0d0aaf29a39ca0fe75588fc077cdd5b5ed54/optrove-sif-99c5b38e7d03"

julia> probname = "BROWNDEN"
"BROWNDEN"

julia> nlp = CUTEstModel(probname)
Problem name: BROWNDEN
All variables: ████████████████████ 4 All constraints: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
free: ████████████████████ 4 free: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 lower: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 upper: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 low/upp: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 fixed: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0 infeas: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
nnzh: ( 0.00% sparsity) 10 linear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
nonlinear: ⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅ 0
nnzj: (------% sparsity)`

Thanks a lot for your help !!

[dpo]

Wonderful! Thanks for your patience.