-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multidimensional arrays #399
Comments
How about making the input an abstract type? At the end of the day the state of the optimisation just needs to be an element of a hilbert space |
Also see @ChrisRackauckas comments in #326. I really see no barrier to admitting a general abstract type as the state of optimisation - only advantages. |
How abstract? |
Not even an AbstractArray. You actually need elements of a Hilbert space over some type This is really not a necessity, BUT: it might actually make the code more readable, and it could have some unexpected applications. E.g., if I minimise over a space of functions, then I don't have to represent functions in subsequent optimisation steps w.r.t. the same basis. |
Yes, ApproxFun would be a very interesting application which takes the generality to the max here.
From BFGS, it looks like you need |
everything in |
Mathematically yes. But in reality, what I was trying to point out is that you're missing that either indexing or broadcast needs to be defined. Right now that's using indexing, hence an |
and in addition CG of L-BFGS or a hessian-free TR or the various accelerated steepest descent methods really just need the small set I mentioned. |
Okay, I'm sold. If we need to do something that sacrifices performance for arrays, we can just have tightly and loosely typed methods |
is there a performance benchmark suite? I'd be quite interested to do a prototype (say, CG or LBFGS) and confirm that the strict typing we use at the moment is not needed at all. |
Not tagged, but the code can be found at OptimTests.jl ..I should really get that fixed up and merged (will do after Optim is tagged) |
I would be heavily surprised if, when you did this, the machine code changed one bit. It would only change if you change the indexing to broadcasting, in which case it's already pretty well documented what the (current) cost is there. |
I agree, but there is lots of stuff like this in the code:
which I think is appalling, so this will also be a good opportunity to get rid of it. |
Agree, and I the time to calculate |
Stumbled upon the following (is this related?) when trying to optimize a sum over multidimensional Rosenbrock function f(x) = sum((1.0 .- x[1]).^2 .+ 100.0 .* (x[2] .- x[1].^2).^2) I get the following error optimize(f, [[12.0,1.0],[1.0,2.0]])
ERROR: MethodError: no method matching zero(::Type{Array{Float64,1}})
Closest candidates are:
zero(::Type{Base.LibGit2.Oid}) at libgit2\oid.jl:88
zero(::Type{Base.Pkg.Resolve.VersionWeights.VWPreBuildItem}) at pkg/resolve/versionweight.jl:80
zero(::Type{Base.Pkg.Resolve.VersionWeights.VWPreBuild}) at pkg/resolve/versionweight.jl:120
...
in initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Void}, ::#f, ::Array{Array{Float64,1},1}) at C:\Users\s.sahm\.julia\v0.5\Optim\src\nelder_mead.jl:132
in optimize(::Function, ::Array{Array{Float64,1},1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Void}) at C:\Users\s.sahm\.julia\v0.5\Optim\src\optimize.jl:172
in #optimize#74(::Array{Any,1}, ::Function, ::#f, ::Array{Array{Float64,1},1}) at C:\Users\s.sahm\.julia\v0.5\Optim\src\optimize.jl:24
in optimize(::#f, ::Array{Array{Float64,1},1}) at C:\Users\s.sahm\.julia\v0.5\Optim\src\optimize.jl:23 My final goal is to regard the subcomponents thanks a lot |
I don't quite get what you're trying to do. For example, do you realize that your provided point cannot be evaluated by julia> f( [[12.0,1.0],[1.0,2.0]])
ERROR: DimensionMismatch("Cannot multiply two vectors")
Stacktrace:
[1] power_by_squaring(::Array{Float64,1}, ::Int64) at ./intfuncs.jl:166
[2] f(::Array{Array{Float64,1},1}) at ./REPL[161]:1 |
Supporting arrays of arrays is a whole different ballgame. You need to make sure most internal calls will automatically be recursive. The repo https://github.com/JuliaDiffEq/RecursiveArrayTools.jl is the tools that are used in DiffEq to support this kind of stuff, but there's a substantial amount of "being careful with types" that has to be done to get recursive arrays working. |
You also need to make sure you agree what it's supposed to mean... I mean, in the example above I'm a bit confused. |
@pkofod I am sorry for the confusing, it should be I would like to have an argument of type It would be really helpful if I can have something like multiple parameters (i.e. arguments) to my optimizable function. Is this still this issue, or rather something else? Looking forward to your thoughts. |
If each array in the "outer" container is of the same size (as in this example), you almost surely want to use StaticArrays and make it an |
I think it is this issue, yes. |
CC @ettersi |
Does @ettersi have a use case here? |
I see I never answered about the recursive arrays. Let me be a little more clear about that. If you do something like I'm not sure if the full mutability makes too much sense for an optimization problem, diffeq does it because things "change over time" but I'm not sure that has much meaning here. But it would be something nice to handle correctly down the road. As for other array types, things like One thing that is missing from the last point though is that I can start over the weekend helping get this all setup. Maybe if you guys are also setup with |
My use-case is optimising an (approximate) lattice of atoms, which I store as a
I get the following error:
I haven't really dug into the details of this error yet, though. It could well be that there is some subtle issue in my code, or that it's an issue that I am still on 0.5. |
Being on v05 will at least mean that you're not getting the latest updates, but supporting static arrray types are high on my priority list |
Vectors of Static Arrays will have other issues though, since indexing gets you one static array each time, so defining a Jacobian and factorizing it won't work very well making it work with some algorithms but not too many. You'll want to wrap the Vector{SArray} in something like RecursiveArrayTools' |
Solved the problem using |
You don't necessarily have to do it yourself. A precise description of what you want goes a long way :) |
Fixed the original issue, if anything we can do comes up we can open a new issue. |
Fix up (L-)BFGS NelderMead, and PSO code such that we can pass multidimensional arrays as input.
The text was updated successfully, but these errors were encountered: