More details on IPNewton implementation

[raphaelchinchilla]

I have recently started using Optim.jl.

I have a complicated control optimization problem to solve, and I would need some more details on which implementation of primal dual method is used. I have tried to read the code in more details, but I could not understand it.

In Optim's online documentation it is referenced chapter 19 of Nocedal & Wright. It is however not clear whether equation 19.15 (with the equality constraints in the Hessian) or equation 19.16 (the condensed version of it obtained by eliminating the direction).

Judging by the response to #813, it seems that equality constraints are not directly implemented, which essentially means that none of the above is the implementation. It also does not seem to be the implementation of Ipopt as described in Wachter & Biegler as inequalities are defined through slack constraints and equalities.

In this case, if equality constraints are not implemented, is there a mechanism to "catch" equality constraints? I only have equality constraints in my problem, but a lot of them, and it seems that defining them as inequality constraints essentially makes the hessian much bigger than it should be.

PS: I have also read through #303 but it is almost 4 years old, it is not clear whether changes have been made or not

[pkofod]

I am going to dive into the implementation soon to try to re-implement it in NLSolvers and prepare for a future version of Optim. When I'm in the middle of that, I will try to write up some documentation. I'm sorry, but it won't make sense for me to write up a good description before that. So, I will have to come back to you on all this. The truth is that the implementation is a mix of different things. So no, it's not exactly chapter 19 and it's not Ipopt either.

[raphaelchinchilla]

No worries, I ended up figuring out many parts of it in the last two months and my application ran very well :).

I don't know what exactly what you mean by "re-implement it in NLSolvers" but if you ever decide to make a separate package, I would like to suggest to make it easier for other people to PR different interior point methods solvers. I have parts of one written and would love to PR it to an existing project one day instead of creating a new package.

[luluywang]

@raphaelchinchilla for future researchers, can you share a bit more about how you resolved the issues with equality constraints?

[raphaelchinchilla]

@luluywang I have not really used Optim.jl lately, so I am a bit rusty. From what I remember after my comment on Sep 5 2020, when one sets an equality constraint with lower_bound==upper_bound, Optim.jl converts it into an equality constraint, so there is no "two constraints at the same time".

I do have to say that it is a bit clunky to have to set equality constraints as inequality constraints with upper and lower bound, it is not really that user friendly. I know that GalacticOptim.jl has a similar issue that they have not solved yet.