Is there a return code for the result of the derivative checker?

[ferrolho]

I would like to programmatically check the outcome of the derivative checker. Is that possible?

This could be useful e.g. within an integration test to cover for any breaking changes to the sparsity or user-supplied derivatives. But if this is not possible, I can think of this workaround that would be able to save the stdout for the solving of a sample problem and then simply check if it includes an occurrence matching "No errors detected by derivative checker.".

[odow]

There is no current support for this at the MOI level. I'm not sure if anything exists in the C API, you'd need to check the code in https://github.com/coin-or/Ipopt.

Redirecting stdout is probably the simplest approach.

[ferrolho]

Yup, that's what I ended up doing. Thanks!

Utility method:

"""
    capture_stdout(f::Function) -> String

Capture the standard output (stdout) of the given function `f`.

This function runs the provided function `f`, redirects its output to a temporary file,
and then reads the contents of that file to return the captured output as a string.
The temporary file is deleted after use.
"""
function capture_stdout(f::Function)::String
    # Generate a temporary file to capture stdout
    temp_file = tempname()

    # Redirect stdout to the temporary file while running the function
    open(temp_file, "w") do file
        redirect_stdout(file) do
            f()
        end
    end

    # Read the captured output from the temporary file
    captured_output = read(temp_file, String)

    # Ensure the temporary file is deleted after use
    rm(temp_file)

    return captured_output
end

Unit tests:

@testset "capture_stdout tests" begin

    # Test 1: Basic test to capture simple output
    function test_function()
        println("Hello, world!")
    end
    output = capture_stdout(test_function)
    @test output == "Hello, world!\n"

    # Test 2: Capture multiline output
    function multiline_function()
        println("Line 1")
        println("Line 2")
        println("Line 3")
    end
    output = capture_stdout(multiline_function)
    @test output == "Line 1\nLine 2\nLine 3\n"

    # Test 3: Capture output with no content
    function empty_function()
        # No output
    end
    output = capture_stdout(empty_function)
    @test output == ""

end

Example usage for Ipopt derivative checker:

@testset "Use Ipopt's derivative checker to ensure we got the derivatives right" begin
    # For details about Ipopt's derivative checker, please see:
    # https://coin-or.github.io/Ipopt/SPECIALS.html#DERIVCHECK

    problem = setup_small_problem(num_knots=5, dt=0.1)

    ipopt_options_for_derivative_checker = Dict(
        "hessian_approximation" => "limited-memory",
        "derivative_test" => "first-order",      # none (default), first-order, second-order, only-second-order
        "derivative_test_perturbation" => 1e-6,  # 1e-8 (default)
        "point_perturbation_radius" => 1.0,      # 10 (default)
        "derivative_test_tol" => 5e-3,           # 1e-4 (default)
    )

    logs = capture_stdout() do
        solve_optimisation_problem_with_ipopt(problem, ipopt_options_for_derivative_checker)
    end

    @test occursin("Starting derivative checker for first derivatives.", logs)
    @test occursin("No errors detected by derivative checker.", logs)
end

Not the best to write to a file just to get its content as a string, but redirect_stdout does not take an IOBuffer and the alternative with IOStream would probably end up quite extensive, so I settled on the temporary file for the code simplicity.