How to get latest minimizer (u) instead of the best one for optimization? about optimization.jl HOT 5 OPEN

I really like the quality of the documentation,

I am genuinely surprised 🤣

You can use a callback to store the us and pass it to the solve call -

ustore = []
function my_storing_callback(state, l)
    push!(ustore, state.u)
    return false
end

from optimization.jl.

I am genuinely surprised 🤣

Yeah, I actually found the things that I'd been looking for, so 👍 (except the one above)

Thank you! Ok, this is actually the way I am doing it right now... maybe this is something for a feature request (e.g. adding u_latest to the result struct) or does this feel like an exotic requirement?

from optimization.jl.

Since some solvers already store this in their results I haven't seen a reason to do this and increase the memory usage even more for the results (and you as the user can do it with the callback)

from optimization.jl.

If some solvers already store this, then you just use the same reference and there's no extra memory cost?

from optimization.jl.

I have another example where this is very common: Stochastic batching when optimizing ML models... without the option to pick the latest instead of the best minimizer, the optimization will return the minimizer of the best batch element loss - which might be a completely outdated parameter set ...

Also: The callback gets called one last time for the best element after optimization finished, so with the code above:

ustore = []
function my_storing_callback(state, l)
    push!(ustore, copy(state.u)) # copy required because inplace modification of `state.u`
    return false
end
ustore[end] # the best value
ustore[end-1] # actually the latest value during optimization

I didn't check the implementation, but wouldn't it be possible to let a keyword decide if either the state best or latest gets stored without additional memory required?

Thanks!

from optimization.jl.