Hessian being called separately using only_fgh!() about optim.jl HOT 1 OPEN

Hm, yes I appreciate the comment, and I am aware of the issue. The problem is, that there are many different combinations of what is expensive when called with what etc. I'll think about an option to force it. The problem is that in many cases (not all, thing quadratic or when a model solution is the major computational expense), adding the hessian calculation does indeed bring a computational cost. However, in trust region methods, it is often the case you might reject a step, reduce the radius, and then solve the subproblem again. The hessian does not update here, but you cannot figure out if you are in this case without evaluating the objective. You can create a cache variable that you only update when the parameters change. I think there's a tutorial somewhere. But I'll consider it.

from optim.jl.