Check the definition of knew after a trust-region step about cobyqa HOT 3 CLOSED

A carefully chosen weight can boost the performance

Update 20230131: The performance of UOBYQA and COBYLA is improved substantially by choosing this weight carefully.

This is not strange, because the definition of KNEW after a TR step is 1/4 of the algorithm. A DFO TR algorithm is completely defined by four components:

1. TR step
2. KNEW after a TR step
3. KNEW before a geometry step,
4. geometry step.

I firmly believe this will be the case for COBYQA as well. Try it, please.

from cobyqa.

Update 20230502: I realize very recently that it is WRONG to take np.square(dsq) (or any power of it) as the weight for np.abs(sigma) when defining knew. In other words, the following line is WRONG:

Remember to talk to me about this.

from cobyqa.

Update 20230503: see the comments at

https://github.com/libprima/prima/blob/3684cc52e7055a6b793156cefe19c1169073ea90/fortran/lincoa/geometry.f90#L126-L152

from cobyqa.