tsani / dm_optimizer Goto Github PK

When fixed-points are encountered, rather than abort, we should force a target adjustment and press on. Perhaps we should abort if the new target is insufficiently different from the current one.

DM appears to produce a number of function evaluations comparable to DifferentialEvolution if their number of iterations are adjusted.

DM makes 20k evaluations over 100 iterations whereas DE makes 20k evaluations over 1000 iterations.

The RandomSearch method seems to not care so much about the number of iterations; past about 15 iterations, it consistently produces the same number of function evalutions in one of my tests. Perhaps the MaxIterations option passed to RandomSearch is simply passed on to its local optimizer; past a certain number of iterations required for convergeance, the local search stops. Perhaps adjusting the RandomSearch's SearchPoints option will produce higher numbers of function evaluations and should be prioritized.

Simulated Annealing also produces a very small number of function evaluations, even after boosting MaxIterations considerably. It turns out that if the iterate doesn't move for a certain number of iterations (50 by default), then the optimization is aborted. Hence, Simulated Annealing can get caught in local minima if its step scaling factor isn't large enough.

NelderMead also produces fewer function evaluations, since it presumably converges and then bails out.

In the case of SimulatedAnnealing and NelderMead we can implement a random-restart process that will repeat itself until some maximum number of function evaluations is reached. For RandomSearch, we can play with ways of adjusting MaxIterations and SearchPoints to get a high enough number of function evaluations, and otherwise use a random-restart process.