Hey there,

I just wanted to inform you that if a user utilizes your code from C++ the compiler won't find anything of the boundary_transformation stuff, because the extern "C" call is missing in the header file.

Thanks for your code and best regards
Max

Hi! Congratulations on your CMA-ES method! I myself am developing an optimization method, and I'm using your method for comparison purposes. Your method is very competitive, I can't reach its convergence speed yet in dimensions above 10.

My benchmarking suite uses random offsetting and rotation in the same manner as BBOB benchmark suite.

I'm having problems with CMA-ES optimizing such function in 14 dimensions, x[] is in the range -25 to 25. CMA-ES simply crashes the program without providing any output. I have a diverse set of functions, and basically only this function causes issues. Note that a random offsetting and rotation should be applied to this function.

static double calcZeroSum( const double* const x, const int N )
{
    double s = 0.0;
    int i;
    for( i = 0; i < N; i++ )
    {
        s += x[i];
    }
    return( s == 0.0 ? 0.0 : 1.0+pow(10000.0*fabs(s),0.5) );
}

The member gen of struct cmaes_t is described as "generation number" but its type is defined here as double. Then it is assigned to a long here which causes the following compiler warning in VC++:

cmaes.c(742): warning C4244: 'initializing' : conversion from 'double' to 'long', possible loss of data

Even in 64 bit mode, a Visual C++ long is 32 bits (weird, but allowed by the C standard), so this warning is correct.

Is gen intentionally a double or should it be an integer type? If intentional there should be an explicit cast here to avoid the warning, assuming the generation can be guaranteed to fit in a signed 32 bit integer (i.e., less than about 2 billion). Otherwise a long long type could be used to guarantee a 64 bit integer.

cc/ @nikohansen, @chrisdembia

I see that the parameters are stored in evo->sp. What is the best and safest way to modify these parameters programmatically instead of via input files?

My hunch is that the following sequence of calls would be safe:

cmaes_t evo;
cmaes_init(&evo, ...);
evo.sp.stopMaxIter = 500;
readpara_SupplementDefaults(&evo.sp);

I want to modify the readpara_t entries but I don't want to put the parameters in an inconsistent state, and I still want to benefit from the supplemented defaults. It seems that readpara_t.damps is set based on the value of stopMaxIter. If I update stopMaxIter, I must call readpara_SupplementDefaults to have a consistent value of damps. Are there other additional steps I must take?

Thanks!

Hi, Niko. While incorporating cma-es into Simbody, we noticed some external functions boundary_transformation() etc. that don't have the usual cmaes_ prefix so would potentially cause link-time conflicts for our users. @chrisdembia removed those files from our port for now, but I would like to be able to include the whole package unchanged to facilitate later updating, and in case someone wants to take advantage of those functions later. Please prefix all the extern functions if possible.

Sherm

Hey there,

me again. I used now this library with boundaries quite a bit and I just noticed that there is no option to write out the transformed values natively. I'd like to help out, since I'm profiting from this library, however I'm asking myself, if you would even like to have it as a feature in the library. If so, is there any preferred way of implementing it, e.g. a new pointer for the transformed mean or transformed xbest in cmaes_t?

Best
Max

This is not so much an issue, but a fellow researcher question. In CMA-ES an Eigen decomposition is used. While it provides a good estimation of space directionality of sample distribution, this method only ranks the standard deviations in the sample distribution. Yet, CMA-ES uses t->rgD[i] that are ranked. So, when a Gaussian noise vector is generated, its components do not correspond to standard deviations of the sample distribution along its axes. Maybe this is already handled, but I have not found how.

Also I've spotted that in cmaes_interface.cpp:1939 the t->rgD[i] = sqrt(t->rgD[i]); equation is used. From my personal experience, the problem is, Householder and QLalgo may produce small negative values in t->rgD, so a sqrt(abs(t->rgD[i])) is needed to handle such cases.

We are making good progress incorporating cmaes into simbody; see simbody/simbody#253.

Currently, there are a few extern functions in cmaes.c (e.g., random_*). Can these be named with a cmaes_ prefix, or otherwise be placed under local visiblity?

If you'd like, I could submit a pull request for the change, but @sherm1 and I would like to know what you think.

In my application, I first optimize a function f(x1, ..., xn) which is relatively cheap; then I add a perturbation to that function which is very expensive to compute, so I would like to retain as much information from the first optimization as possible. It is clear how to use the optimum (x1, ..., xn) from the first optimization as a starting point for the second, but I am unsure how best to retain covariance matrix and step size information. There are two related scenarios:

1. The perturbation takes the form g(x1, ..., xn), so the dimension is the same. In this case, is this the right approach?

/* Evaluations use f(x1, ..., xn) up to here. */
cmaes_WriteToFile(evo_ptr, "resume", filename);
cmaes_exit(evo_ptr);
cmaes_init(evo_ptr, dimension, initialX, initialStdDev, seed, lambda, input_parameter_filename);
cmaes_resume_distribution(evo_ptr, filename);
/* Evaluations continue with f(x1, ..., xn) + g(x1, ..., xn). */

2. The dimension of the problem also increases, so the function becomes f(x1, ..., xn) + g(x1, ..., xn, ..., xm). Is there a recommended way to retain information about the n dimensions which have already been studied, while feeding in default values for new dimensions?

Thanks!

Sorry to bother you with my request, but I have developed an alternative optimization method based on CMA-ES concept. I've literally reassembled CMA-ES. The method is called SA-ES (sigma adaptation evolution strategy). It performs vector sigma adaptations directly plus covariance matrix update via simple leaky integrator filtering.

It can be downloaded here: https://github.com/avaneev/biteopt
required files: saesopt.h, biteoptort.h, biternd.h. saesopt.h is a front-end class compatible with my BiteOpt optimizer. biteoptort.h is the actual implementation - it was suitably commented on the update steps performed.

SA-ES is a little bit more effective than CMA-ES in my tests, but not by a huge margin. Maybe you can find some creative ideas in it.

cmaes.c uses the formula (

t->chiN = sqrt((double) N) * (1. – 1./(4.*N) + 1./(21.*N*N));

The number “21” is also used in your tutorial at https://www.lri.fr/~hansen/cmatutorial.pdf. However, I believe that it is not the correct asymptotic expansion of chiN. See http://www.wolframalpha.com/input/?i=series+expansion+for+sqrt%282%29*gamma%28%28n%2B1%29%2F2%29%2Fgamma%28n%2F2%29+at+n%3Dinfinity, which indicates that “21” should be changed to “32”.

Alternatively, C99 standardizes the tgamma function, so if you assume C99 compilers then it could be replaced entirely with

t->chiN = sqrt(2.) * tgamma((double)(N+1)/2)/tgamma((double)N/2);

or, for to avoid overflow with roughly N > 340, the slightly less accurate

t->chiN = sqrt(2.) * exp(lgamma((double)(N+1)/2) – lgamma((double)N/2));

Hello,

I use CMAES and its C implementation to deal with stress inversion.
When my problem is very ill conditionned, I have a floating point exception in cmaes.c at line 893/894

/* cumulation for covariance matrix (pc) using B*D*z~N(0,C) */                                                                                                                                      
 hsig = sqrt(psxps) / sqrt(1. - pow(1.-t->sp.cs, 2*t->gen)) / t->chiN 
  < 1.4 + 2./(N+1); 

its with the "pow" function, t->gen = 752 and 1.-t->sp.cs = 0.62435950746181024, so the result must be really close to 0....

Have you got an idea how to avoid this kind of problens, moreover, it has begin to converge !!

Cheers,

Antoine

Some warnings are unconditionally printed to stdout (e.g. here). We would like to control the output that our library produces. Would it be possible to add a verbosity setting? The setting could default to "on" or "1", and we could override it to be zero by default in our library. What do you think?