The 'configureMlr' can not work with the parallel computing about mlrmbo HOT 4 CLOSED

I found one possible solution is set the level of parallel computing as resample by,

parallelStartSocket(cpus=5,level="mlr.resample").

The default option should be mlrMBO.propose.points, which can result in the above problem.

I also tried other options, which may call multiple cores but can not assign the most computational expensive CV training process to the cores. In this way, the resample level parallel may be the best choice for the CV based hyperparameter tuning.

This is a simple analysis. It is appreciated if someone has any suggestions for implementing mlrMBO.propose.points with the configuremlr(on.learner.error = "warn"). Thanks

from mlrmbo.

The following is not a solution to your problem, but a general hint:

If you want to speed up the optimization through parallelization it is advisable to parallelize the evaluation of the black box (i.e. the train/test-resampling) rather than proposing multiple points because two sequential proposals are generally worth more than tow parallel proposals. Why? Because the second sequential point has been proposed with the knowledge of the first point whereas the second parallel point is generated from the same knowledge as the first.

from mlrmbo.

Yes, Parallizing on the resample level seems good. However, does this means if I have a 5-folds CV, I can only use 5 cores simultaneously? Is there a method to take advantage more?

from mlrmbo.

Closing, because this is not a bug in mlrMBO, but rather a problem in parallelMap and mlr.

from mlrmbo.