RuntimeError: symeig_cuda: the algorithm failed to converge about online-3d-bpp-drl HOT 5 CLOSED

The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:

1. Increase the value of the mask (in config.py) during training.
2. Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error.
3. This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.

from online-3d-bpp-drl.

Thank you I will try with these parameters.

from online-3d-bpp-drl.

The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:

1. Increase the value of the mask (in config.py) during training.
2. Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error.
3. This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.

Hi @alexfrom0815
If I use a2c instead of ACKTR will there be any performance downside? Or its ok to use it

from online-3d-bpp-drl.

The reason for this error is that the ACKTR implementation we borrowed from https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail is not robust enough. Three suggestions to solve this problem:

1. Increase the value of the mask (in config.py) during training.
2. Modify the hyperparameters in the algorithm so that the training algorithm can avoid this convergence error.
3. This convergence problem only appears in the implementation of ACKTR, you can try to change the training algorithm to a2c or other reinforcement learning algorithms.

Hi @alexfrom0815 If I use a2c instead of ACKTR will there be any performance downside? Or its ok to use it

Unfortunately, the a2c algorithm is indeed much worse than ACKTR. We are currently looking for better alternatives to the ACKTR algorithm. We encourage you to try some other algorithmic hyperparameters to avoid this ACKTR error (random seed, mask value, etc.).

Another tip is that although we cannot determine the cause of this error, from experience, when the error appears, the performance of the algorithm is generally close to the highest value, so its existence may not affect the final effect of the algorithm in a certain sense.

In addition, we have completed the implementation of a new online BPP algorithm, which is more stable than the current algorithm and can work at arbitrary resolution. Due to the double-blind protocol of the paper review, we will publish the new algorithm code after our new work is accepted.

from online-3d-bpp-drl.