About MCTSnet about dream-go HOT 2 CLOSED

Hi, I have not had much success with MCTSnet. I have a half implementation [1] but I have encountered numerous problems that made me pause the implementation for the moment:

• Unsatisfying performance. This is partially an implementation detail, but due to the limited batch size I do not get good GPU utilization which limits its training.
• The accuracy of the MCTSnet policy did not converge to a better solution than a traditional policy network.
• The networks of the original article did not work, they did not converge at all for non-trivial data. I had to change all of them to GRU cells which is unsatisfying.

I have not taken the project further than the training step, so I do not know how it would affect the actual playing strength.

[1] https://github.com/Chicoryn/dream-go/tree/dev/mctsnet/contrib/mctsnet

from dream-go.

Thank you for your reply, we have encountered similar problems with you. Our GPU utilization is also not high.
Although Deepmind's ideas are interesting, they don't give any reference code. We were surprised to see that they used MCTSnet to perform a simulation training 3 million times in a Sokoban environment to achieve a 60% success rate.

from dream-go.