Week2 MCTS Speed-Up Question about practical_rl HOT 3 CLOSED

Hi!
(sorry for the delay)
Yes, that's a drawback of vanilla MCTS. You can, however, circumvent it with approximations:

• In games where you can train with discounted rewards you can use partial rollouts. Say, you use the discount factor gamma = 0.95. In this case, running for just 100 steps will accumulate ~99.5% of the infinite discounted returns.
• In any other case, you can train a state value model that estimates V(s). You can then use it to estimate the rest of rollout: just add up rewards for first N steps and then use V(s_{t+N}) for the rest.

Some efficient means to training such models are covered in weeks 3, 4, 6 (for that reason i recommend going through base track of those weeks first and returning to week2_mcts later).

from practical_rl.

Absolutely no worries, I'm sure you are busy! I can't wait for the Coursera class to start so that I don't have to bug you here.

Thanks for the suggestions to backtrack to week2_mcts. I'll try the hint of estimating V(s) and then using that instead of rolling out for infinity and see how that works. Much appreciated!

from practical_rl.

[imho the question was answered, if it wasn't, just reopen]

from practical_rl.