i run the code but get the error of :
raise NoSuchDisplayException('Cannot connect to "%s"' % name) pyglet.canvas.xlib.NoSuchDisplayException: Cannot connect to "None" [2017-03-07 11:41:49,394] Finished writing results. You can upload them to the scoreboard via gym.upload('/home/sushuting/workspace/openai-cartpole/cartpole-hill')

heres a cleaner way to calculate discounted rewards:

np.array([sum([gamma**t*r for t, r in enumerate(rewards[i:])]) for i in range(len(rewards))])

Hi Kevin,

Very nice tutorial and clean code. One very small nit to do with as you please: in the cartpole-policygradient.py at the bottom you use a variable called reward to store the cumulative return for an episode. In the run_episode you use the more correct variable name totalreward. I suggest changing reward to return for the sake of accuracy. Corresponding issues exist for the random policy and hill-climbing. This is a minor issue though. Great job!

Cheers,
Zack