Episode reward does not include gamma about keras-rl HOT 2 CLOSED

Hi, the episode_reward is just used for reporting. The actual algorithms use properly discounted rewards to estimate the Q values, for example (compare https://github.com/matthiasplappert/keras-rl/blob/master/rl/agents/dqn.py#L252).

Getting RL algorithms to work can often be quite tricky. Starting points are:

• Scale your inputs (i.e. the observations) such that they are all on the same scale and, preferably, have a mean of 0 and a variance of 1.
• Experiment with different scales for your reward function. This can be especially problematic if your rewards are orders of magnitudes different throughout the game (i.e. very low rewards first and very large rewards in later stages of the env).

I hope that helps.

from keras-rl.

Hi, thank you very much for your suggestion.
My environment has linar constraints for the state-action space of the form Ax<=b, so that I have included a penalty function that squares the output if the constraints are violated. Hence, the rewards could be high at some point. But as you said, this may be an issue, so I am going to try another thing: I will consider any violated constraint as a terminal state with some fixed negative reward.
Also, the state space is [0, 5]*ones(2,1) and the action space is [0, 0.5]*ones(4,1). I will try to shift and scale them.

from keras-rl.