https://github.com/cyoon1729/Reinforcement-learning/blob/d157d0d86c37734be4b430b7d311eb9bb0379d93/Policy-Gradient-Methods/a2c/a2c.py#L42

Sir, can you please explain why value_targets = rewards + discounted_rewards ?
Why do you need to add rewards and discounted rewards together ?

This is a great job :)

I am confused about sac2018.py implementation，Why are there two q_net in the sac2018.py file? (code link)
I did not see this description in the original paper Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

In addition, I want to verify one thing. If I understand correctly, sac2018.py is the implementation of Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor, and sac2019.py is the implementation of Soft Actor-Critic Algorithms and Applications. Looking forward to your reply.

https://github.com/cyoon1729/Reinforcement-learning/blob/d157d0d86c37734be4b430b7d311eb9bb0379d93/Policy-Gradient-Methods/a2c/a2c_test.py#L25

No need to save next_state, you never use it.

Copy critic target parameters

    for target_param, param in zip(self.critic_target.parameters(), self.critic.parameters()):
        target_param.data.copy_(param.data)

Actor need to sync?

DQN-based algorithms have memory leak while training on Atari. The leak doesn't always happen (inconsistent), or happens suddenly in the middle of training.

Hi,
Do you can add some documentation to your code? Or a HowTo? I was running your code but dont see that you save the learned model right? Because i would like to test it.