replay data memory usage? about dreamer-torch HOT 5 CLOSED

Hi tominku,

Thank you for your interesting on my codes!

As I know, Dreamer stores the reply buffer on the disk not memory, but did you experience the memory leak on training? Memory leak means increasement of the memory usage while training.

Bests, Jaesik.

from dreamer-torch.

I'm looking around the code before the full run. I noticed that

1. the code saves an episode at disk everytime an epispde is finished,

2. But it also maintains the episode in the memory thus every new episode is added to the memory replay buffer.

3. In training, the agent samples data from the memory, not from the disk. It assumes that all the available episodes data are stored in the memory. Please check the code.

from dreamer-torch.

Hi tominku,

Yes, you are right. I assumed they load reply buffer from the disk when it requires because they stored the episodes on disk, but it is not.

One thing we can choose is to modify the codes to load the reply buffer from disk not storing it in memory.

Or a hybrid way could be the reply buffer is stored in memory to the pre-defined limitation and beyond that, they can load it from disk. -> It is harder to implement than naively loading reply buffer from disk I think.

Bests, Jaesik.

from dreamer-torch.

Thanks! That might be a good solution.

from dreamer-torch.

If you have no comments for this issue, I will close :)

from dreamer-torch.