somthing about replay buffer about d4pg HOT 2 CLOSED

I don't think its really necessary to have a lock between the agents themselves. Of course, if you want to be 100% 'thread-safe' then yes you could do this, but any other implementations I've seen of multi-threaded data gathering do not bother with this. The way I see it is that the worst that could happen is that on the rare occasions two or more agents try to add their samples at the same time, one sample may be overwritten. This isn't a significant issue given that we aren't adding to the buffer every step, only at the end of an episode, so the chances of trying to add at the exact same time as another agent are fairly low.

from d4pg.

I don't think its really necessary to have a lock between the agents themselves. Of course, if you want to be 100% 'thread-safe' then yes you could do this, but any other implementations I've seen of multi-threaded data gathering do not bother with this. The way I see it is that the worst that could happen is that on the rare occasions two or more agents try to add their samples at the same time, one sample may be overwritten. This isn't a significant issue given that we aren't adding to the buffer every step, only at the end of an episode, so the chances of trying to add at the exact same time as another agent are fairly low.

thanks!

from d4pg.