Q: reduced #process on GPU? about pytorch-a2c-ppo-acktr-gail HOT 5 CLOSED

There was a bug in PyTorch that initialized cuda in every separate process (that wasn't needed).

Apparently, it was fixed recently.

Does it change performance in any way?

from pytorch-a2c-ppo-acktr-gail.

Oh, I found I made a mistake: the single GPU process scenario is not an Atari game. (It's a game env that I created with no graphics/images). And I rerun the Atari game, it still use N+1 GPU processes, and performance is the same.

So I guess it's the gym game env that create a GPU process for each game in training; because Atari game are video (image) games? thus N+1 (main).

Comparing the two scenario, my next question is: so all the CNN network (CNNPolicy) are running on that single main CPU process, instead of each game's env GPU process?

Then I wonder if it's possible to distribute CNN computation on each of the GPU process and get some performance gain?

from pytorch-a2c-ppo-acktr-gail.

Each process just initializes (but doesn't use CUDA), it's a bug (feature?) in PyTorch.

At the moment, all environments run in CPU threads. Then, the observations get concatenated and sent to GPU.

from pytorch-a2c-ppo-acktr-gail.

gym doe not depend on PyTorch, so it's because of WrapPyTorch, is this absolute necessary?

from pytorch-a2c-ppo-acktr-gail.

Yes. It's because of data format.

from pytorch-a2c-ppo-acktr-gail.