[Question] Running Multi-threaded PPO training independently with no interference about stable-baselines3 HOT 3 OPEN

Hello,
if you want to parallelize gradient steps, you need to have a look at Stable-Baselines-Team/stable-baselines3-contrib#11 and linked issues.
if you want to parallelize data collection, you need to use VecEnv.

I'm also not sure why you have some TF code in there...

from stable-baselines3.

Hi @araffin, thanks for your response.

I may have not been quite clear about what I wanted to achieve.

I am parallelizing external to the model instance and not within it. That is, with each different XML file, I run a train_ppo.py instance per thread. Depending on the XML file count, the number of parallel processes increases, and thus gym envs. are created and PPO models are trained in parallel. It is here I face the problem.

I notice that the gradient steps are somehow taking longer and longer to process with the increase in the thread count (i.e. increase in independent gym env count), which normally shouldn't be the case because each thread must be treated independently and I should have models trained independently. (Please notice the time_elapsed in seconds for just 4000 env. steps in the attached screenshot)

Hence your suggestions about parallel gradient steps and data collection through Stable-Baselines-Team/stable-baselines3-contrib/issues/11 and VecEnv, though useful, aren't addressing my problem, because I still have num_cpu=1 even in VecEnv.

Hope this clarifies things further. Please let me know how I may go about this problem, thanks.

And yes, the TF code is a blunder please ignore it.

from stable-baselines3.