transferrable models about stable-baselines3 HOT 5 CLOSED

Thanks for reply. I have a related question. I was considering exporting actor network only but I noticed that model.predict and model.actor.predict returns different values. Is it expected behavior?

model = SAC('MlpPolicy', 'CartPole-v1', verbose=1)
env = gym.make ('CartPole-v1')
obs = env.reset()
print(model.predict(obs))
print(model.actor.predict(obs))

Returns: (array([-0.20559013], dtype=float32), None)
(array([-0.9510132], dtype=float32), None)

Edit: My mistake. It works as expected when both is called as deterministic.

from stable-baselines3.

You can do this with original stable-baselines using get_parameters and load_parameters, and with bit of manual tinkering. You need manually create the mismatching parameter arrays for env_2 agent and update correct parameters with ones from env_1 agent. E.g. if only the last fully-connected layer changes, you need to manually crate final_params = np.zeros((N, 20)) and then assign the weights of original parameters to it final_params[:, :10] = original_params. If you want to modify save .zip files, the format is specified here.

Similar support is planned / partially working in SB3, but still needs to go through a check and review.

from stable-baselines3.

So, is exporting saved model as Pytorch model not supported yet in SB3? Is there a way to get model parameters as in stable-baselines' get_parameters function?

from stable-baselines3.

So, is exporting saved model as Pytorch model not supported yet in SB3?

It is for policy. (but not properly documented yet). There are nn.Module anyway.

from stable-baselines3.

Similar support is planned / partially working in SB3, but still needs to go through a check and review.

done in #138 (will merge today)

from stable-baselines3.