Centralized-V between IPPO and MAPPO about on-policy HOT 4 CLOSED

Yes, the value network architecture is not affected by use_centralized_V. but i think the first dimension is related to the batch_size instead of num_agents.

from on-policy.

Hi @zoeyuchao

Just a follow-up question, so if I use a separate runner, does that mean each agent has its' own policy and value network? so n agents means n policy and n value networks. And even though I set use_centralized_V, that means for each agent's value network, it accepts the same shared observation, and outputs its' own value estimation, right? use_centralized_V does not mean there is only one value network for all agents (under separate runner), right?

just want to be more clear about this. Thanks!

from on-policy.

@xubo92 yes, u are right:)

from on-policy.

Ok, yeah that makes sense! thanks!

from on-policy.