koulanurag / ma-gym Goto Github PK

Is it possible to add the capability of adjusting the field of view for the agents? I find the current 5x5 view to be too big and naively changing that parameter would break the code. Thanks!

When calling gym.make() to create any ma-gym environment, the following warning pops up

UserWarning: WARN: Box bound precision lowered by casting to float32
Warning link

I have created a pull request #29 to avoid this warning.

Thank you.

Hi, I was wondering could you share the weights of the agents that you trained in these

Hi!
Thank you for your multiagent gym.
I have tested the Switch2-v0 environment.
Unfortunately, the agent (1)-red and (2)-blue cannot go to a certain location such as ([2,1] and [2,5]).
Because the inactivated agents (green and yellow) occupied those locations.
Is there any method to inactivate green and yellow agents in the grid world?
(ex. I want to move the red agent from [1,1] to [2,1])

Thanks!

Hi,
Is any of the environment a cooperative multi-agent rl where human and agent can interact with the environment at the same time to get rewards.

Raises error as below

traffic_junction/traffic_junction.py", line 85, in get_agent_obs
    pos = self.agent_pos[agent_i]
KeyError: 0

I tried printing agent_pos which has empty dictionary {}. Checking the code, I realised agent_pos is never being updated like in other envs (Switch, Checkers etc...)

Hi, I actively use your ma-gym. Thank you.
I have a question about the number of agents in switch environment.
If I choose the 'switch2-vo' environment, then the number of agents set to 2 automatically?
Or do I have to revise the number of agents in the switch_one_corridor.py file?
Thank you. :)

Hello.
I'm trying to execute the code in colab
which is https://colab.research.google.com/github/matteokarldonati/Counterfactual-Multi-Agent-Policy-Gradients/blob/master/COMA.ipynb#scrollTo=cvdZvb-V6d63

But there is an error in the init.py like below

Do you know what's the problem with it??
Thank you.

Thank you very much for creating the environment.
However, I am having the following warnings. Is there anything I should update?
Thanks

env=gym.make('Checkers-v0')
envs\marl\lib\site-packages\gym\logger.py:34: UserWarning: ←[33mWARN: Box bound precision lowered by casting to float32←[0m
warnings.warn(colorize("%s: %s" % ("WARN", msg % args), "yellow"))

Hi all,

Currently, sampling the observation space of a multi-agent environment seems to be failing.
Specifically:

env = gym.make('Switch2-v0')  
env.observation_space.sample()

raises

NameError: name 'agents_observation_space' is not defined

This appears to be a simple typo. PR #7 fixes this.

Hi, not really an issue found but I hope I can get answers to my question.

Taking PongDuel-v0 as example, the observation returns a 2-D list.
obs=[[0.625, 0.03333, 0.5, 0.3333, 0, 0, 0, 1, 0, 0], [0.625, 0.9333, 0.325, 0.6, 0, 0, 0, 1, 0, 0]]

According to the wiki, it is the agent coordinate and ball location. Are both agent coordinates represented in obs[0] and ball coordinate is represented in obs[1]? May I know how are they being represented using the arrays with size 10?

Thanks.

Code, which is written with Gym in mind often expects both action spaces and observation spaces provided upon the creation of the environment.

An example of the observation_space can be seen in the classic Atari environment of gym.

Without it, the library can't be used with frameworks such as Ray RLLib. It explicitly requires both action spaces and observation spaces to be provided in policies.

Therefore, if someone wants to use ma-gym with ray, observation spaces dictionaries must be deduced and constructed manually.

From the picture of switch2, row3 and column2 is accessable, while when I run the programme, the agent can't reach here?

I'm really enjoying using ma-gym. I looked through the VDN paper and my understanding is that they only test on Switch2. I was wondering if there is a paper that tests on Switch4, or if you guys just designed that task yourselves (asking because I'm currently testing out a value decomposition method on it and would love to see if anyone else has solved it using a similar approach).

For example, with a grid size of (7,7) and an agent's position (6,6) this yields [6/7,1].
Is there a reason for treating the x-coordinate differently? Or is it a bug?

Using pip install -e . gives me the following error:

Collecting scipy==1.3.0 (from ma-gym==0.0.1)
  Could not find a version that satisfies the requirement scipy==1.3.0 (from ma-gym==0.0.1) (from versions: 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.16.0, 0.16.1, 0.17.0, 0.17.1, 0.18.0rc2, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 1.0.0b1, 1.0.0rc1, 1.0.0rc2, 1.0.0, 1.0.1, 1.1.0rc1, 1.1.0, 1.2.0rc1, 1.2.0rc2, 1.2.0, 1.2.1, 1.2.2, 1.2.3)
No matching distribution found for scipy==1.3.0 (from ma-gym==0.0.1)

Elif branch in Checkers L204 will never be executed, because after the all agents are done the conditions for if branch holds.

Here is the error message:
`Python 3.8.5 (default, Sep 4 2020, 07:30:14)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import gym
import ma_gym
env = gym.make('Switch2-v0')
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 145, in make
return registry.make(id, **kwargs)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 90, in make
env = spec.make(**kwargs)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 59, in make
cls = load(self.entry_point)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 18, in load
mod = importlib.import_module(mod_name)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 961, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ma_gym.envs'
env = ma_gym.make('Switch2-v0')
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'ma_gym' has no attribute 'make'
env = gym.make('Switch2-v0')
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 145, in make
return registry.make(id, **kwargs)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 90, in make
env = spec.make(**kwargs)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 59, in make
cls = load(self.entry_point)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/site-packages/gym/envs/registration.py", line 18, in load
mod = importlib.import_module(mod_name)
File "/root/anaconda3/envs/marl3.8/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 961, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'ma_gym.envs'`

/content/ma-gym/ma_gym/init.py in
8 # Register openai's environments as multi agent
9 # This should be done before registering new environments
---> 10 env_specs = [env_spec for env_spec in envs.registry.all() if 'gym.envs' in env_spec.entry_point]
11 for spec in env_specs:
12 register(

AttributeError: 'dict' object has no attribute 'all'

Hello, whenever I try to use ma-gym to create a multiagent environment in Colab it always gives the same error:

/content/ma-gym/ma_gym/init.py in
13 id='ma_' + spec.id,
14 entry_point='ma_gym.envs.openai:MultiAgentWrapper',
---> 15 kwargs={'name': spec.id, **spec._kwargs}
16 )
17

AttributeError: 'EnvSpec' object has no attribute '_kwargs'

I'm trying to run the following:

import gym
env = gym.make('ma_gym:Switch2-v0')
done_n = [False for _ in range(env.n_agents)]
ep_reward = 0

obs_n = env.reset()
while not all(done_n):
env.render()
obs_n, reward_n, done_n, info = env.step(env.action_space.sample())
ep_reward += sum(reward_n)
env.close()

I'm new to all of this so sorry if this doesn't meet the standards of an issue.

As I have point it out in Lumberjacks pull request there are several places where the environments use module random without any seeding and/or not using self.np_random attribute.

I will come up with a patch for this, however, what is the purpose of Chekers L239? And why the method seed return the variable seed2 as well?

_agent_i_obs = [pos[0] / self._grid_shape[0], pos[1] / (self._grid_shape[1] - 1)] # coordinates
Should be
_agent_i_obs = [pos[0] / (self._grid_shape[0] - 1), pos[1] / (self._grid_shape[1] - 1)] # coordinates

Hello.
I want to ask about the reward in switch2-v0.
The reward for agents is coming out as [-0.1, -0.1], but when I looked at the environment code, it seems there is no reason for it to output like this. Do you know Why is this happening?
Thank you :)

Env: Combat-v0
obs_n: [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 3.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.26666666666666666, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.6, 0.6, 0.6, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.6, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, 0.0, 3.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 4.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 3.0, 3.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.4, 0.4, 0.4, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4, 0.0, 0.0, 0.0, 0.4, 0.0, 0.0, 0.0, 0.4666666666666667, 0.4666666666666667, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 3.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.26666666666666666, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5333333333333333, 0.5333333333333333, 0.5333333333333333, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5333333333333333, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 3.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.26666666666666666, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.26666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.4666666666666667, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 4.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.4666666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.6, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.6, 0.0, 0.0, 0.0, 0.6, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.6, 0.0]]
how to understand?

I use: env= gym.make("TrafficJunction-v1")
want to get a traffic control task.
But it cannot work, it returns that:

Traceback (most recent call last):
File "xxx/traffic_test.py", line 74, in
testcmarl()
File "xxx/traffic_test.py", line 37, in testcmarl
obs = env.reset()
File "xxx/ma-gym/ma_gym/envs/traffic_junction/traffic_junction.py", line 141, in reset
return self.get_agent_obs()
File "xxx/ma-gym/ma_gym/envs/traffic_junction/traffic_junction.py", line 85, in get_agent_obs
pos = self.agent_pos[agent_i]
KeyError: 0

Hi~ Currently, the default PongDuel will terminate in the 11th round.
If the max_round=10 means the final round, should it terminate then rather than 11? And It will also not take the final reward into account.
Therefore, It seems that it may more straightforward to just stop in the 10th.

https://github.com/koulanurag/ma-gym/blob/master/ma_gym/envs/pong_duel/pong_duel.py#L233-L243

    # if ball is beyond paddle, initiate a new round
    if self.ball_pos[1] < 1:
        rewards = [0, self.reward]
        self.__rounds += 1
    elif self.ball_pos[1] >= (self._grid_shape[1] - 1):
        rewards = [self.reward, 0]
        self.__rounds += 1


    if self.__rounds > self._max_rounds:
        self._agent_dones = [True for _ in range(self.n_agents)]
        rewards = [0 for _ in range(self.n_agents)]

If looks good, I can submit a PR with this small change:

if self.__rounds == self._max_rounds:
    self._agent_dones = [True for _ in range(self.n_agents)]
else:
    ...

ERROR: Could not find a version that satisfies the requirement scipy==1.7.1 (from versions: 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.12.1, 0.13.0, 0.13.1, 0.13.2, 0.13.3, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.16.0, 0.16.1, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 1.0.0b1, 1.0.0rc1, 1.0.0rc2, 1.0.0, 1.0.1, 1.1.0rc1, 1.1.0, 1.2.0rc1, 1.2.0rc2, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.3.0rc1, 1.3.0rc2, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0rc1, 1.4.0rc2, 1.4.0, 1.4.1, 1.5.0rc1, 1.5.0rc2, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.6.0rc1, 1.6.0rc2, 1.6.0, 1.6.1, 1.6.2, 1.6.3, 1.7.0rc1, 1.7.0rc2, 1.7.0, 1.7.1)

Getting this error as I'm trying to install all the packages required from a git repo in a virtual environment. This specific error is from the package scipy==1.7.1 I have upgraded pip and python to its latest version but still getting this result. For some reason this installment works perfectly fine on my other windows PC, but doesn't work on my Mac M1.

Python version: 3.9.7
Pip version: 21.2.4