airi-institute / pogema Goto Github PK

POGEMA stands for Partially-Observable Grid Environment for Multiple Agents. This is a grid-based environment that was specifically designed to be flexible, tunable and scalable. It can be tailored to a variety of PO-MAPF settings.

License: MIT License

Shell 0.07% Python 99.93%

reinforcement-learning gym-environment simulation pathfinding po-mapf marl mapf

pogema's Introduction

Partially-Observable Grid Environment for Multiple Agents

Partially Observable Multi-Agent Pathfinding (PO-MAPF) is a challenging problem that fundamentally differs from regular MAPF. In regular MAPF, a central controller constructs a joint plan for all agents before they start execution. However, PO-MAPF is intrinsically decentralized, and decision-making, such as planning, is interleaved with execution. At each time step, an agent receives a local observation of the environment and decides which action to take. The ultimate goal for the agents is to reach their goals while avoiding collisions with each other and the static obstacles.

POGEMA stands for Partially-Observable Grid Environment for Multiple Agents. It is a grid-based environment that was specifically designed to be flexible, tunable, and scalable. It can be tailored to a variety of PO-MAPF settings. Currently, the somewhat standard setting is supported, in which agents can move between the cardinal-adjacent cells of the grid, and each action (move or wait) takes one time step. No information sharing occurs between the agents. POGEMA can generate random maps and start/goal locations for the agents. It also accepts custom maps as input.

Installation

Just install from PyPI:

pip install pogema

Using Example

from pogema import pogema_v0, Hard8x8

env = pogema_v0(grid_config=Hard8x8())

obs, info = env.reset()

while True:
    # Using random policy to make actions
    obs, reward, terminated, truncated, info = env.step(env.sample_actions())
    env.render()
    if all(terminated) or all(truncated):
        break

Environments

Config	agents density	num agents	horizon
Easy8x8	2.2%	1	64
Normal8x8	4.5%	2	64
Hard8x8	8.9%	4	64
ExtraHard8x8	17.8%	8	64
Easy16x16	2.2%	4	128
Normal16x16	4.5%	8	128
Hard16x16	8.9%	16	128
ExtraHard16x16	17.8%	32	128
Easy32x32	2.2%	16	256
Normal32x32	4.5%	32	256
Hard32x32	8.9%	64	256
ExtraHard32x32	17.8%	128	256
Easy64x64	2.2%	64	512
Normal64x64	4.5%	128	512
Hard64x64	8.9%	256	512
ExtraHard64x64	17.8%	512	512

Baselines

The baseline implementations are available as a separate repository.

Interfaces

Pogema provides integrations with a range of MARL frameworks: PettingZoo, PyMARL and SampleFactory.

PettingZoo

from pogema import pogema_v0, GridConfig

# Create Pogema environment with PettingZoo interface
env = pogema_v0(GridConfig(integration="PettingZoo"))

PyMARL

from pogema import pogema_v0, GridConfig

env = pogema_v0(GridConfig(integration="PyMARL"))

SampleFactory

from pogema import pogema_v0, GridConfig

env = pogema_v0(GridConfig(integration="SampleFactory"))

Gymnasium

Pogema is fully capable for single-agent pathfinding tasks.

import gymnasium as gym
import pogema

# This interface provides experience only for agent with id=0,
# other agents will take random actions.
env = gym.make("Pogema-v0")

Example of training stable-baselines3 DQN to solve single-agent pathfinding tasks:

Customization

Random maps

from pogema import pogema_v0, GridConfig

# Define random configuration
grid_config = GridConfig(num_agents=4,  # number of agents
                         size=8, # size of the grid
                         density=0.4,  # obstacle density
                         seed=1,  # set to None for random 
                                  # obstacles, agents and targets 
                                  # positions at each reset
                         max_episode_steps=128,  # horizon
                         obs_radius=3,  # defines field of view
                         )

env = pogema_v0(grid_config=grid_config)
env.reset()
env.render()

Custom maps

from pogema import pogema_v0, GridConfig

grid = """
.....#.....
.....#.....
...........
.....#.....
.....#.....
#.####.....
.....###.##
.....#.....
.....#.....
...........
.....#.....
"""

# Define new configuration with 8 randomly placed agents
grid_config = GridConfig(map=grid, num_agents=8)

# Create custom Pogema environment
env = pogema_v0(grid_config=grid_config)

Citation

If you use this repository in your research or wish to cite it, please make a reference to our paper:

@misc{https://doi.org/10.48550/arxiv.2206.10944,
  doi = {10.48550/ARXIV.2206.10944},  
  url = {https://arxiv.org/abs/2206.10944},
  author = {Skrynnik, Alexey and Andreychuk, Anton and Yakovlev, Konstantin and Panov, Aleksandr I.},
  keywords = {Machine Learning (cs.LG), Artificial Intelligence (cs.AI), Multiagent Systems (cs.MA), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {POGEMA: Partially Observable Grid Environment for Multiple Agents},
  publisher = {arXiv},
  year = {2022},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

pogema's People

Contributors

Stargazers

Watchers

Forkers

tviskaron techthiyanes eles13 panyshevalex konstantingordeev aandreychuk atsuya-ktd2 lvyv edvard88 umoony vilaksh01 kjman678 vineettambe hanzleader marimeireles zereaf

pogema's Issues

Feature request: update PettingZoo version

Hi, would it be possible to update this repo to use the most recent version of PettingZoo? We want to list this project in PettingZoo's third-party-environments, but we can only include environments which work with the current version.

If you need any help working out issues due to different versions feel free to ask, there were some breaking changes in version 1.2, so it requires a bit of code changes to adapt. The previous API returned done in the step() function, whereas the new one returns truncated and terminated (matching gymnasium). There is a migration guide for gymnasium explaining the changes further, the steps should be basically the same (we're working on making resources for updating old PettingZoo repositories as well): https://gymnasium.farama.org/content/migration-guide/

Custom observation

Is it possible to generate custom observation like this?
[[obstacles in agent's FOV], [other agents in agent's FOV], [guide to destination]]

At the end I hope to implement a similar environment defined in this paper (V. B. Observation representation):
Reference of the code

Thanks a lot!

How to get the big map from the environment?

I want to plot the global map with agents though matplotlib. How can I do that? Thank you.

from pogema import pogema_v0, Hard8x8


def main():
    env = pogema_v0(grid_config=Hard8x8())
    my_map = Hard8x8()  # there is no map inside :(
    obs = env.reset()

    while True:
        # Using random policy to make actions
        obs, reward, terminated, info = env.step(env.sample_actions())
        env.render()
        if all(terminated):
            break

Unable to perform training pogema_v0 environment

I'm trying to run the below code

import gymnasium as gym
from pogema import pogema_v0, GridConfig

# Create Pogema environment with PettingZoo interface
env = gym.make("Pogema-v0", grid_config=GridConfig(size=8, density=0.3, num_agents=1, max_episode_steps=30))
from stable_baselines3 import DQN

dqn_agent = DQN('MlpPolicy', env, verbose=1, tensorboard_log="./dqn_pogema_tensorboard/")
dqn_agent.learn(1000000, log_interval=1000, eval_env=env, tb_log_name="baseline")

But it fails in the last before line and I get the following error:

AssertionError: The algorithm only supports (<class 'gym.spaces.discrete.Discrete'>,) as action spaces but Discrete(5) was provided

It appears to me that there is a confusion between gymnasium and gym since both are used in this piece of code. Not sure how to fix this error.

Example not working

installing latest version using pip and the example doesn't work - seems you still implement the gym API instead of gymnasium.
pogema version - 1.1.6

How to choose map in Pymarl style?

Hi,
How to choose map in pymarl style code?

Rendering Video Issues

Hi bro,
I don't want it to be displayed on the command line. I want it to be a video of the whole pathfinding process. How do I do that? I checked render() and found there was very little I could do...

Run the algorithms

Can you provide a small example of running the baseline algorithms without docker or additional projects, please?
It is ok for me to run the learning phase by myself. I just want to keep everything as simple as possible and I want to create my own algorithms to compare with yours. Thank you in advance.

Goal positon is far away from the agent

if goal position is far away from the agent (out of the observation area), what is represented by the third layer of the observation? How can agent understand where to move?

Gym do not see pogema env.

Consider the code i took from https://pypi.org/project/pogema/1.0.1/

import gym
from pogema import GridConfig

# Define random configuration
grid_config = GridConfig(num_agents=4,  # number of agents
                         size=8, # size of the grid
                         density=0.4,  # obstacle density
                         seed=1,  # set to None for random 
                                  # obstacles, agents and targets 
                                  # positions at each reset
                         max_episode_steps=128,  # horizon
                         obs_radius=3,  # defines field of view
                         )

env = gym.make('Pogema-v0', grid_config=grid_config)
env.reset()
env.render()

I got the error:

Traceback (most recent call last):

  Cell In[5], line 15
    env = gym.make('Pogema-v0', grid_config=grid_config)

  File ~\anaconda3\envs\rl_hw5\lib\site-packages\gym\envs\registration.py:569 in make
    _check_version_exists(ns, name, version)

  File ~\anaconda3\envs\rl_hw5\lib\site-packages\gym\envs\registration.py:219 in _check_version_exists
    _check_name_exists(ns, name)

  File ~\anaconda3\envs\rl_hw5\lib\site-packages\gym\envs\registration.py:197 in _check_name_exists
    raise error.NameNotFound(

NameNotFound: Environment Pogema doesn't exist. Did you mean: `Pong`?

My environment is defined below, also inform I install gym as pip install gym[atari,accept-rom-license]

packages in environment at C:\Users\satyr\anaconda3\envs\rl_hw5:

The example in README.md is incorrect

the env.reset() returns only obs (without info)
the env.step() returns obs, reward, terminated, info and not obs, reward, terminated, truncated, info

from pogema import pogema_v0, Hard8x8


def main():
    env = pogema_v0(grid_config=Hard8x8())
    obs = env.reset()  # here

    while True:
        # Using random policy to make actions
        obs, reward, terminated, info = env.step(env.sample_actions())  # and here
        env.render()
        if all(terminated):
            break


if __name__ == '__main__':
    main()