Road traffic simulator for OpenAI Gym

git clone [email protected]:rltraffic/gym-graph-traffic.git
cd gym-graph-traffic
pip install -e .

python3 examples/minimal.py 

All parameters considering configuration are available in gym_graph_traffic/envs/params.py file:

According to default values:

• Experiment is a full run of a learning algorithm.

  1 experiment = ??? episodes (set manually by user in training loop)

• Episode is a single complete simulation run from initial state to terminal state. The Agent’s goal it to maximize the total reward it receives during an episode. Different episodes are independent of one another.

  1 episode = 200 steps

• Step is a part of an episode executed under single action. (Number of steps = number of actions taken).

  1 step = 60 seconds (of real traffic simulation)

  1 step = 22 updates (STEP_LENGTH / SECONDS_PER_UPDATE)

• Update is a single step of a Nagel-Schreckenberg cellular automaton model.

  1 update = 2.7 seconds (of real traffic simulation)

Because a single action is a vector representing as an int. We introduced action_int – a single number encoding the action_array. The action_array is a base x representation of action_int, where x is equal to the length of the RED_DURATIONS array.

Example of translation for easy preset:

                 0    1    2    3   # indices
RED_DURATIONS = [0,  20,  40,  60]

The length of the RED_DURATIONS array equals 4, so encoding has a base of 4 (quaternary) and 4 bits of encoding – easy road network has 4 intersections.

      4^3  4^2  4^1  4^0
20   [0,   1,   1,   0]

Above can be seen the quaternary representation of 20 (4^2 + 4^1 = 20). There is a mapping between the bits in the quaternary representation and the RED_DURATIONS array indices. Therefore, the translated action_int = 20 equals [0, 20, 20, 0].

• First and forth intersections will have full, 60-second phase of horizontal flow.
• Second and third intersection will have 20 seconds of vertical, and then 40 seconds of horizontal flow.

Diagram representation of single step.

Total distance covered by all cars in one step.

Two vectors – average number of cars and average speed per road segment.

The model is toroidal (periodic).

By default there are 4 available presets:

• easy - one-lane, one-way road with 4 intersections.
• two_roads - two-way road with 4 intersections.
• grid_4x2 - 4 by 2 grid of two-way roads (with a total of 8 intersections).
• grid_3x3 - 3 by 3 grid of two-way roads (with a total of 9 intersections).

grid_3x3 preset.

Two create different sizes of presets there are two functions supplied in util/grid.py: make_grid and make_line.

The environment is not yet compatible with Gym-like rendering. Thus all rendering options have to be supplied within aforementioned params.py file.