This is the official repo for MARVIN implemented in pytorch. For more details on our findings and the results, to learn more about the findings of this work, please check out our paper and our blog post

In this work, we address the problem of routing multiple agents in a coordinated manner, specifically in the context of autonomous mapping. We created a simulated environment to mimic realistic mapping performed by autonomous vehicles with unknown minimum edge coverage and traffic conditions. Here you can find our training code both for training with reinforcement learning and with imitation learning.

• Python>=3.6
• tqdm>=4.36.1
• torch==1.4.0
• tensorboardX>=1.9
• numpy>=1.18.1
• cvxpy==1.1.0a1
• scipy==1.4.1

In a new python environment, run the following command to install all dependencies.

python setup.py install

• Step 1: Download the ground truth model

bash download.sh

It should be able to be found in the marvin/data folder

• Step 1: Begin training

python train.py --tag <your-unique-tag>

This will train the system on a randomly generated set of toy graphs. In order to add in our realistic traffic conditions, run

python train.py --tag <your-unique-tag> --traffic --pass_range 3

If you want to train the system using reinforcement learning, add the --rl tag as an argument

python train.py --tag <your-unique-tag> --traffic --pass_range 3 --rl

Note that the kind of trainer and many of the other hyperparameters can be found in marvin/utils/trainer_parameters.py. Feel free to play around with them as you see fit. In order to set the seed for the entire setup for reproducibility, use the --seed argument.

In order to evaluate a model, simple add the --eval and --load <path-to-model.pb> tags to your original training command. This will skip the training loop and only run the validation loop on the testset instead. The validation loop will also produce records of the previous set of runs in the logs/testing_data for visualization and future qualitative analysis.

python train.py --tag <your-unique-tag> --traffic --pass_range 3 --rl --eval --load <your-weights>

If you want to evaluate using our pretrained models, then run the following for our imitation learning model:

python train.py --tag <your-unique-tag> --traffic --pass_range 3 --eval --load marvin/pretrained/il.pb --seed 1

or the following to run our reinforcement learning model

python train.py --tag <your-unique-tag> --traffic --pass_range 3 --rl --eval --load marvin/pretrained/rl.pb --seed 1

We are currently working to release our street graph dataset. Currently we are in the licensing phase and we will be providing updates on its status as we receive them. As it stands, these are the average runtime costs when we train our system on our toy dataset with realistic conditions with a seed of 0.

Here we illustrate what training should look like for imitation learning (left) and reinforcement learning (right) when the seed is set to zero.

Here is the loss function (orange is the Graph Attention Network, blue is the generalized value iteration network, and green is MARVIN)

Here is the relative performance of the above models, and how they evolve during the training process.

The value function that each agent creates can be visualized to determine what their intentions are and how they can evolve over time.

• Release dataset and dataset description
• Parallelize training
• Generalize training environment
• Port to tensorflow
• Integrate path visualization functions into the codebase