This is the implementation of the paper SHAQ: Incorporating Shapley Value Theory into Q-Learning for Multi-Agent Reinforcement Learning (https://arxiv.org/abs/2105.15013).

The implementation is based on PyMARL (https://github.com/oxwhirl/pymarl/). Please refer to that repo for more documentation.

The baselines used in this paper are from the repo of Weighted QMIX (https://github.com/oxwhirl/wqmix). To know more about baselines, please refer to that repo.

The model implemented in this paper is based on Pytorch 1.4.0.

In particular implementations for:

• SHAQ

Note that in the repository the naming of certain hyper-parameters and concepts is a little different to the paper:

• $\hat{\alpha}$ in the paper is alpha in the code

The install_sc2.sh script will install SC2.4.6.2.69232.

The config file (src/config/algs/shaq.yaml) contains default hyper-parameters for SHAQ. These were changed when running the experiments for the paper (epsilon_anneal_time = 1000000 for the super-hard maps in SMAC and Predator-Prey).

About the hyperparameter settings of variant experiments, the full details are listed as below.

Please see the Appendix of the paper for the exact hyper-parameters used.

As an example, to run the SHAQ on SMAC: 2c_vs_64zg with epsilon annealed over 50k time steps:

python3 src/main.py --config=shaq --env-config=sc2 with env_args.map_name=2c_vs_64zg alpha_lr=0.002 epsilon_anneal_time=50000

If you use part of the work mentioned in this paper, please cite

@misc{wang2021shaq,
      title={SHAQ: Incorporating Shapley Value Theory into Q-Learning for Multi-Agent Reinforcement Learning},
      author={Jianhong Wang and Jinxin Wang and Yuan Zhang and Yunjie Gu and Tae-Kyun Kim},
      year={2021},
      eprint={2105.15013},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}