This repository contains the code used in the undergraduate thesis in Mechatronics Engineering, at the University of Brasilia, entitled "Behavior selection for multiple autonomous agents with reinforcement learning in stochastic environments".

The idea is to have multiple simulated robotic agents learning to select appropriate behaviors in a stochastic environment. The uncertainty of a state is handled through Bayesian Programming, and the agents learn by applying Q-learning with function approximation.

Currently, the approach is tested in a predator-prey problem using a modified version of the Pac-Man game with introduced uncertainties. Therefore, this simplified multi-agent situation aims to answer the following question: can the ghosts learn to get the Pac-Man?

The Pac-Man AI Projects provides six Pac-Man-like simulators that are free to use for educational purposes. The one we will be using is Project 5: Classification, which provides an arena mimicking the complete Pac-Man game, including various ghosts.

This project requires the following Python packages:

• Tkinter: graphical user interfaces
• ZeroMQ: interprocess communication
• Matplotlib: graphics plotting
• Numpy: numerical computation

This assumes a GNU/Unix distribution (Ubuntu), but everything is in Pỳthon so the setup shouldn't be too different for other platforms.

Install by running the following commands.

sudo apt-get install python python-dev python-pip python-tk libzmq-dev python-matplotlib
sudo pip install pyzmq

Installing on an OS X distribution requires a special setup using Homebrew and the XCode.

• Homebrew: package manager
• XCode: IDE developed by Apple

XCode shall be downloaded from the provided link or in your App Store. Then you need to run the following commands.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install python --with-tcl-tk --enable-threads --with-x11
pip install matplotlib bumpy pyzmq

All documentation are kept in our wiki, with usage examples and modules explanations.

• Undergraduate thesis: PORTELA, M.V. - Behavior selection for multiple autonomous agents with reinforcement learning in stochastic environments - University of Brasilia - Brasilia, Brazil - 2015
• Conference paper: PORTELA, M. V., RAMOS, G. N. - State estimation and reinforcement learning for behavior selection in stochastic multiagent systems - Proceedings of SBGames 2015 - Teresina, Brazil - 2015