Repository for the class 'Software Technology Course with a Varying Content' and the topic 'Introduction in Artificial Intelligence'.

This repository contains two different controlling algorithms. PID and Reinforcement learning (Q-Learning). The purpose of the lecture was to control a segway-like platform and compare the controlling behavior.

The result is that the PID controller is able to control the model. But it takes a lot of optimization to get the best parameters. Also the reinforcement learning agent is able to control the model. It is learning really fast and after four to five failures it balances the segway multiple minutes in place.

Some useful links:

• V-Rep Doku

1. Download, extract and install the education version of VREP (Download-Link)
2. Clone this repository
3. Start VREP and open one of the scenes in the folder vrep

• There are three different sizes of inverted pendulums. All work but have different behavior as mass and height changes

4. Run python rl_simulation.py from a console
5. See how the the reinforcement learning agent learns to balance the inverted pendulum

If you want to have more information of the reinforcement agent, change the debug flag at line 14 in the file rl_simulation.py from 0 to 1

In this section you find the evaluated values for a good controlling with a PID controller

A few different tunings found out to work more or less well.

• 50ms - 13.7 0.199 1286
• 10ms - 56.25, 2.736, 1389

• 50ms - 13.76 0.1990 1609

• 50ms - 13.7 0.2197 1194

• 50ms - 36.72, 0.5333, 2036 (bad)
• 10ms - ?

• 10ms - 26.11, 1.971, 2925

• 10ms - 12.73 0.3582 3042

• Doomed to drift...
• 50ms - 0.5, 0.005, 12.5

• 50ms: 21.000000 9.009500 16.550000
• 10ms: 79.0 19.9 41.4

• 10ms - 324 3.24 4500

• 10ms - 80.77 0.5011 330.0

The basic Q-Learning approach is used to implement an easy and fast solution to control the segway.