Code and notes from the AI Saturdays (Madrid, 3rd ed.) Reinforcement Learning Track.

For this first session we had to watch the following lectures from CS 285 at UC Berkeley (Deep Reinforcement Learning):

• Lecture 1: Introduction and Course Overview
• Lecture 2: Supervised Learning of Behaviors
• Lecture 4: Introduction to Reinforcement Learning

We also had to take a general look al Pytorch, since this would be the DL framework used throughout the course.

During the session we had a look at some basic Reinforcement Learning concepts:

• Overview of environments, statuses, observations, policies and actions.
• Commonly used notation
• Markov chains
• V and Q functions
• Model Free and Model Based Reinforcement Learning
• Imitation Learning

Afterward, we started to code a couple of small examples in Gym. This included:

• A script that executes a series of episodes of a game with a random agent, which samples actions uniformly from the action space of the environment.
• A script that allows the user to play the game and record the gameplay (observations, actions and rewards).

For the second session we decided to review the following pages from OpenAI Spinning Up:

• Part 1: Key Concepts in RL
• Part 2: Kinds of RL Algorithms

We also decided to watch the following lecture from David Silver:

• Video
• Slides

Finally, we read the 4th chapter from Deep Reinforcement Learning Hands On, by Maxim Lapan, (Chapter4: The Cross-Entropy Method).

Sadly, I could not attend the session personally, but it consisted on reviewing the Cross-Entropy method, analyzing in which environments it would be most applicable, and implementing them for a few environments.