For this project, the Reacher environment is used to train the agent. In this environment, a double-jointed arm moves to target locations. the agent is trained so that the arm follows a desired trajectory.

A reward of +0.1 is provided for each time step that the agent's hand is in the goal location.

The observation space consists of 33 variables corresponding to position, rotation, velocity, and angular velocities of the arm.

Each action is a vector with four numbers, corresponding to torque applicable to two joints. Every entry in the action vector is a number between -1 and 1.

The code in this repo solves two separate versions of the Unity environment.

• The first version contains a single arm.
• The second version contains 20 identical arms, each with its own copy of the environment. In this version, multiple arms are used to collect feedback in parallel. This version is useful for algorithms like PPO, A3C, and D4PG that use multiple (non-interacting, parallel) copies of the same agent.

Agents must get an average score of +30 (over 100 consecutive episodes, and over all agents).

• Specifically, after each episode, the rewards that each agent received (without discounting) is added up, to get a score for each agent. Then average of these scores is taken.
• This yields a moving average score for each episode (where the average is over all agents).
• The environment is considered solved, when the average (over 100 episodes) of those average scores is greater than 30.

Follow the instructions below to set up your python environment to run the code in this repository,

1. Create (and activate) a new environment with Python 3.6.

   • Linux or Mac:

   conda create --name drlnd python=3.6
   source activate drlnd

   • Windows:

   conda create --name drlnd python=3.6 
   activate drlnd

2. Clone the repository DRLND, and navigate to the python/ folder. Then, install several dependencies.

   • git clone https://github.com/udacity/deep-reinforcement-learning.git
     cd deep-reinforcement-learning/python
     pip install .

3. Create an IPython kernel for the drlnd environment.

   •   python -m ipykernel install --user --name drlnd --display-name "drlnd"

4. Before running code in the notebooks, change the kernel to match the drlnd environment by using the drop-down Kernel menu.

1. Download the environment from one of the links below. You need only select the environment that matches your operating system:

   • Version 1: One (1) Agent

     • Linux: click here
     • Mac OSX: click here
     • Windows (32-bit): click here
     • Windows (64-bit): click here

   • Version 2: Twenty (20) Agents

     • Linux: click here
     • Mac OSX: click here
     • Windows (32-bit): click here
     • Windows (64-bit): click here

   (For Windows users) Check out this link if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.

2. Place the file in the directory where you have cloned this project, and unzip (or decompress) the file.

3. Name the directory where the 20-agent version is unzipped (decompressed) as Reacher20.app and the one agent version as Reacher.app

4. The repo contains the zip files for the Mac OSX operating system.

Run the Continuous_Control_Submission.ipynb notebook to get started.

1. Navigate to the root directory of this repo.
2. Start the Jupyter Notebook in the activated drlnd conda environment

   • jupyter notebook

3. Open Continuous_Control_Submission.ipynb in the Jupyter file browser

The details can be found in REPORT.ipynb file