Small project testing the functionalities of Keras Tuner and benchmarking the three available optimization algorithms.

Given a learning algorithm and a problem to solve (in the simplest case these are classification or regression tasks) we know that exists a set of hyperparamenters which allow the algorithm to converge to the optimal solution both in terms of loss reduction and generalizability to previously unseen data.

Given the magnitude of all the possible combinations of hyperparameters and their associated values (i.e. the size of the hyperparameters space) the process of finding the best set is often left to optimization algorithms which aims to find promising candidates in an efficient manner.

Therefore, understanding which algorithm is able to achieve the best performance in the shortest ammount of time (or employing the least ammount of computational resources) becomes of pivotal impartance.

Here we aim to compare a set of three optimization algorithms provided by the library Keras Tuner, namely:

• Random Search
• Bayesian Optimization using Gaussian Processes
• Hypberband

The three algorithms are tasked to find the best hyperparameters of a Multilayer Perceptron (MLP) which is used for classifiying digits from 4 variations of the MNIST dataset.

We will compare the three algorithms on 3 main metrics:

1. Score achieved by the MLP using the proposed configuration.
2. Time required to complete the optimization process.
3. Number of hyperparameters configurations explored.

Vanilla MNIST

Back MNIST

Rotated MNIST

RotBack MNIST

1. Download your local version of the repository
2. Install Anaconda
3. Open the Anaconda Powershell Prompt in the repository directory:

# create anaconda environment
conda create -n tuner_bench_env tensorflow-gpu

# activate the environment
conda activate tuner_bench_env

4. At this point install all the requirements with:

# install the requirements
conda install -c conda-forge --file requirements.txt