jrios6 / adam-vs-sgd-numpy Goto Github PK

A 3-layer neural network with SGD and Adam optimizers built with numpy.

This is a response to Siraj Raval's Coding Challenge to implement the Adam Optimization Strategy. In this notebook, we are building a 3-layer neural network with numpy for the Kaggle Titanic Dataset, and comparing the performance difference between a standard Stochastic Gradient Descent and Adam.

• numpy
• pandas
• matplotlib

Run jupyter notebook in your Python 3 conda environment

1. Adam: A method for Stochastic Optimization by Diederik P. Kingma, Jimmy Ba
2. CS231: Neural Networks by Andrej Karpathy
3. Optimizing Gradient Descent by Sebastian Ruder