​GraphAttack is a simple computational graph library for machine learning in pure Python (you can think of simple tensorflow). The main point is to combine mathematical operation together to form a workflow of choice in the form of a graph. The graph then takes care of pushing the data forward and propagating the gradients backwards.

I have aimed for the library to be simple and transparent so that it would be easy to understand and modify to fit individual needs. The performance was not the primary objective as there are plenty of fast alternatives; the aim was smaller, educational models.

All feedback and ideas for improvement welcome.

• To be found in tutorial.ipynb, covers basic usage and a simple linear regression model

Setting up and running / training a dense neural network model

• controlDense.py
• controlTrainDense.py

Setting up and running / training a Convolution neural network model

• controlCNN.py
• controlTrainCNN.py

Setting up and running / training a Recurrent neural network

• controlRNN.py
• controlTrainRNN.py

Matrix operations

• Dot product
• Hadamard product
• element-wise division
• addition

Regularization

• Dropout
• Batch Normalisation

Activations

• ReLU
• Sigmoid
• Tanh
• Softmax

Convolution

• 2d convolution
• Max Pooling

Cost operations

• Quadratic cost
• Cross-entropy for softmax activation

Optimisation

• Adam SGD minimizer, support for batches and continuous series

Misc

• Reshape / flatten
• Slice
• Sum all elements/axis
• Sum all elements squared
• element-wise exponent

Limited GPU support can be made avaliable by modifying the file: graphAttack/gaUtilities/graphAttackFunctions.py

Commenting out lines 1-2 and uncommenting lines 4-22 will send the dot product computations to your GPU

• pyCUDA for GPU computations
• scikit-cuda for easy access to cuBLAS

http://www.deeplearningbook.org/ section 6.5.1 for more information on computational graphs and the rest of the book for more details about ML/deep learning.

• Python 3.5 or above
• numpy - linear algebra for Python
• scipy - Scientific Python library, here used for utilities

MIT