• This page
• Command line usage
• Neural Net API
• Python wrapper
• Q-learning
• To build
• Testing
• Architecture
• Changes

Table of Contents generated with DocToc

• DeepCL
• To use the pre-built binaries
  • What if it doesn't run?
• What if I need a new feature?
• What if I want to contribute myself?
  • Development technical details
  • Architecture
• Third-party libraries
• Related projects
• License

OpenCL library to train deep convolutional networks

• C++
• OpenCL
• Deep convolutional
• (New!) includes Q-learning module (draft)
• (New!) Python wrappers available (draft too :-) )

Functionalities:

• convolutional layers
• max-pooling
• normalization layer
• random translations, as in Flexible, High Performance Convolutional Neural Networks for Image Classification
• random patches, as in ImageNet Classification with Deep Convolutional Networks
• multinet, ie Multi-column deep convolutional network, McDnn
• simple command-line network specification, as per notation in Multi-column Deep Neural Networks for Image Classification
• pad-zeros possible for convolutional layer
• various activation functions available:
  • tanh
  • scaled tanh (1.7519 * tanh(2/3x) )
  • linear
  • sigmoid
  • relu
  • softmax
• fully-connected layers
• various loss layers available:
  • square loss
  • cross-entropy
  • multinomial cross-entropy (synonymous with multinomial logistic, etc)
• Q-learning

Example usage:

• intend to target 19 x 19 Go boards, eg something similar to Clark and Storkey or Maddison, Huang, Sutskever and Silver
  • obtained 36.3% test accuracy, on next move prediction task, using 33.6 million training examples from kgsgo v2 dataset
  • commandline used ./deepclrun dataset=kgsgoall netdef=32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-32c5{z}-500n-361n numepochs=3 learningrate=0.0001
  • 3 epochs, 1.5 days per epoch, on an Amazon GPU instance, comprising half an NVidia GRID K520 GPU (about half as powerful as a GTX780)
• obtained 99.5% test accuracy on MNIST, using netdef=rt2-8c5{padzeros}-mp2-16c5{padzeros}-mp3-150n-10n numepochs=20 multinet=6 learningrate=0.002
  • epoch time 99.8 seconds, using an Amazon GPU instance, ie half an NVidia GRID K520 GPU (since we are learning 6 nets in parallel, so 16.6seconds per epoch per net)

For Python wrappers, please see python/README.md

Pre-built binaries are available for Windows, for certain releases. In order to use them you need:

• An OpenCL driver for your GPU
• A recent release with Windows binaries is v3.2.1

• Check if you have an OpenCL-enabled device on your system
  • ideally a GPU, or accelerator, since there is no attempt to optimize DeepCL for CPUs (at least, not currently, could change, feel free to submit a pull request :-) )
• Try running gpuinfo (from OpenCLHelper, but built as part of this project too, for ease of use )
  • it should output at least one OpenCL-enabled device
  • if it doesn't, then you need to make sure you have an OpenCL-enabled device, and that appropriate drivers are installed, and that the ICD is configured appropriately (registry in Windows, and /etc/OpenCL/vendors in linux)

Please raise an issue, let me know you're interested.

• If it's on my list of things I was going to do sooner or later anyway (see below), I might do it sooner rather than later.
• If it's to do with usability, I will try to make that a priority

• please feel free to fork this repository, tweak things, send a pull request

• cogapp generator is used extensively, to accelerate development, reduce the number of manual copy-and-pasting and so on. Specifically, it's used for:
  • generating header declarations from .cpp definition files
  • generating fluent-style argument classes for certain tests
  • ... and more uses will surely be found :-)
• You need Python installed and available for this to work. You don't need python just to build the sources, but if you do have python installed, and you flip the PYTHON_AVAILABLE switch in the cmake configuration, then a lot of manual editing will no longer be necessary :-)

• OpenCLHelper
• clew
• libpng++

• kgsgo-dataset-preprocessor Dataset based on kgsgo games; 33 million data points

Mozilla Public License 2.0