Resources and Links to Deep Learning Tutorial

1. Andrew Ng's Deep Learning Course on Coursera ---- for general understanding, good for beginners https://www.deeplearning.ai/

   4-week course; $49 per month

• Course Structure -
• Week 1: Introduction to Deep Learning
• Week 2: Neutral Network Basics
• Week 3: Shallow Neutral Networks
• Week 4: Deep Neutral Networks

2. Google - Udacity https://www.udacity.com/course/deep-learning--ud730

   3-month self-paced; free

• Course Structure -
• Lesson 1: From Machine Learning to Deep Learning
• Lesson 2: Deep Neutral Networks
• Lesson 3: Convolutional Neutral Networks
• Lesson 4: Deep Models for Text and Sequences

3. Geoffrey Hinton's Neural Networks for Machine Learning on Cousera ---- goes deeper, might not be easy to understand for beginners https://www.coursera.org/learn/neural-networks

   16-week course; $49 per month

• Course Structure -
• Week 1: Introduction
• Week 2: The Perceptron learning procedure
• Week 3: The backpropagation learning proccedure
• Week 4: Learning feature vectors for words
• Week 5: Object recognition with neural nets
• Week 6: Optimization: How to make the learning go faster
• Week 7: Recurrent neural networks
• Week 8: More recurrent neural networks
• Week 9: Ways to make neural networks generalize better
• Week 10: Combining multiple neural networks to improve generalization
• Week 11: Hopfield nets and Boltzmann machines
• Week 12: Restricted Boltzmann machines (RBMs)
• Week 13: Stacking RBMs to make Deep Belief Nets
• Week 14: Deep neural nets with generative pre-training
• Week 15: Modeling hierarchical structure with neural nets
• Week 16: Recent applications of deep neural nets

4. Stanford CS231n Spring 2017 Convolutional Neutral Networks for Visual Recognition

课程视频地址：https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv Bilibili视频地址：http://www.bilibili.com/video/av13260183/#page=1

Free 16-lecture course

• Course Structure -
• Lecture 1: Introduction to Convolutional Neutral Networks for Visual Recognition
• Lecture 2: Image Classification 包括数据驱动（data-driven）方法，K 近邻方法（KNN）和线性分类（linear classification）方法
• Lecture 3: Loss Function and Optimization 1> 继续上一讲的内容介绍了线性分类方法； 2> 介绍了高阶表征及图像的特点； 3> 优化及随机梯度下降（SGD）。
• Lecture 4: Introduction to Neutral Networks 包括经典的反向传播算法（back-propagation）；多层感知机结构（multilayer perceptrons）；以及神经元视角。
• Lecture 5: Convolutional Neutral Networks 1. 卷积神经网络的历史背景及发展； 2. 卷积与池化（convolution and pooling）; 3. ConvNets 的效果
• Lecture 6: Training Neutral Networks I 介绍了各类激活函数，数据预处理，权重初始化，分批归一化（batch normalization）以及超参优化（hyper-parameter optimization）。
• Lecture 7: Training Neutral Networks II 介绍了优化方法（optimization）、模型集成（model ensembles）、正则化（regularization）、数据扩张（data-augmentation）和迁移学习 （transfer learning）。
• Lecture 8: Deep Learning Software 1. 详细对比了 CPU 和 GPU； 2. TensorFlow、Theano、PyTorch、Torch、Caffe 实例的具体说明； 3. 各类框架的对比及用途分析。
• Lecture 9: CNN Architectures 该课程从 LeNet-5 开始到 AlexNet、VGG、GoogLeNet、ResNet 等由理论到实例详细描述了卷积神经网络的架构与原理。
• Lecture 10: Recurrent Neutral Networks 该课程先详细介绍了 RNN、LSTM 和 GRU 的架构与原理，再从语言建模、图像描述、视觉问答系统等对这些模型进行进一步的描述。
• Lecture 11: Detection and Segmentation 该课程在图像分类的基础上介绍了其他的计算机视觉任务，如语义分割、目标检测和实例分割等，同时还详细介绍了其它如 R-CNN、Fast R-CNN、Mask R- CNN 等架构。
• Lecture 12: Visualizing and Understanding 该部分不仅讲述了特征可视化和转置，同时还描述了对抗性样本和像 DeepDream 那样的风格迁移系统。
• Lecture 13: Generative Models 该章节从 PixelRNN 和 PixelCNN 开始，再到变分自编码器和生成对抗网络详细地讲解了生成模型。
• Lecture 14: Reinforcement Learning 该章节先从基本概念解释了什么是强化学习，再解释了马尔可夫决策过程如何形式化强化学习的基本概念。最后对 Q 学习和策略梯度进行了详细的刻画，包 括架构、优化策略和训练方案等等。
• Lecture 15: Efficient Methods and Hardware for Deep Learning 该章节首先展示了深度学习的三大挑战：即模型规模、训练速度和能源效率。而解决方案可以通过联合设计算法-硬件以提高深度学习效率，构建更高效的推 断算法等.
• Lecture 16: Adversarial Examples and Adversarial Training 该章节由 Ian Goodfellow 于 5 月 30 日主讲，主要从什么事对抗性样本、对抗性样本产生的原因、如何将对抗性样本应用到企业机器学习系统中、及对 抗性样本会如何提升机器学习的性能等方面详细描述对抗性样本和对抗性训练。

5. Stanford CS224n ---- Natural Language Processing with Deep Learning http://web.stanford.edu/class/cs224n/

6. Oxford - DeepMind ---- Deep Learning for Natural Language Processing (2016-2017) https://www.cs.ox.ac.uk/teaching/courses/2016-2017/dl/

   Course Materials: https://github.com/wxlu718/lectures

7. Fast.ai - Making Neutral Nets Uncool Again http://www.fast.ai/

   Free course

8. Tensorflow提供的机器学习教程，分为两篇

初学者篇：https://www.tensorflow.org/get_started/mnist/beginners 进阶篇：https://www.tensorflow.org/get_started/mnist/pros

9. <> by Yoshua Bengio and Ian Goodfellow Online reading only: http://www.deeplearningbook.org/