This repo contains lecture slides for Deeplearning book. This project is maintained by InfoLab @ DGIST (Large-scale Deep Learning Team), and have been made for InfoSeminar. It is freely available only if the source is marked.
The slides contain additional materials which have not detailed in the book.
Also, some materials in the book have been omitted.
InfoLab @ DGIST(Daegu Gyeongbuk Institute of Science & Technology)
(Advisor: Prof. Min-Soo Kim )
This repo covers Chapter 5 to 20 in the book.
Name | Chapters |
---|---|
Jinwook Kim | 5-1, 8, 16, 17 |
Heechul Lim | 9-2, 11, 12, 15, 19 |
Hyun-Lim Yang | 5-2, 7-2, 10-2 |
Keonwoo Noh | 7-1, 9-1, 14, 20 |
Eunjeong Yi | 6, 10-1, 13, 18 |
- Learning algorithms
- Capacity, overfitting and underfitting
- Hyperparameters and validation sets
- Estimators, bias and variance
- Maximum likelihood estimation
- Bayesian statistics
- Supervised learning algorithms
- Unsupervised learning algorithms
- Stochastic gradient descent
- Example: Learning XOR
- Gradient-Based Learning
- Hidden Units
- Architecture Design
- Back-Propagation and Other Differentiation
- Parameter Norm Penalties
- Norm Penalties as Constrained Optimization
- Regularization and Under-Constrained Problems
- Dataset Augmentation
- Noise Robustness
- Semi-Supervised Learning
- Multitask Learning
- Early stopping
- Parameter Tying and Parameter Sharing
- Bagging and Other Ensemble Methods
- Dropout
- Adversarial Training
- How Learning Differs from Pure Optimization
- Challenges in Neural Networks
- Basic Algorithms
- Algorithms with Adaptive Learning Rates
- Parameter Initialization Strategies
- Approximate Second-order Methods
- Optimization Strategies and Meta-algorithms
- The Convolution Operation
- Motivation
- Pooling
- Convolution and Pooling as an Infinitely Strong Prior
- Variants of the Basic Convolution Function
- Structured Outputs
- Data Types
- Efficient Convolution Algorithms
- Random or Unsupervised Features
- The Neuroscientific Basis for Convolutional Networks
- Unfolding Computational Graphs
- Recurrent Neural Networks
- Bidirectional RNNs
- Encoder-Decoder Sequence-to-Sequence Architectures
- Deep Recurrent Networks
- Recursive Neural Networks
- The challenge of Long-term
- Echo State Networks
- Leaky Units and Other strategies for Multiple Time Scales
- The Long Short-Term Memory and Other Gated RNNs
- Optimization for Long-Term Dependencies
- Explicit Memory
- Performance Metrics
- Default Baseline Models
- Determining Whether to Gather More Data
- Selecting Hyperparameters
- Debugging Strategies
- Computer Vision
- Probabilistic PCA and Factor Analysis
- Independent Component Analysis
- Sparse Coding
- Introduction
- Stochastic Encoders and Decoders
- Regularized autoencoders
- Representational Power, Layer Size and Depth
- Unsupervised pre-training
- Introduction of supervised(SL) and unsupervised learning(UL)
- Representation
- Clustering
- K-means
- Gaussian Mixture Model
- EM algorithm
- Practical example
- The Challenge of Unstructured Modeling
- Using Graphs to Describe Model Structure
- Sampling from Graphical Models
- The Deep Learning Approach to Structured Probabilistic Models
- Sampling and Monte Carlo Methods
- Markov Chain Monte Carlo Methods
- Gibbs Sampling
- The Log-Likelihood Gradient
- Stochastic Maximum Likelihood and Contrastive Divergence
- Estimating the Partition Function
- Approximation
- Maximum Likelihood(MLE) and Maximum A Posteriori(MAP)
- Inference
- Taxonomy of deep generative models
- KL-Divergence
- Variational Inference
- Generative models
- Boltzmann Machines
- Restricted Boltzmann Machines
- Deep Belief Networks
License: CC-BY