This repo was inspired by Paper Bag of Tricks for Image Classification with Convolutional Neural Networks
trick | referenced paper |
---|---|
xavier init | Understanding the difficulty of training deep feedforward neural networks |
Kamming init | Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification |
warmup training | Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour |
no bias decay | Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes |
label smoothing | Rethinking the inception architecture for computer vision) |
random erasing | Random Erasing Data Augmentation |
cutout | Improved Regularization of Convolutional Neural Networks with Cutout |
linear scaling learning rate | Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour |
cosine learning rate decay | SGDR: Stochastic Gradient Descent with Warm Restarts |
and more to come......