shubhampachori12110095 / dsd Goto Github PK

This repo contains pre-trained models by Dense-Sparse-Dense(DSD) training on Imagenet.

Compared to conventional training method, dense→sparse→dense (DSD) training yielded higher accuracy with same model architecture.

Sparsity is a powerful form of regularization. Our intuition is that, once the network arrives at a local minimum given the sparsity constraint, relaxing the constraint gives the network more freedom to escape the saddle point and arrive at a higher-accuracy local minimum.

Feel free to use the better-accuracy DSD models to help your research. If you find DSD traing useful, please cite the following paper:

@article{han2016_DSD,
  title={DSD: Dense-Sparse-Dense Training for Deep Neural Networks},
  author={Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Enhao Gong, Shijian Tang, Erich Elsen, Peter Vajda, Manohar Paluri, John Tran, Bryan Catanzaro, William J. Dally},
  journal={International Conference on Learning Representations (ICLR)},
  year={2017}
}

AlexNet_DSD

VGG16_DSD

GoogleNet_DSD

SqueezeNet_DSD

ResNet18_DSD

ResNet50_DSD

The beseline of AlexNet, VGG16, GoogleNet, SqueezeNet are from Caffe Model Zoo. The baseline of ResNet18, ResNet50 are from fb.resnet.torch commit 500b698.