This is taken from https://github.com/meliketoy/wide-residual-network/ and modified to create solution for STAT946 kaggle challenge - Deep Learning and gives 0.83971 on the private leaderboard: https://www.kaggle.com/c/fall2017stat946/leaderboard

NOTE: Before submitting the solution, make sure to remove "true_labels" column from the csv file generated.

All my experiments were run on GeForce GTX 1080.

Wide-residual network implementations for cifar100

Torch Implementation of Sergey Zagoruyko's Wide Residual Networks.

In order to figure out what 'width' & 'height' does on wide-residual networks, several experiments were conducted on different settings of weights and heights. It turns out that increasing the number of filters(increasing width) gave more positive influence to the model than making the model deeper.

Last but not least, simply averaging a few models with different parameter settings showed a significant increase in both top1 and top5 accuracy. The CIFAR dataset test results approached to 84.19% for CIFAR-100 with only meanstd normalization.

See the installation instruction for a step-by-step installation guide. See the server instruction for server setup.

• Install Torch
• Install cuda-8.0
• Install cudnn v5.1
• Install luarocks packages

$ luarocks install cutorch
$ luarocks install xlua
$ luarocks install optnet

• modelState : The best model will be saved in this directory
• datasets : Data preparation & preprocessing directory
• networks : Wide-residual network model structure file directory
• gen : Generated t7 file for each dataset will be saved in this directory
• scripts : Directory where the run file scripts are contained

Adapting weight adjustments for each model will promise a more improved accuracy.

You can see that the ensemble network improves the results of single WRNs.

Test error (%, random flip, meanstd normaliztion, median of 5 runs) on CIFAR:

You can train each dataset of cifar100 by editing and running the script below.

$ ./scripts/train.sh

You can test your own trained model of cifar100 by editing and running the script below. This will print test results and save the .csv file for submission to kaggle

$ ./scripts/test.sh

To train an ensemble follow the steps below.

$ vi ensemble.lua
# Press :32 in vi, which will move your cursor to line 32
ens_depth         = torch.Tensor({28, 28, 28, 28, 40, 40})
ens_widen_factor  = torch.Tensor({10, 20, 20, 20, 10, 14})
ens_nExperiment   = torch.Tensor({ 1,  1,  2,  3,  1,  1})

then edit and run

$ ./scripts/ensemble_train.sh

This will create .csv file for kaggle submission

If you want to just create the ensemble of already trained models, edit ensemble.lua as mentioned above and then edit and run scripts/ensemble_test.sh

$ ./scripts/ensemble_test.sh

• CIFAR-100

Below is the result of the test set accuracy for CIFAR-100 dataset training.

NOTE: 28x20 configuration consumes the highest GPU memory ~11 GB while running on a single GPU, if you get error, it is possibly due to GPU memory insufficiency. Also note, while running script/ensemble_test.sh, you might hit into memory issues, in which case, either reduce number of models in the ensemble by modifying, ensemble.lua or edit test.lua to load models differently.

Accuracy is the average of 5 runs