Seminars for CV course based on cs231n at ABBYY, 2019

Seminar 1 (backprop in numpy)

• learn how backpropagation really works
• implement backward pass for main layers (Linear, ReLU, Sigmoid, Softmax)
• implement backward pass for most common criterion (Crossentropy)
• implement batch stochastic gradient descent
• train your tiny neural network on MNIST

Exercises

• why sigmoid/tanh activations may be bad for deep networks?
• can you train classification problem with MSE loss? Why this may not be a good idea?
• "dead ReLU" problem - provide example when it may happen
• derive analytically (on paper) gradients w.r.t. all inputs and parameters for small network (Linear -> ReLU -> Linear -> Softmax) with NLL loss

Homework 1

• backward for many other layers (Conv, Pool, BatchNorm, Dropout, etc)

Seminar 2 (training your network with PyTorch)

• PyTorch in 5 minutes
• training your neural net in PyTorch
• training on GPU
• reproducibility
• explore different activations and loss functions

Seminar 3 (layers & architectures)

• Most confusing layers: Dropout and BatchNorm
• Most common architectures: Alexnet, VGG, Inception, Resnet, Resnext, Densenet, SENet

Seminar 4 (high level frameworks)

• PyTorch ecosystem
• augmentations
• fasi.ai vs pytorch lightning vs catalyst
• catalyst

Seminar 5 (segmentation)

• semantic segmentation with catalyst config api (based on segmentation tutorial)

Seminar 6 (detection)

• understanding detection with your hands
• detectron2 - handy detection framework, colab notebook

Seminar 7 (visualization)

• attribution with TorchRay