This is a approach to understand the Facial Keypoint Detection problem explained in Kaggle using torch7/lua
Reference: - https://www.kaggle.com/c/facial-keypoints-detection - http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/
Input: - Train: - 30 features for facial key points or 15 pairs of (x,y) position - 7049 images - Test: - 1783 images - Each image there are a set of features which has to be calculated and submitted
Understanding Input:
-
Num images with all the 30 feature vectors 2140
-
Num images for each feature: left_eye_center_x 7039 left_eye_center_y 7039 right_eye_center_x 7036 right_eye_center_y 7036 left_eye_inner_corner_x 2271 left_eye_inner_corner_y 2271 left_eye_outer_corner_x 2267 left_eye_outer_corner_y 2267 right_eye_inner_corner_x 2268 right_eye_inner_corner_y 2268 right_eye_outer_corner_x 2268 right_eye_outer_corner_y 2268 left_eyebrow_inner_end_x 2270 left_eyebrow_inner_end_y 2270 left_eyebrow_outer_end_x 2225 left_eyebrow_outer_end_y 2225 right_eyebrow_inner_end_x 2270 right_eyebrow_inner_end_y 2270 right_eyebrow_outer_end_x 2236 right_eyebrow_outer_end_y 2236 nose_tip_x 7049 nose_tip_y 7049 mouth_left_corner_x 2269 mouth_left_corner_y 2269 mouth_right_corner_x 2270 mouth_right_corner_y 2270 mouth_center_top_lip_x 2275 mouth_center_top_lip_y 2275 mouth_center_bottom_lip_x 7016 mouth_center_bottom_lip_y 7016
Model: - Assuming a linear 1D model with 96x96(9216) inputs - Without having non-linear layers inbetween we are actually creating a single layer effectively. So a ReLU is introduced inbetween Linear layers - Since this problem is a Linear Regression problem i.e. the output data can take any values between (0 to 96 or 0 to 1 in normalised form) we use MSE (Mean Square Error) Criterion
Normalising the input: - The input data had to be scaled by 255 to map it between (0,1) - Feature points scaled by 96 (image width) to map it between 0,1. - Any Feature which is not present is currently initialised with -1 - the idea is to map them back to what the network generates thus making the error zero during the criterion:forward pass - this had to be done since the forward passes due to 96x96x100 + 100*30 weights scaled addition results in NaN
initialising the weights: - The default way of initialisng the weights is given by - stdv is the 1.0/sqrt(Num Weights in the layer) - self.weight:uniform(-stdv, stdv) - self.bias:uniform(-stdv, stdv) - One good reference for initialisation: http://jmlr.org/proceedings/papers/v9/glorot10a/glorot10a.pdf - Could try other modules like https://github.com/e-lab/torch-toolbox/tree/master/Weight-init ??
Training: - Splitting the train data of 2140 into 80-20 for learning and testing - Current algorithm used for training gradient descent is Nesterov’s Accelerated Gradient Descent (NAG) and the optimisation parameters used are - sgd_params = { learningRate = .01, learningRateDecay = .001, weightDecay = 0, momentum = .9 }
Results: - With the 80-20 split the current validation to train loss ratio is around 63.1607208073 i.e. training loss is 0.00087948051525779 and the validation/test set loss is 0.055548623279652. - The issue was a coding error. Train loss was in MSE while current loss was norm - 247 current loss = 0.026864869890844 247current validation loss 0.056043914015213
Points to remember:
-
convolution networks expects input to be a 3d or 4d vector and not just a 2d vector so add view as 1x96x96
-
add model supporting convolutional network
-
nn.Reshape helps in converting the 2D netwrok to linear network. Size is image width * image width * num features in convolutional layer
-
Converting to cuda:
-
require 'cunn'
-
torch.setdefaulttensortype('torch.FloatTensor')
-
model:cuda()
-
criterion:cuda()
-
input = input:cuda()
-
cuda supports only floats
-
reinstalling torch7:
-
sudo curl -s https://raw.githubusercontent.com/torch/ezinstall/master/clean-old.sh | bash
-
sudo curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-all | bash
-
test.csv should have one extra field at the end for csvigo to work so adding ,Location to file
-
only float is supported in cuda
-
scp -i ../../../../aws_information/aws_torch7_key_pair.pem [email protected]:/home/ubuntu/workspace/kaggle_face_feature_detection/data/test_output.csv .
-
-
ssh -i ../../../../aws_information/aws_torch7_key_pair.pem [email protected]
-
cuda based training of 47 iterations gave a error of 4.09751 decrease from the previous best :(
-
to make the process reproducible set the seed value for
- torch.manualSeed(1)
- model ???
-
Performance numbers:
- i5 ssd hardisk
- mlp = 60ms per train example
- convnet 140ms per train and 10 ms per validation
- gpu running:
- 1 current loss = 0.0044833544608836 ==> time to learn 1 sample = 57.466043768642 ms 1 current validation loss 0.029351259363066 ==> time to validate 1 sample = 5.8282541337414 ms 2 current loss = 0.0029634770826176 ==> time to learn 1 sample = 57.538097567647 ms 2 current validation loss 0.026195562610115 ==> time to validate 1 sample = 5.8021699992296 ms
- cuda optimised
- 1 current loss = 0.011088124601783 ==> time to learn 1 sample = 6.5466875784865 ms 1 current validation loss 0.033954345414074 ==> time to validate 1 sample = 0.40231422286167 ms
- i5 ssd hardisk
-
Garbage collection
- http://luatut.com/collectgarbage.html
- check in which iteration it prints out of memory and based on that do garbage collection
- if nidx % 50 == 0 then collectgarbage() end
- http://luatut.com/collectgarbage.html
-
cuda output [https://github.com/torch/cutorch]:
- cutorch.getDeviceCount() = 1, cutorch.getDevice() = 1
- cutorch.getMemoryUsage(1) = (3687022592 4294770688) (freeMemory, totalMemory)
- cutorch.seed([devID])
- cutorch.manualSeed(seed [, device])
- cutorch.getRNGState([device]) - RNG RandomNumberGenerator
-
https://groups.google.com/forum/#!topic/torch7/Id04ETdGQHU
- model:training() - if DropOut is used then only during training we need to drop but this should not happen during validation or test
- model:evaluate() - is to be called before validation/test
-
136 current loss = 0.0036384740181395 136 current validation loss 0.030288503368102 after kaggle submission 4.29478
-
182 current loss = 0.028159003744146 182 current validation loss 0.040512910721829 4.38013
-
1291 current loss = 0.022439730285428 1291 current validation loss 0.035369344458706 Your submission scored 3.92498
-
tanh seems to reduce the validation error consistently while reLu does not seem to reduce the validation error
Questions to be answered:
- storing and restoring models
- using batch mode for updation instead of sgd
- using random image index to access
- using all the input images instead of only images which have all the 30 features
- using data agumentation i.e. increasing the number of data inputs by doing image horizontal flips and vertical flips