What are the use of validation set and testing set about matconvnet HOT 7 CLOSED

you could also check the following references for more tips:

http://research.microsoft.com/pubs/192769/tricks-2012.pdf (gradient optimization tips)

http://www.springer.com/computer/theoretical+computer+science/book/978-3-642-35288-1 (if you don't have access to the book, you can find the individually the papers)

from matconvnet.

Hi,

what is a common practice is that on the validation set you tune hyperparameters - which is in case of CNNs the structure of the network itself. Because you are effectively optimising these parameters 'by hand' it may be prone to overfit on the validation dataset.

That is why most of the challenges do have seperate test set where the ground truth labels are not known and which is usually done on a remote server of the organiser. In this way, the organiser is able to limit the number of evaluation on the test set preventing any further over-fitting (in compute vision this is the case of Pascal VOC and ILSVRC datasets).

And if you want to prevent overfitting on the validation dataset, of course you can do any sort of cross validation. It's just that you usually do not have access to test labels...

There is also a lot of other resources on the internets describing this, e.g. this question...

from matconvnet.

Great. Sorry, being dumb. My question should have been avoided. Thanks

But, one questions, what parameters can I tune by validation set? Like learning rate, hidden layer etc?

from matconvnet.

Why sorry, I had no clue about this as well a year ago, so I'm quite happy to help :) But what I found is that google usually knows the most in the end...
Yeah, exactly what you say are the hyperparameters which people usually tune... Hyperparameters as those are parameters influencing the training and the model architecture (number of hidden layers, number of hidden units, size of the convolution kernels and their stride, max pooling sizes, ... it's gazilion of them in the end...).

from matconvnet.

Great! Thanks a lot. I am refreshing memory and learn a lot too. :)

from matconvnet.

thanks!

from matconvnet.

Thanks for the tips too! Haven't heard about the "Neural Networks: Tricks of the Trade" book before...

from matconvnet.