skkeem / mnist Goto Github PK

Google docs로 작성했는데, 역시 붙이기 하니 깨지는구나.
내가 한 부분을 나름대로 정리해봤어.
대략 이걸로 두페이지 정도 될 것 같은데, 비슷하게 네가 한 부분도 만들면 되지 않을까 한다.
내 노트북에 PPT가 없는데, 혹시 편집은 너한테 부탁해도 될까?^^;;;

• Presentation Contents
• Used Model : Support Vector Machine with Linear, Polynomial, and Gaussian Kernels
• Main Issues
  1. Multiclass SVM
     • Basically, SVM is a binary classifier. How can SVM work on multiclasses?
     • Created 10 SVMs which classifies for each digit 0 to 9 and selected the classification result with the largest positive margin. (OVA(one-versus-all) classification)
  2. Lagrangian dual problem
     • How can I get the solution of the optimization problem of SVM's quadratic problem?
     • Used the publicly available Python library called "cvxopt"(http://cvxopt.org).
  3. Feature selection
     • How should I choose the feature set as input to the classifier?
     • As a brute-force way, used the original pixel data as the feature. The input data dimension is 784(28*28) and the each value has been normalized to a range [0, 1).
• Result & Comments
  1. The error rate is about 5.58%. (when the train data size is 5000)
  2. Gaussian kernel shows the best performance among the three implemented kernels.
  3. Clever selection of C is critical for the prediction performance. Sometimes, soft margin case shows poor performance than hard margin case.
  4. For faster execution time, feature size reduction is necessary. Through PCA of the original pixel data, coupled with explicitly designed feature sets, the input data dimension could be reduced less than 100 with little impact on prediction power.

SVM으로 하려다보니, 결국 Lagrangian dual problem의 해를 찾는 부분 구현이 핵심인 것 같아서,
이것저것 뒤져보다가 발견한 것...
CNN과 MLP 관련 부분이 포함된 것 같다. Theano 기반으로...

https://groups.google.com/forum/#!topic/theano-users/on4D16jqRX8

냉무