sdemyanov / convnet Goto Github PK

Hi, thank you very much for sharing the project. I plan to conduct experiments on it, but i am wondering whether it supports local convolutional layer or not?

Hi! Congratulations on this great work, and thanks for sharing.

I'm trying to use the weights generated by the training procedure but I could not figure out how they are organized inside that huge column vector. By inspecting the "getweights.m" file I saw that the weights comes first, followed by the biases; but are the convolutional weights in the row-major or in the column-major layout (I'm supposing that the filter layers are disposed sequentially in the vector)? And in the feedforward layers, are the weights from one given neuron consecutive in the vector?

Thanks a lot!

Hi.
I'm using the ConvNet library with Matlab on medical images (2 classes: positive or negative). In the training phase I obtained the same good results that I have obtained previously with the DeepLearnToolbox but when I test the net on new images loading the weights saved during training something wrong happen: the images are classified all positive or all negative. With the DeepLearnToolbox instead, using the same test images, I have some images classified positive and some negative. What can I do?

Hi,

Forgive me if am asking a question that is too simple (am new to deep learning).

Please, which mat file should one edit if he wants to train his own classifier.

Is it cnnexamples.m or train_mnist.m

Furthermore, I was just wondering for face recognition are 100 examples per class sufficient for the training?

Thanks

I'm using VS2010, win7 x64,CUDA SDK 6.5 .

I change version of Windows SDK to 7.0A in mex_CUDA_win64.xml и vcvars64.bat

because of error :

compile(2)
Building with 'Microsoft Visual C++ 2010'.
MEX completed successfully.
C++ object files created
Verbose mode is on.
... Looking for compiler 'NVIDIA CUDA Compiler' ...
... Looking for environment variable 'VS100COMNTOOLS' ...Yes ('C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools').
... Looking for file 'C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\cl.exe' ...Yes.
... Looking for folder 'C:\Program Files (x86)\Microsoft Visual Studio 10.0' ...Yes.
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0' InstallationFolder ...No.
Did not find installed compiler 'NVIDIA CUDA Compiler'.
Error using mex
No supported compiler or SDK was found. For options, visit
http://www.mathworks.com/support/compilers/R2014a/win64.

Error in compile (line 115)
mex(mexfile, ['"' cudafiles '"'], ['"' cppfiles '"'], ...

and I get error in filter_acts.cu

Error using mex
filter_acts.cu

G:/Matlab/ConvNet-master/c++/cuda/sources/filter_acts.cu(1201): error: more than one
instance of overloaded function "sqrt" matches the argument list:
function "sqrt(double)"
function "sqrt(float)"
function "sqrt(long double)"
argument types are: (int)
1 error detected in the compilation of
"C:/Users/user/AppData/Local/Temp/tmpxft_00000f28_00000000-11_filter_acts.compute_30.cpp1.ii".

Error in compile (line 115)
mex(mexfile, ['"' cudafiles '"'], ['"' cppfiles '"'], ...

I changed line 1201
int filterSize = (int) sqrt(filterPixels);
to
int filterSize = (int) sqrt((double)filterPixels);

same in img_acts.cu and weight_acts.cu.
and there are lot's of warnings about type conversion.

Hello,

First of all, thanks a lot for sharing your work, it is really interesting.

About my problem, I'm trying to train a CNN using RGB images as input. My training set has a size of kXSize = 33x33x3x55000. I use a typical architecture I,C,S,C,S,C,S,F,F, no modifications to any hyperparameter. I define the input layer as :
struct('type', 'i', 'mapsize', kXSize(1:2), 'outputmaps', kXSize(3)).
Input matrices are normalized and correctly defined.

Problem is, after the first epoch of training, I keep getting a vector of NaN weights. Would you have any idea why weights are not computed properly? It seems I don't specify the input properly, but I can't find out where, as your program runs 'normally' (it takes about 1500sec to do 1 epoch of training).

Thanks for your attention,

Quentin

Hi,

Firstly, thank you for making your package available for us.
In fact, I'm working on developing iris recognition system based on CNN and trying to use your package for this purpose. However, I have problem with compiling GPU version with script compile.m.
I followed the same instructions that you explained in (COMPILATION FOR WINDOWS) section and I am getting the error:

(Error in compile (line 111) mex(mexfile, ['"' cudafiles '"'], ['"' cppfiles '"'], ... No supported compiler or SDK was found.).

Please, can you help me to sort this problem out.
Your help in this regard is highly appreciated.

Hi When I make changes in ./c++/Makefile and run make its doing nothing?

Thank you very much about your toolbox which helped us a lot. For now, I just do the image segmentation problems. To improve the performance of final results, I need to check the value of the kernel in each layer in the toolbox, when I try to use GPU, the output of the weight is a vector. Could you tell me how to change this weight in to standard structure (like N-by-N matrix)? Thank you very much.

Hi. I'm using ConvNet with medical images. Using the code without SVM in the final layer I had good results. Then I tried to introduce the SVM and the results are very strange. With values of the C constant equal to 1, 10, 100, 1000 I always obtain a training error of 1.4e+4 and a test error of 50%. What can I do? In the cnnbp.m function i found a commented line in which C is present. I have to uncomment it?

Hi!
Is it possible to use multiple gpu in this package?

Hi, i'm using your CNN code. I need some theoretical/matemathical information.
I understood that in the convolutional layers each outputmap is obtained by the sum of the convolutions of the different inputmaps with the respective kernel. Is it right?
I can't find papers/tutorials that explains this passage in a detailed way. Where can i find it? Can you help me?

I tried to run cnnexamples.m(funtype = 'matlab';) and get error

Undefined function 'flip' for input arguments of type 'single'.

Error in flipall (line 3)
X = flip(X, dimind);

Error in filtn (line 3)
c = convn(a, flipall(b), type);

Error in forward (line 38)
layers{l}.a(:, :, i, :) = layers{l}.a(:, :, i, :) + ...

Error in cnntrain_mat (line 44)
[layers, pred] = forward(layers, 1);

Error in cnntrain (line 12)
[weights, trainerr] = cnntrain_mat(layers, weights_in, params, train_x, train_y);

Error in cnnexamples (line 77)
[weights, trainerr] = cnntrain(layers, weights, params, train_x, train_y,
funtype);

how can it be resolved? I'm using Matlab R2012a win7 x64.

Hi!
Now I am trying to adapt your ConvNet to train my database(faces 400 pictures (each 112*92 pix.)) and then I want to do face recognition.

My question is.. is it possible for ConvNet to do face recognition? I know that it's possible to recognize digits... is it possible to apply this network for face recognition? If so how many layers should I use to get good(applicable) results? how long does it usually take to train big database? (I tried to train your network just by changing input images(and target) to my face database... and result was... 95% error... after 10 epoch of training) Is it possible to train so that it could recoginize faces(with error less than 5%)?

Maybe you have tried face recognition with your network?

I am really looking forward to see your answer! Thank you!!

This is an excellent work for grayscale image. Will it support RGB input image?

Hi There,

Thank you very much for sharing the code. I am hoping to use the CNN to classify cartilage in knee mri scans. Unfortunately when running in Matlab it seens the .mexa64 function is not working. Could you please advise to any action I could take to resolve this?

Thank you again

Hello

I only find the .mexa64 file in the build folder, which means this code only works on Linux. Could you provide .mexw64 file? I want to use this toolbox in the windows environment. Thank you very much.

Is it possible to use your CNN implementation for prediction?
If so, please give a simple example.
Ty.

I have some troubles with compiling GPU version with script compile.m

I have cuda supported GPU(gtx760) and have already set up 'CUDA_PATH' and 'VS100COMNTOOLS'
by using 'getenv' and 'setenv'. Also, have Microsoft visual studio 2012 and SDK. However I don't know how to set up the CUDA settings for 'mex' (mex_CUDA_win64.xml). Therefore, I guess I am getting the error: (Error in compile (line 122) mex(mexfile, ['"' cudafiles '"'], ['"' cppfiles '"'], ... No supported compiler or SDK was found.)

How can I configure the CUDA settings (mex_CUDA_win64.xml)?
Or, do I have other problems?

Looking forward to see your reply! TY

Hi Sergey,

thank you a lot for sharing your project.
Did you also tried to compile it under Mac Os X (particularly Yosemite)?
And in your readme file under Linux compilation stays: Copy the settings file "MatlabRoot"/bin/mexopts.sh to ~/.matlab/"MatlabVersion". Should I copy this file in matlab directory in ConvNet and rename it to eg. MATLAB_R2015a without extension?

Thank you in advance,

Oleksandr Vorobiov

HI

I want to use convolution layers to do feature extraction. That means after training, I can get the result from each layers.Could you tell me how to do this work base on gpu function in this code.

Hi,thanks for your work!

I have encountered a strange problem,that in order to express a more complicated

model,I use 2 fully connected layer as a classifier.But the error rate have been greatly

improved from 3.3% to 77%.

What should I do ?

Hi Sergey,

thank you for sharing this code.
Now, I'm trying to train a cnn model from the data which have 6 channels (RGB and some other informations, for example: the depth information). The CPU version and Matlab version can work well, but it fails with the following error when I try to use the CUDA version:

Error using cnntrain_mex
Assertion Failed: fa1

Error in cnntrain(line 10)
[weights, trainerr]=cnntrain_mex(layers,weights_in,params,train_x,train_y);

How can I fix it, THX.

Hi Sergey,

There is new problem in my work.
I found the parameter 'biascoef' may be not work well as described in the ReadME file.

This is my params and structure:

params.batchsize=128;
params.epochs = 1;
params.alpha = 0.1;
params.momentum = 0.9;
params.lossfun = 'logreg';
params.shuffle = 1;
params.seed = 0;
dropout = 0.5;

layers = {
struct('type', 'i', 'mapsize', kXSize(1:2), 'outputmaps', kXSize(3)) %32
struct('type', 'c', 'filtersize', [3 3], 'outputmaps', 32,'function','sigm', 'dropout', dropout, 'biascoef' ,0) %30
struct('type', 's', 'scale', [2 2], 'function', 'max', 'stride', [2 2], 'dropout', dropout) %15
struct('type', 'c', 'filtersize', [2 2], 'outputmaps', 64,'function','sigm', 'dropout', dropout, 'biascoef' ,0) %14
struct('type', 's', 'scale', [2 2], 'function', 'max', 'stride', [2 2], 'dropout',dropout) %7
struct('type', 'c', 'filtersize', [2 2], 'outputmaps', 64,'function','sigm', 'dropout', dropout, 'biascoef' ,0) %6
struct('type', 's', 'scale', [2 2], 'function', 'max', 'stride', [2 2], 'dropout', dropout) %3
struct('type', 'c', 'filtersize', [2 2], 'outputmaps', 128,'function','sigm', 'dropout', dropout, 'biascoef' ,0) %2
struct('type', 'f', 'length', 256, 'function','sigm', 'dropout', dropout, 'biascoef' ,0)
struct('type', 'f', 'length', kOutputs, 'function', 'soft', 'biascoef' ,0)
};

I have set all the ‘biascoef’ of the convolution layers and the full-connection layers to 0, but the loss and test prediction accuracy are still change after each training epoch.

As far as I know, the network weights will not change when all ‘biascoef’ params are set to 0, but it seems inconsistent with the situation here.

Yours,
Xuan Li.

first thank you sharing the code,
I was trying to compile the code, at first I got an error missing "cudnn.h" but I found a solution for it. when I compiled again, I got the following error:
Error using mex
mex_util.cpp
C:\Users\Anes\Desktop\ConvNet-master\c++\sources\mex_util.cpp(175) : error C2057: expected constant expression
C:\Users\Anes\Desktop\ConvNet-master\c++\sources\mex_util.cpp(175) : error C2466: cannot allocate an array of constant size
0
C:\Users\Anes\Desktop\ConvNet-master\c++\sources\mex_util.cpp(175) : error C2133: 'dims' : unknown size

Error in compile (line 52)
mex(['"' cppfiles '"'], '-c', ...

it seems the issue is that C++ does not support size '0' array or something.
can you please advise.
thank you

Hi Sergey,

thank you for sharing this code.

I am trying to run one of your examples ( the cpu version of it), but it fails with the following error:

Error using genweights_mex
Assertion Failed: In 'mexGetScalar' mx_array is not numeric

Error in genweights (line 4)
weights = genweights_mex(layers, params);

Error in cnnexamples (line 82)
weights = single(genweights(layers, params, funtype));

looks like the signature of genweights_mex changed?

Hello,
I just trying to complile your code using CPU, but get the error as follows:

Error using genweights_mex
Assertion Failed: In 'mexGetScalar' mx_array is not numeric

Error in genweights (line 4)
weights = genweights_mex(layers, params);

Error in cnnexamples (line 82)
weights = single(genweights(layers, params, funtype));

Can you help me to resolve this error?