kyunghyuncho / deepmat Goto Github PK

Hi,

  I am following your master thesis to understand RBM and its variants. I am trying to implement krizhevsky's  Gaussian -bernoulli RBM in matlab. When i do, as you said in your document i am getting sigmas in -ve range. After 10 batch, i am getting error as inf .When i visualize weights , data and reconstructed data, resconstructed data appears like white or black pixels. I could not see any image.

my sigma learning rate=10^-8. U might have come across this kind of issu. Could u able to suggest me?

Hi

It seems there is an issue in how you're choosing to visualize the training of GRBM. I encountered some problems running the above example.

Though the fix/workaround seems easy. I think you haven't fully modified the visualize_grbm.m file yet. A slight change in definition solves the problem.

Errors were:

Error using visualize_grbm
Too many input arguments.

Error in rbm (line 560)
R.debug.display_function (R.debug.display_fid, R, v0, v1, h0, h1, W_grad, vbias_grad,
hbias_grad);

Error in train_rbm (line 49)
R_out = rbm(R, patches);

Error in example_faces (line 46)
R = train_rbm (R, X);

when using adaptive_lrate, it runs the script candidate_lrates where the variable max_iter_up is not defined, so an error occurs

line 23 is: valid_targest = [];
should be: valid_targets = [];
It is only an issue if less than 3 parameters are passed to the function (nargin<3)

Also, I have a question:

In default_mlp.m, why are you using binary input and binary outputs?

% data type
M.data.binary = 1;
%M.data.binary = 0;

% output type
M.output.binary = 1;

Thanks,
Vikrant.

Hey,

Two more things.

1. In example_mnist_mlp_rbm around line 25, you're hardcoding the dimensionality of the output layer. It can be done automatically using:

C=unique(X_labels);
% Layers configuration
layers = [size(X,2), 1000, 500, length(C)];

1. Also, in a related way in mlp.m, the way you're populating the label matrices in not optimal (line 53 onwards). You can do the following:

disp('Now calculating the training label matrix')
C=unique(targets);
% new_targets = strcmp(repmat(targets,1,length(C)),repmat(C',length(targets),1));
new_targets = bsxfun(@eq,repmat(targets,1,length(C)),repmat(C',length(targets),1));
% clear C, label;
disp('done!')
size(new_targets)

Where either of bsxfun and strcmp are used depending on whether the labels are numeral or characters. It should be faster than what you're doing right now, and will work for character labels as well. The same also applies to populating label matrices for validation set.

Best,
Vikrant.

After a couple of hours running example_mnist_mlp.m, it outputs:
Correctly classified test samples: 980/10000

It simply predicts class 1 for all test samples.
I checked it twice and the results are the same. I could not figure out the problem, any help is greatly appreciated.

in line 45, the code should be

subwindow_sum = fspecial('gaussian', C.lcn.neigh);

Hi,
When I run C:\Users\xn-lab\Desktop\deepmat\example\example_mnist_convnet.m.m , there are so many problems:

1. If use n_conv = size(conv_layers, 2);
   I think it should define conv_layers = [5_5, 32; 5_5, 32];
   rather than conv_layers = [5_5, 32, 5_5, 32].
   2.In convnet_avgpool.m, so many parameters' problems.
   3.in convnet.m,
   dconv = dconv .* h0_conv{l-1, 4};
   dimension doesn't match, is 28_28 ._ 10*10.

Hi,
When I run this example, the call to predict on line 60 returns null values for [L accuracy] and I get the exit_with_help message from the predict function. The only change I made was to use liblinear-1.94 instead of liblinear-1.7.

The model that was trained in line 55 is:

model =
Parameters: 0
nr_class: 10
nr_feature: 500
bias: -1
Label: [10x1 double]
w: [10x500 double]