low rank deep neural networks about caffe HOT 7 OPEN

Yes.
But I didn't find anything new in the sfm branch. Would you mind offering some documents about this branch? Thanks

from caffe.

@bachml We have a simple tutorial on usage of this code. We will update more details recently.

from caffe.

@wenwei202 thanks to your remarkable work.
But there's a problem comes up. There's no speedups was observed when a convolution layer with rank M = 1 (high layer in ResNet) was decomposed. Also I didn't find any experiment about speedups of ResNet by force regularization.
Did you meet this issue?

from caffe.

@bachml I did not measure speedup by ResNet. Decomposing to rank 1 should have some benefits. It is the issue of the implementation?

from caffe.

@wenwei202 More test in my baseline case(a 27 layers ResNet) shows that the issue is related to multi-threaded blas performance (Caffe with CPU).
It did has 1.20x speedup, which is not significant, with 8 threads openblas backend. When run with single-threaded openblas, it has 2.00x speedup. I guess it's because rank 1 convolution corresponds small matrix multiplication, which do not have much benefits from multi-threaded operation with im2col trick.

from caffe.

@bachml In the rank one case, the conv layer is decomposed to a conv layer with only one filter plus a linear combination layer which essentially is a conv layer with kernels of 1x1. Some code optimization may be required to fully exploit this kind of compactness.

from caffe.

In case you still have interest in this research topic, the details are covered in the paper which is just accepted by ICCV 2017.

from caffe.