scut-ailab / dcp Goto Github PK

Please add a license.

Here is going to copy modules of pruned_modules, why don't you just do

self.ori_model = copy.deepcopy(self.pruned_model)

,but copy modules by modules?
Is there any concern?

I think the additional discrimination-aware loss in this project is suitable for classification task but not detection task ，the classification task focus on the whole feature map , therefore averagepooling+softmax works , but the detection task focus on feature in receptive field , if I want to use this method on yolo-v3 , how can I modify the discrimination-aware loss ?

RuntimeError: one of the variables needed for gradient conputation has been modified by an inplace operation.
When I use torch.autograd.set_detect_anomaly(True) to find the operation that failed to compute its gradient, it shows "x = self.conv(2) " in dcp/models/preresnet.py. I can't find the solution, Could anyone help me?

In Algorithm 1, "Learn theta and Fine-tune M with (L^S)_p and L_f." which means that the weights of the first layer to layer L_p will be updated w.r.t to (L^S)_p and whole model weight will be updated w.r.t L_f (the final loss), is that right?

In this line, I can understand that i != index is going to update weights of the first layer to layer L_p by self.seg_optimizer[i].step() but the i != len(self.pruned_segments) - 1 is going to update weights of the first layer to the end of the second-last segment.

According to Algorithm 1, it should be the last segment, that is, the whole model, but why the second-last segment?

I'm trying to reproduce your experimental results on ImageNet, however I find that training resnet networks really takes super long time !!! So I want to know how much time did you use to train resnet on ImageNet dataset??? Thanks !!!

Thanks for your paper and sharing code. I have some confusions about code, can you explain it for me.

1. In function channel_selection()， why the original model is replaced by pruned model? Is it since the loaded pre-trained model without fine-tuning.
2. In function update_model, why the network is splited repeatly(the network will be split a time if the update_model() be called)？But the segment seems same after each split,
3. Is it original net just be a baselone. Why origin net is continualy updated(line 503 in channel_pruning.py) if it is just be a baseline ?
   Thank you very much.
   Expect for yor reply.

Hi,

Great work! You released the structures of ResNet nets in models.py except ResNet56, could you please make the definition of ResNet56 available as well? I'm aware of many online versions available, but I just would like to figure out some details, like whether bias=True or not in conv layers etc.. Thanks.

when i donwload your pruned model(imagenet-resnet50-0.5), then test it with test.py.
the result is as below:
|===>Validation Error: 28.2700 Loss: 1.1303, Top5 Error: 9.4360
but in your paper the result is lower:
top1 error:25.05 top5 error:7.68

why? is there some difference?

I like your feature maps shown in the paper very much. The color matching is so good. Do you draw the feature maps with imshow function in matplotlib? Which colormap do you choose? Or other methods to show the beautiful feature maps? I try to show feature maps but they look ugly. Are there any advice? Thanks you very much!

learning rate, pruning ratio, e.g.

Hi:
release the code?

how do you prune the LResNet34E-IR model,and where can i get the "LResNet34E-IR (prune 50% channels) " model?

Hello, what a good work you did!
I have the problem that when I reproduce the work, I find that the self.pruned_weight (in line 25 of /dcp/mask_conv.py) is not loaded from the original module (It may be generated from lib of nn.Conv2d? because the MaskConv2d derives from nn.Conv2d).
The weight of original layer:

The weight of self.pruned_weight:

Why the weight is not loaded from the original module?
Thanks a lot.

Firstly, congratulations on the excellent performance of your proposed model on specific data sets and some pre-training models.
As the title shows, I need to tailor my own pre-trained model which datasets are related to audio rather than images.And the pre-trained model includes not only CNN but also RNN.
Therefore, I would like to ask you if I can use your given model in my case.If not, do you have other ideas for model pruning?
Thanks a lot
Best wishes
GUOhm230

Could you, please, share a link to fine-tuned pruned models (VGGNet and ResNet-56) for CIFAR10?

How to prun MobileNet？

From Clipping gradient by 10 and nan correction, you made some manipulation of gradients.

Why is the gradients clipped by 10, what's the effect do you find if the gradients is too large?
Why will the gradients be nan in segment wise training process, you encounter the nan first or you know the nan is inevitable then you correct it, which one?

Hi, if I want to prune channels on regression task. What are the discrimination-aware loss labels? Assume that an image has 100*100 pixels, every pixel has a depth as its label like depth estimation. So in channel prunning, what labels should I define？