scut-ailab / dcp Goto Github PK
View Code? Open in Web Editor NEWCode for “Discrimination-aware-Channel-Pruning-for-Deep-Neural-Networks”
License: BSD 3-Clause "New" or "Revised" License
Code for “Discrimination-aware-Channel-Pruning-for-Deep-Neural-Networks”
License: BSD 3-Clause "New" or "Revised" License
Please add a license.
Here is going to copy modules of pruned_modules, why don't you just do
self.ori_model = copy.deepcopy(self.pruned_model)
,but copy modules by modules?
Is there any concern?
I think the additional discrimination-aware loss in this project is suitable for classification task but not detection task ,the classification task focus on the whole feature map , therefore averagepooling+softmax works , but the detection task focus on feature in receptive field , if I want to use this method on yolo-v3 , how can I modify the discrimination-aware loss ?
RuntimeError: one of the variables needed for gradient conputation has been modified by an inplace operation.
When I use torch.autograd.set_detect_anomaly(True) to find the operation that failed to compute its gradient, it shows "x = self.conv(2) " in dcp/models/preresnet.py. I can't find the solution, Could anyone help me?
In Algorithm 1, "Learn theta and Fine-tune M with (L^S)_p and L_f." which means that the weights of the first layer to layer L_p will be updated w.r.t to (L^S)_p and whole model weight will be updated w.r.t L_f (the final loss), is that right?
In this line, I can understand that i != index
is going to update weights of the first layer to layer L_p by self.seg_optimizer[i].step()
but the i != len(self.pruned_segments) - 1
is going to update weights of the first layer to the end of the second-last segment.
According to Algorithm 1, it should be the last segment, that is, the whole model, but why the second-last segment?
I'm trying to reproduce your experimental results on ImageNet, however I find that training resnet networks really takes super long time !!! So I want to know how much time did you use to train resnet on ImageNet dataset??? Thanks !!!
Thanks for your paper and sharing code. I have some confusions about code, can you explain it for me.
Hi,
Great work! You released the structures of ResNet nets in models.py except ResNet56, could you please make the definition of ResNet56 available as well? I'm aware of many online versions available, but I just would like to figure out some details, like whether bias=True or not in conv layers etc.. Thanks.
when i donwload your pruned model(imagenet-resnet50-0.5), then test it with test.py.
the result is as below:
|===>Validation Error: 28.2700 Loss: 1.1303, Top5 Error: 9.4360
but in your paper the result is lower:
top1 error:25.05 top5 error:7.68
why? is there some difference?
I like your feature maps shown in the paper very much. The color matching is so good. Do you draw the feature maps with imshow function in matplotlib? Which colormap do you choose? Or other methods to show the beautiful feature maps? I try to show feature maps but they look ugly. Are there any advice? Thanks you very much!
learning rate, pruning ratio, e.g.
Hi:
release the code?
how do you prune the LResNet34E-IR model,and where can i get the "LResNet34E-IR (prune 50% channels) " model?
Hello, what a good work you did!
I have the problem that when I reproduce the work, I find that the self.pruned_weight (in line 25 of /dcp/mask_conv.py) is not loaded from the original module (It may be generated from lib of nn.Conv2d? because the MaskConv2d derives from nn.Conv2d).
The weight of original layer:
The weight of self.pruned_weight:
Why the weight is not loaded from the original module?
Thanks a lot.
Firstly, congratulations on the excellent performance of your proposed model on specific data sets and some pre-training models.
As the title shows, I need to tailor my own pre-trained model which datasets are related to audio rather than images.And the pre-trained model includes not only CNN but also RNN.
Therefore, I would like to ask you if I can use your given model in my case.If not, do you have other ideas for model pruning?
Thanks a lot
Best wishes
GUOhm230
Could you, please, share a link to fine-tuned pruned models (VGGNet and ResNet-56) for CIFAR10?
How to prun MobileNet?
From Clipping gradient by 10 and nan correction, you made some manipulation of gradients.
Why is the gradients clipped by 10, what's the effect do you find if the gradients is too large?
Why will the gradients be nan in segment wise training process, you encounter the nan first or you know the nan is inevitable then you correct it, which one?
Hi, if I want to prune channels on regression task. What are the discrimination-aware loss labels? Assume that an image has 100*100 pixels, every pixel has a depth as its label like depth estimation. So in channel prunning, what labels should I define?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.