dongyp13 / stochastic-quantization Goto Github PK

Hi . Thank you for provide your great code!

I have a two question.!

1. In paper, vgg9 model(cifar10) don't use data augmentation. But , this code use data augmentation like crop and mirror. In paper vgg 9 get 10% error in cifar 10. Is this the result of using data augmentation? ...

2. Is this code saved as a 32bit model or binary model ?

Thank you !!

@dongyp13 Hi, I read your paper and thanks for sharing code.
I am now trying to reproduce your paper with tensorflow. Since I do not know anything about caffe, so I just need to reproduce it with the information only on paper.

My question is,

1. did you quantize the first and last layers of the network?
2. and I wonder if you got the results by ensemble method.
3. Finally, as an example of the first stage, I wonder whether 50% stochastic sampling was performed in the entire layer or 50% sampling was performed in each layer.

Again, thanks for sharing the code and congratulations on your acceptance at the BMVC conference.
Best Regards.

我最近在研究量化相关的方向，看到了您的论文。注意到您使用ResNet-56在cifar100上取得了35.01的错误率，但我自己实现时最高只有43。

我去看了您的resnet56的代码，但因为我没学过caffe所以看不懂。

想请教您一下，您选取的optimizer、学习率退火方式以及相应的超参数是什么？

还有，我看您在别的issue里提到您只量化了weight，请问这里的weight包括bn层及bias吗？

十分感谢

Hi @dongyp13,
Thank you for sharing the code.
I am training the ImageNet/AlexNet-BN/SQ-TWN from scratch. The network seems to be learning, 50K iterations of the first stage have run so far, and I can see the Top1/Top5 accuracy as 0.12/0.28.
I extracted the trained weights from this caffemodel snapshot, and was expecting that 50% (= r) of the weight values would be +1/0/-1, but I dont see such quantized weights for any conv/fc layer in the caffe model.
Could you please guide from where to obtain the quantized weights? Is there any special way to dump them, or will they get reflected in the caffe model after all the stages have run?

Hi, 您好：
看了您的论文以及源码，获益颇多，在此有几个问题想请教一下，还请帮忙解惑~
1. 权重量化为低比特，input 以及features map是float32，是否有什么方法也可以做到量化？我想过直接映射到8bit，但是与-1这样的权重乘积会出错，除非把权重量化到8bit，然后扩成16bit计算，防止溢出，效率会下降，而且每层还需要取max和min。
2. bias，batchnorm，prelu这些一个参数作用于整个feature map，量化的话，是否会有很大损失？

@dongyp13 Hi, thanks for sharing code. Could you release your pretrained models (specifically ImageNet AlexNet/ResNet SQ-TWN)? I try to reproduce your results but accuracy is very low even at first iterations...

Wow.很棒的工作！
我现在是仿照您的工作自己实现了下BWN,我浏览下了您的代码，貌似在梯度更新的时候并没有按照paper里推导公式来更新的 是直接将算出的梯度更新到原来的权重里的，或者代码是在其它的地方实现了这一更新吗