tbung / pytorch-revnet Goto Github PK

Hi,
I was wondering if you are planning to implement the bottleneck module.
Congrats for the paper, very good work.

it tell me "torch' has no attribute 'no_grad"

Hi,

Thank you for your implementation on it, it seems really interesting!

I tried to take a crack at modifying this code for some of my own work but ended up getting an error along the lines of RuntimeError: function RevBlockFunctionBackward returned an incorrect number of gradients (expected 33, got 32. Would it be a (presumably not too hard) feature request to have padding+dilation options added into the RevBlock? It would be greatly appreciated.

Kind regards,
Chris

Hey!

First of all, thank you for your hard work. I think reversible networks are very interesting and will open up a lot of new use-cases for ML.

I noticed two things in using your code.

First of all, I had to rename the revnet subdirectory to models because train_cifar.py imports models and then models.revnet.

Secondly (and more importantly), when I run train_cifar.py --model revnet38 my loss quickly goes to NAN and the accuracy never surpasses 10%. Same thing for revnet110. The resnet architectures on the other hand all work perfectly fine with good accuracy. Any idea what's going on?

hello, @tbung
My Pytorch version is 0.4.0, cuda version is 9.0 and python is 3.5.
But when I run this command:

python train_cifar.py --model revnet38 --clip 0.25

Problem happened:

File "pytorch-revnet/revnet/revnet.py", line 191, in _grad 
        dy1, dy2 = Variable.chunk(dy, 2, dim=1) 
AttributeError: type object 'Variable' has no attribute 'chunk'

Can you help me to fix the bug? Thank you.

I cant decide where and why exactly this is happening.. but this happens when I run the code from your repo.. Probably somewhere in the model definition

As you mentioned, this implementation meets with exploding gradients. Is there any reason for this? Is it due to the reversible connection or implementation problem?