WGAN GP detach is necessary? about pytorch-gan HOT 5 CLOSED

Wouldn't it be better to detach the fake data here:

fake_validity = discriminator(fake_imgs.detach())

We don't want to calculate gradients for the generator when doing backpropagation on the critic:

Although it doesn't matter in practice since we later zero out the gradients here:

It still introduces unnecessary computation. Or is there a reason we don't want to detach that I'm missing?

from pytorch-gan.

I see no problem in this case (although using .detach() wouldn't hurt). Neither real_samples or fake_samples are used for the backward pass. I.e. the fact that real_samples.detach() has the benefit of having it's in-place changes reported by autograd should make no difference as real_samples is not used.

from pytorch-gan.

They are already detached since they are fed to the function as real_imgs.data, fake_imgs.data.

from pytorch-gan.

@eriklindernoren I am not sure about it.
pytorch/pytorch#6990

What do you think about this issue?

from pytorch-gan.

@cwarny I agree with you! My answer is similar to yours, here

from pytorch-gan.