How to compute the two-order partial derivative in a non-graph based framework about improved_wgan_training HOT 4 CLOSED

It's not currently possible to implement this model in torch/pytorch unfortunately, but there's an open PR which will add support for gradients-of-gradients, at which point it might be possible: pytorch/pytorch#1016

Computing gradients of gradients is typically done in tensorflow/theano/etc. as backprop through backprop, with 2x compute cost of just computing the gradients. It's done in several other works, https://arxiv.org/abs/1611.02163 and https://arxiv.org/abs/1502.03492 are the first that come to mind.

from improved_wgan_training.

@igul222 actually it is possible, but you'll need to compute the backward twice, and estimate the hessian-vector product using a finite difference approximation. See this tweet for more details.

from improved_wgan_training.

@apaszke Thanks for the note! However, I'm sorry that I'm still confused. Is there any example showing how to do this in pytorch?

from improved_wgan_training.

Unfortunately I'm not aware of any examples.

from improved_wgan_training.