A question about the structure of resnet about improved_wgan_training HOT 3 CLOSED

I have another question about resnet：

def ResnetGenerator(n_samples, noise=None, dim=DIM):
...
output = tf.tanh(output / 5.)

Why /5.?

from improved_wgan_training.

Re. scaling the residual by 0.3: it's a stability trick from https://arxiv.org/abs/1602.07261 (section 3.3) which I use by default. Should work fine without it though.

Re. dividing by 5, it's to (approximately) scale the variance of the outputs so the network doesn't start off in the saturated regime of the tanh. Again, you can probably do without, but this was the first thing I tried.

from improved_wgan_training.

(Fixed the _dev_disc_cost bug, thanks!)

from improved_wgan_training.