Number of critic iterations about improved_wgan_training HOT 5 CLOSED

You're right: the theory says that the critic should be trained to optimality at each step; in practice, the closer we get to optimal, the better. The tradeoff is that optimizing the critic for longer takes more time for each iteration. We picked 5 iterations because it was a good tradeoff: stable enough in most settings, but not terribly slow. Increasing this value might help for harder problems though.

Re. Swiss roll specifically, the results in the paper show the optimal critic (i.e. trained for 10,000 iterations) against a fixed "generator" (i.e. the generator distribution is held fixed at the data distribution plus Gaussian noise), so the plots aren't really comparable. That said we were able to train full WGAN-GPs on Swiss roll to full convergence (which it seems like your plot hasn't reached yet. How long did you train for?)

from improved_wgan_training.

Just 400 generator iterations (times 50 disriminator) as I say above :)

from improved_wgan_training.

Would it be possible to increase the sample count used for showing how well you approximate the distribution, looks quite patchy after 400 iterations.

from improved_wgan_training.

@LukasMosser Sure, you just need to modify the number of samples here https://github.com/igul222/improved_wgan_training/blob/master/gan_toy.py#L147

I only ran this to 400 iterations make the point :) Not very interested in letting it run all day for the full iterations. Maybe @igul222 will if he updates the paper with the fixed contours but he did the calculations differently by keeping the generator fixed so it's not entirely equivalent.

from improved_wgan_training.

@stefdoerr big thanks for suggesting this. I too have found better results improving the critic iterations to 50. It takes about an eon to train but it does help nevertheless.

from improved_wgan_training.