This is the code associated with the paper Two steps at a time --- taking GAN training in stride with Tseng's method. If you find this code useful please cite as:

Axel Böhm, Michael Sedlmayer, Robert Csetnek, Radu Bot Two steps at a time --- taking GAN training in stride with Tseng's method, 2020

The code base was mostly taken from here associated with the paper A Variational Inequality Perspective on Generative Adversarial Networks.

The code is in pytorch and was tested for:

• pytorch=1.3.1

(Optional) The inception score is computed using the updated implementation from A Note on the Inception Score which can be found here.

A conda environement is also provided (requires CUDA 10): conda env create -f environment.yml

The Forward-Backward-Forward method is packaged as a torch.optim.Optimizer with an additional method extrapolation(). Two variants are available FBFSGD and FBFAdam.

Example of how to run FBF:

for i, input, target in enumerate(dataset):
    FBF.zero_grad()
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    if i % 2:
      FBF.extrapolation()
    else:
      FBF.step()

To run the WGAN experiment with weight clipping using FBFAdam and the DCGAN architecture on CIFAR10 with the parameters from the paper: python train_fbfadam.py output --default --cuda

To run the WGAN-L1 experiment with L1 regularization using FBFAdam and the DCGAN architecture on CIFAR10 with the parameters from the paper: python train_fbfadam.py output --default --cuda -rp 0.0001

The --default option loads the hyperparameters used in the paper for each experiments, they are available as JSON files in the config folder.

The weights for our WGAN-L1 and DCGAN model trained with FBFAdam are available in the results folder.

For evaluation: python eval_inception_score.py results/FBFAdam/best_DCGAN_WGAN-L1.state and python eval_fid.py results/FBFAdam/best_DCGAN_WGAN-L1.state

An ipython notebook is also available for the bilinear example here.

IS of all compared methods for WGAN (with weight clipping):

IS of all compared methods for WGAN-L1: