This is the implementation of the Sequential VAE in Towards a Deeper Understanding of Variational Autoencoding Models. The paper identifies a link between power of latent code and sharpness of generated samples. We are able to generate fairly sharp samples by gradually augmenting the power of latent code.

The model is a (non-homogeneous) Markov chain. For each step, we take the results from previous step, learn a new latent code, and train the model to perform reconstruction conditioned on both. We extensively use shortcut/resnet connections so that result from previous step can be well preserved. Samples look like this

For CelebA download the dataset from http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

Download the aligned and cropped version, unzip into a folder, and pass the path of that folder as db_path argument. Note that your db_path folder should directly contain all the image files without any additional nesting. You folder should look like this

Now you can train by running

python main.py --dataset=celebA --db_path=/path/to/celebA/dataset

You can see the visualizations in the folder $pwd/models/[network_name]/samples

For LSUN bedroom we provide a preprocessed version available for download here (Google Drive). The images have been center cropped, resized, and organized into batches with 10000 images each for efficient I/O.

Download this dataset, unzip into a folder. This folder should also directly contain all the npy files without additional nesting. You can train by running

python main.py --dataset=lsun --db_path=/path/to/lsun/dataset

For other LSUN scene classes you should first preprocess the files into batches of 10000, where each batch is a numpy array [batch, height, width, channel]. Place these batches into the db_path folder.

• To use a particular GPU/GPUs add option --gpus=[ids] such as --gpus=0,1 to use GPU 0 and 1. The visualizations in the paper are produced after 2 days of training on a single Titan X.
• To use other architectures other than default, use --netname=[name]. For supported architectures please refer to code. The name is the unique identifier for the network, and all related training log, visualizations, and checkpoint files, etc will be stored in the directory $pwd/model/netname. For example, to run visualization with tensorboard use tensorboard --logdir=[that directory]
• To also visualize the training process with a GUI window add --use_gui. By default all plots will be stored to network directory, this will also plot them in a window in addition to that.
• To change batch size, add --batch_size=[size]
• To visualize and plot the autoencoding reconstruction of the model, add --plot_reconstruction
• To add Gaussian and salt and pepper noise to perform denoise training add --denoise_train
• To control the number of batches before we visualize and make plots, use --vis_frequency=[num_batch]

The baseline architecture used is the Variational Ladder Autoencoder

This model can be viewed as a generalization of Infusion training. The difference is that for each time step we explicitly learn a latent code rather than using random pixels from ground truth as latent code.