The code is not based on any paper. But it does something similar to this paper or (less similar to) this paper.

Before running the code, you should install improved-diffusion package. Check the readme of the improved-diffusion folder. Note: install from my copy, and not from the original repository.

Also models imagenet64_uncond_100M_1500K.pt and upsample_cond_500K.pt need to be downloaded. Links are in the improved-diffusion folder.

To generate 16 samples of doge, run:

python generate.py --model_path imagenet64_uncond_100M_1500K.pt --input_image doge.png --batch_size 16

To upsample these samples from 64 to 256 resolution, run:

python upsample.py --upsample_model_path upsample_cond_500K.pt --lowres_images generated_doge.png.npy --input_image doge.png

The doge image

Turns into this (one of possible outcomes):

As you can see, the model gave doge a body.

The image of myself turned into this (right half of the image was masked off):

As you can see, the model gave me a bird.