giorgiaauroraadorni / gansformer-reproducibility-challenge Goto Github PK

File "/content/gansformer-reproducibility-challenge/src/dnnlib/tflib/tfutil.py", line 16, in
import tensorflow.contrib # requires TensorFlow 1.x!
ModuleNotFoundError: No module named 'tensorflow.contrib'

Dear Giorgia, Felix and Stefano,
Thank you very much for the interest in the GANformer model and for working on reproducing it!
I just read the reproducibility report and wanted to comment.

Regarding the attention in the generator vs. discriminator: In earlier stages of the model development, I have explored the use of bipartite attention in both the generator and the discriminator, and similarly to the observations you've had, as I kept working on it, I noticed that indeed the model performs better when incorporating attention to the generator only, and so the pre-trained models + default command-line settings of the public repository reflect that.

In the GANformer2 paper we released later last year, we also followed this similar design (of using bipartite attention over the generator only). Table 5 in the first GANformer paper, that shows the number of parameters for the generator and the discriminator for different approaches also matches that (indicating that the GANformer and StyleGAN2 discriminators have the same number of parameters). The paragraph about the discriminator using bipartite attention at the bottom left of page 7 should have been removed and it stayed there due to a mistake on my side. I updated the paper to address that and it should become public through arxiv later today!

Regarding the report's empirical section, we believe that the benefits of the bipartite attention approach are obtained mainly due to the better support of long-range interactions across the image, and so that mainly comes into play for high-resolutions, compositional/complex scenes and for a full coarse of training (where the model starts focusing on fidelity of small details rather than on being coarsely right about the overall shape, as happens in earlier phases of the training). The empirical evaluation in the reproducibility report potentially misses these factors, by lowering the data resolution, focusing on faces only, and comparing results after short training (300 kimgs in the report vs. 10,000 kimgs in the GANformer paper, and 25,000-70,000 kimgs in StyleGAN2).

As is discussed in the paper, the gains in FFHQ after completing the full training are indeed the smallest compared to the other datasets due to being less compositional and structurally diverse than multi-object scenes (as is the case for CLEVR, LSUN-Bedrooms and Cityscapes). Indeed, note that the learning curves comparison provided in the paper are for the CLEVR dataset. We chose CLEVR over e.g. FFHQ since we believe it to be a good example of a compositional scenes dataset that could express the benefits of our more-compositional model.

Finally, wanted to mention that in the report where it compared epsilon values, note that epsilon doesn't stand for learning rate, but rather it's used as a small value that is added by the optimizer for stability. It was set to 1e-8 in both StyleGAN2 and GANformer2 repositories. Meanwhile, the learning rate in the GANformer's repository is 0.001 in line with the paper. (For the other optimizer settings of beta1, beta2 where the Supp. says "beta1=0.9, beta1=0.999", I have fixed that to "beta1=0.0, beta2=0.9" to comply with the repository).

Thanks again for looking into the paper! If you'd like to chat about the paper or have any thoughts, questions or feedback please don't hesitate to concat me at [email protected]. Wishing you all the best,
Drew