What makes the gan upsampler so effective about gigagan-pytorch HOT 10 CLOSED

@salvadog yea, i'll get the upsampler finished by mid this month, and we can figure out if it is just scale, or something else

from gigagan-pytorch.

@lucidrains Certainly, thank you! In my current work focused on image super resolution or restoration, the top priority is to preserve the faithfulness of the input data rather than solely pursuing visually pleasing outcomes. Although cGAN-based techniques are widely used, their capabilities in terms of generating high-quality images are still somewhat restricted. Considering that diffusion-based methods inevitably modify the input data to some extent, it would be highly advantageous to introduce an upsampler similar to gigagan that can mitigate this issue.

from gigagan-pytorch.

@salvadog i'm not sure... maybe it really is just stabilizing the attention via the l2 distance and then stack-more-layers-meme.jpeg

i honestly didn't even think i would ever have to build a GAN again after the DDPM revolution happened

from gigagan-pytorch.

@salvadog imo the multi-scale discrimination contributes the most, it somehow teaches the model how to do super resolution. And i feel it provides more supervison than "progressive growing", because the model would always receive gradients from all scales.

from gigagan-pytorch.

@salvadog we can leave this issue open and maybe someone who has been following GAN literature can chime in

from gigagan-pytorch.

"The power of GANs in super-resolution lies in their ability to navigate the complex and nuanced landscape of high-resolution images, weaving together the delicate threads of detail and structure to create a tapestry of realistic and stunning resolution."

from gigagan-pytorch.

"The power of GANs in super-resolution lies in their ability to navigate the complex and nuanced landscape of high-resolution images, weaving together the delicate threads of detail and structure to create a tapestry of realistic and stunning resolution."

Interesting. Where does this sentence come from?

from gigagan-pytorch.

moved

from gigagan-pytorch.

Hello, am I not totally lost in here thinking this "multi-scale discriminator" result in the model that could perhaps be considered "scale equivariant"?

I mean sure, in the SG3 it's actually the generator that is equivariant (rotation, translation).
However, in the limitations, discussion, & future directions section of the paper authors state:
"In this work we modified only the generator, but it seems likely that further benefits would be available
by making the discriminator equivariant as well."

from gigagan-pytorch.

@salvadog imo the multi-scale discrimination contributes the most, it somehow teaches the model how to do super resolution. And i feel it provides more supervison than "progressive growing", because the model would always receive gradients from all scales.

yes, I agree with this. perhaps the unet architecture for generator too

from gigagan-pytorch.