Training on Colab - CUDA out of memory about maskgit-pytorch HOT 2 OPEN

Hi. I am also facing the same problem. Have you solved it now? I am going to try the pre-trained models provided in the https://github.com/CompVis/taming-transformers/tree/master.

from maskgit-pytorch.

Hi, unfortunately not. A last thing I would like to try is gradient accumulation but I don't think it will solve the problem.
By the way I was looking to do the same as you. If I succeed I can post my solution here.
You can find also here a pretrained VQGAN and also MaskGIT: https://huggingface.co/llvictorll/Maskgit-pytorch/tree/main
Their GitHub: https://github.com/valeoai/Maskgit-pytorch.
They modified some parameters and other few things of transformer, so I suggest to use only their VQGAN if you want to follow the original implementation of MaskGIT.

from maskgit-pytorch.