mT0-xxl finetuning about xmtf HOT 6 OPEN

Hey there are some more details on mT0 fine-tuning here: #12
The config is here: #6 (comment)

from xmtf.

Thanks for reply! Will try mentioned config.

from xmtf.

Hey @Muennighoff , It's seems that I still can't get a couple of things. Would be very appreciate If you could give me a hand here.
Well, I need to finetune your model mT0-xxl (not the initial T5X-xxl), so accordingly to the manual https://github.com/google-research/t5x/blob/main/docs/usage/finetune.md I need 3 components (excluded SeqIO Task, which is clear as for now) to proceed:

1. Checkpoint -- Could you please share with mT0-xxl checkpoint? In the manual all used checkpoints are the TensorFlow weights etc, but on the HuggingFace there are only PyTorch weights. So I do need either mT0-xxl checkpoint in TensorFlow or finetune the model in PyTorch (is it even possible?)
2. Gin file for the model to finetune (mT0-xxl in the case) -- Could I use the default one like https://github.com/google-research/t5x/blob/main/t5x/examples/t5/mt5/xxl.gin?
3. Gin file configuring finetuning process -- I write it by my own based on https://github.com/google-research/t5x/blob/main/t5x/configs/runs/finetune.gin with some overrides, right?
   Please, correct me if I wrong in some points.

from xmtf.

There's a t5x ckpt here: https://huggingface.co/bigscience/mt0-t5x
I don't remember which size that model is though; I don't have the other ones anymore, maybe @adarob does

For 2. & 3., yes I think so

from xmtf.

from xmtf.

Thanks a lot, guys!

from xmtf.