pretraining hyperparamters about clip4clip HOT 4 CLOSED

Hi, Yes, the hyperparameters are the same as the finetune on the msvrtt except for the batch size. The learning rate is also 1e-7. We have not tested other hyperparameters yet. Because the pretrain is time-consuming.

from clip4clip.

if we choose to use the seqTransf, this seqtransf is optimized using the learning rate of 1e-4, right? for training on small datasets like msvd, I think it is fine. but if we pretrain on howto100m dataset, do you think 1e-4 is a bit large?

from clip4clip.

I think 1e-4 is ok, or you can set it smaller.

from clip4clip.

ok got it. thanks very much for your quick response

from clip4clip.