hyperparameter tuning about cct HOT 2 CLOSED

Hi @lqycrystal
Thank you for your interest.

I would suggest starting with a simple case, using only a single perturbation / aux. decoder with say only K=2, this will reduce the training time and gives you the ability to find the best hyperparameters (LR, loss weightings and ramp-up period, etc...), then add additional aux. decoders. You can also try using annealed CE that helps with small labeled sets.

I'd also like to note that the performance gain depends on the amount of labeled / unlabeled data, if the difference if big, then SSL helps. But the biggest the labeled set, the smaller the gains are.

from cct.

Hi @lqycrystal
Thank you for your interest.

I would suggest starting with a simple case, using only a single perturbation / aux. decoder with say only K=2, this will reduce the training time and gives you the ability to find the best hyperparameters (LR, loss weightings and ramp-up period, etc...), then add additional aux. decoders. You can also try using annealed CE that helps with small labeled sets.

I'd also like to note that the performance gain depends on the amount of labeled / unlabeled data, if the difference if big, then SSL helps. But the biggest the labeled set, the smaller the gains are.

Thanks for your reply!

from cct.