How to train tieredimagenet？ about deepemd HOT 7 CLOSED

@icoz69 When I was training your model on tieredimagenet, I encountered some problems. I train as follows. First, train the pretrain model according to the default parameters you give. Then use the default parameters to train the meta stage. We only got 68.30 acc on validation. We adjusted the learning rate lr from 0.0005 to 0.0001 according to the suggestions you gave. The model achieved the highest accuracy rate of 68.25 acc in the first epoch. In order to eliminate the influence of the pretrain stage, we used the pretrain model you gave when training the meta stage, but still only got 68.04acc. Can you give me detailed training strategies so that I can reproduce your results?

from deepemd.

hello, you can use the default hyper-parameters to train all datasets, but you may change it a little bit to get the best performance, though they are close. My training practice is to only tune the learning rate at the meta-learning stage. For example, if you find the validation acc saturates very quickly, then you can try a small lr.

from deepemd.

hi, when we were training the model on other datasets,
we only adjust step_size and gamma during pre-training, and lr during meta-training.
so, you may try other configurations during pre-training, such as step_size=20, while gamma=0.3. since the number of training images varies for different datasets, the optimal lr schedule may also vary.

from deepemd.

@icoz69 Thank you for your reply. If I initialize with the pretrain model you gave, how much should the lr of the meta stage be set on tierdimagenet?

from deepemd.

the provided pretrain models are not necessarily the optimal ones. i train these models using the released codes. so, you may try different values. my practice is that if the val acc stop increasing very early, I set a lower lr.

from deepemd.

@icoz69 Thanks for your advice. I used your default parameters. During the training of the meta stage, although the val acc was only 68.30, the test acc reached 71 in the final test stage. Your code is very robust, thanks for your excellent work!

from deepemd.

you're welcome. good luck to your project!

from deepemd.