[Image Captioning] Training problem about pytorch-tutorial HOT 1 CLOSED

If you have calculated the loss in the same way as my code, it is normal that loss is about 2.5 (loss 2.5 equals to perplexity 11.97). This means that the model is well trained. I think that something went wrong during the evaluation phase. Below are some good things to try.

1. Add batch normalization to the top layer of CNN.
   This makes training more stable. See here.

2. Use 0.01 for momentum in BN
   The default value of the momentum in BN is 0.1, which can not calculate moving average/variance well for the entire training dataset.

3. Change the cnn model to eval mode before sampling the caption
   Assume that the model is training mode and you give 1 image to the model to generate the caption. In training mode, BN uses mini-batch statistics which makes all activation values 0 (for one image). To properly generate the caption, give the model multiple images, or change it to eval mode.See here for how to change the model to eval mode.

I hope this helps.

from pytorch-tutorial.