Using test loss to choose the best model? about multimodal-infomax HOT 4 CLOSED

Thanks for your discovery. Actually here we don't.

Note in

We write in this style to save time, because if at some epoch valid loss reaches a new lowest but test loss doesn't, we can early stop the training process between the current step and last best step.

from multimodal-infomax.

Thanks for the quick response!

Imagine for epoch 1, you got 0.8 val loss and 1.0 test loss. For epoch 2, you got 0.5 val loss and 1.5 test loss. In this case, the first model would be chosen which is wrong.

Thanks for your discovery. Actually here we don't.

Note in

Multimodal-Infomax/src/solver.py

Line 295 in 9a6dc8f

if val_loss < best_valid:

the outer loop is still conditioned on the valid loss.

We write in this style to save time, because if at some epoch valid loss reaches a new lowest but test loss doesn't, we can early stop the training process between the current step and last best step.

from multimodal-infomax.

In this case, you can set the maximum training epoch to 1 as I replied above.

from multimodal-infomax.

I see... Thanks

In this case, you can set the maximum training epoch to 1 as I replied above.

from multimodal-infomax.