Incorrect test accuracy calculation about pytorchnet HOT 3 CLOSED

The problem you refer to does not seem to exist in this repo, so I will close the issue here. Please open an issue on the PNN repo instead. You can re-open here if you think the problem persists in this repo.

from pytorchnet.

It does not exist in this repo, and it does not exist in PNN repo, because neither repo includes a method to calculate test dataset accuracy. Instead, PNN paper seems to use your 'running mean" accuracy to report test dataset results. If that's the case, I suggest you add the proper method here to avoid similar problems in the future.

from pytorchnet.

Actually, let me take that back: your repo does have this problem. You display "running mean" values during progress bar output, and the last shown accuracy value remains as if it were the test dataset accuracy for that epoch. See the second image in your README. The per epoch accuracies displayed in that image are not the actual test dataset accuracies, they are smoothed running means, mostly influenced by the accuracy of the last batch. That's why they vary so wildly in that image from epoch to epoch.

PNN repo, on the other hand, does not display progress bar, but simply prints accuracy for every batch on a separate line (so it's less misleading there). Yet it still seems the values reported in PNN paper used the running mean values (e.g. looking at that image again, you really should not pick 93.75% accuracy from epoch 6 as the best result of your experiment).

from pytorchnet.