About epoch and AUC about rtfm HOT 7 OPEN

@Yii-maker I also tried to run the code on Shanghai pretrained features and I got ~89% AUC . Did you obtain the same score ?

from rtfm.

@Yii-maker I also tried to run the code on Shanghai pretrained features and I got ~89% AUC . Did you obtain the same scor

from rtfm.

Hi. The checkpoint is selected based on the highest AUC, following the previous anomaly detection papers.
I did not apply any mechanism other than the code in this repository. With I3D you should get around 96.XX - 97.XX based on different random seeds. With C3D, you should get around 90.XX - 91.XX AUC.

from rtfm.

Hi @tianyu0207 thank you for your reply. I rerun your code with different random seeds (using your I3D features) but test AUC always converges around 90%. I tried to visualize the training loss and i noticed the total loss (cost term in the code) increases after some epochs. I tried to decompose the loss terms and it seems to me that the increase is due to the margin loss (um_loss term in the code) (in the plot I tried a few configurations with the default being yours implemented in the code, and only for visualization purpose I did not multiply by alpha) :

Did you encounter the same problem ? If so how did you manage to get a decreasing training loss ? Do I need to change any of the hyperparameters to get around 97% AUC ? Thanks !

from rtfm.

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC.

from rtfm.

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC.
I got 94% AUC and I can't get 96-97% AUC.

from rtfm.

@KhalilBergaoui Have you got 96-97% AUC, I have tried the same code using I3D features given in the Readme but got only 92% AUC.
I got 94% AUC and I can't get 96-97% AUC.

I got around 95.7% AUC using I3D features of SHT.

epoch: 4575
0.956566358955364

And I can't get AUC above 97%.

from rtfm.