I am doing 2 class classification using BLVNET-TAM.
When i had given data for validation the accuracy is coming good but on testing same data accuracy is bad.
for instance ,
validation probabilities are somewhat like [0.2 , 0.8]
while during testing probabilities are like [0.5.. , 0.499..] on same data.

Is it intentional to have

if input_channels != 3

only in this line?

As it stands the code only loads the weights for the flow-case and not the RGB-case. I have thus extended the condition to the below and was wondering if it's correct:

        if input_channels != 3:  # flow
            ....
        else:
            print("Loading RGB weights")
            state_d = checkpoint['state_dict']
        model.load_state_dict(state_d, strict=False)

Hi, in training I encountered RuntimeError: Function SqueezeBackward1 returned an invalid gradient at index 0 - expected device cuda:3 but got cuda:4. In the paper I remember you used 8 GPUs. Have you encountered this issue before? Thanks.

The command I am using is python3 train.py --datadir ../../sthsthv2 --dataset st2stv2 -d 101 --groups 16 --logdir ./logs --lr 0.01 -b 64 --dropout 0.5 -j 36 --blending_frames 3 --epochs 50 --disable_scaleup --imagenet_blnet_pretrained. And I have the pretrained model ready under ./pretrained.

The environment I am using under Ubuntu18.04.3 LTS is:

#Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
ca-certificates           2019.11.27                    0
certifi                   2019.11.28               py37_0
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 9.1.0                hdf63c60_0
libstdcxx-ng              9.1.0                hdf63c60_0
ncurses                   6.1                  he6710b0_1
numpy                     1.17.4                   pypi_0    pypi
openssl                   1.1.1d               h7b6447c_3
pillow                    6.2.1                    pypi_0    pypi
pip                       19.3.1                   py37_0
protobuf                  3.11.1                   pypi_0    pypi
python                    3.7.5                h0371630_0
readline                  7.0                  h7b6447c_5
scikit-video              1.1.11                   pypi_0    pypi
scipy                     1.3.3                    pypi_0    pypi
setuptools                42.0.2                   py37_0
six                       1.13.0                   pypi_0    pypi
sqlite                    3.30.1               h7b6447c_0
tensorboard-logger        0.1.0                    pypi_0    pypi
tk                        8.6.8                hbc83047_0
torch                     1.3.1                    pypi_0    pypi
torchvision               0.4.2                    pypi_0    pypi
tqdm                      4.40.2                   pypi_0    pypi
wheel                     0.33.6                   py37_0
xz                        5.2.4                h14c3975_4
zlib                      1.2.11               h7b6447c_3

Thanks for your excellent work!

I tried to train a bLVNet-101-TAM-a2-b4-f8x2 model on Something-Something V1 using the hyperparameters setting reflected in the following bash command:

CUDA_VISIBLE_DEVICES=0,1 python train.py --datadir /media/data/something_v1 --dataset st2stv1 -d 101 --groups 16 --logdir logs --lr 0.01 -b 64 --dropout 0.5 -j 36 --blending_frames 3 --epochs 50 --disable_scaleup --imagenet_blnet_pretrained

At the end of the training, I get only around 31% validation accuracy, whereas the training accuracy is around 79%. Comparing with TSM-ResNet50, the training accuracy is actually higher (79% vs 78%), so it is even more confusing on why the validation accuracy is that low. Is it overfitting by any chance? Or is there something wrong with preprocessing the validation data (note that I used the scripts you provided to preprocess data)? Any comments on what I may be doing wrong here? I am sharing the corresponding training/validation log.
log.log

Update: Trained bLVNet-101-TAM-a2-b4-f8x2 on Something-Something V2 as well. With the training script provided, validation accuracy reaches around 53% only (less than the ~60% accuracy reported). Also, for Something-Something-V2, the training accuracy is lower compared to TSM-ResNet50 (88.2% with TSM vs. 78.5% with bLVNet). Please let me know your thoughts on what may be going wrong here. Log corresponding to SthV2 training -
log.log

I found that you haven't provide flow models, so if i want to retrain a flow model.
Are all training parameters the same with RGB training?
And do i need to train the model from scratch with random initialization?
Thanks~

Thank you very much for posting the codebase! However, I'm having difficulty replicating the 71.0% acc mentioned in the paper for 'bLVNet-TAM-8×2', I only get around 55% when validating each epoch. Training accuracy is 58%. If I do 10 crops per video for validation this rises to around 57%

Would it be possible to share your training log file? Or know your final training accuracy to see if something is wrong with my validation/eval script?

Wanted to check if any of the below was wrong:

• TAM Backbone is initialised from 'ImageNet-bLResNet-50-a2-b4.pth.tar'
• For bLVNet-model:

{'depth': 50, 'alpha': 2, 'beta': 4, 'groups': 16, 'num_classes': 400, 'dropout': 0.5, 'blending_frames': 3, 'input_channels': 3, 'pretrained': None, 'dataset': 'kinetics400', 'imagenet_blnet_pretrained': True}

• For the training: 0.01 LR with a total batch of 64 (8 * 8 per GPU); cosine-annealing LR-schedule trained for 50 epochs

I thought perhaps this should be trained for 100 epochs; similar to TSM, not 50?