stream inference about movinet-pytorch HOT 5 CLOSED

You can do streaming inference either submitting the whole clip to the network, or submitting subclips without cleaning the buffer (similarly to how it is done in training).
The simplest way is the first one and you perform inference almost like the non streaming network, you just have to remember to clear the stream buffer every new clip like in the code snippet below.
There is not much documentation because currently the streaming models are not available pretrained.

def evaluate(model, data_load, loss_val):
    model.eval()
    
    samples = len(data_load.dataset)
    csamp = 0
    tloss = 0
    with torch.no_grad():
        for data, _, target in data_load:
            model.clean_activation_buffers()
            output = F.log_softmax(model(data), dim=1)
            loss = F.nll_loss(output, target, reduction='sum')
            _, pred = torch.max(output, dim=1)
            
            tloss += loss.item()
            csamp += pred.eq(target).sum()
    aloss = tloss / samples
    loss_val.append(aloss)

from movinet-pytorch.

thanks .
if models is trained on let us assume on (50,224,224,3) input size then how are they doing inference ? because on official implementation ,we can pass any input size .
but how it is possible ,once model is trained on 50 frame inputs.
can you explain.
https://github.com/tensorflow/models/tree/master/official/vision/beta/projects/movinet

from movinet-pytorch.

If you have question about the official implementation you should ask them, I'm not aware of every details of their implementation.
Anyway, In a general sense you can always change the input shape, generally neural networks are resilient to changes in input shapes. You probably won't get the same accuracy though.
To further explain, in the notebook linked below I finetune with a different number of frames (16), on another dataset and it is perfectly fine.
https://github.com/Atze00/MoViNet-pytorch/blob/main/movinet_tutorial.ipynb

from movinet-pytorch.

can you explain how to do inference on single video? you didnt mentioned it into your notebook.
also where are you saving model checkpoint and how do you loading it during inference .

from movinet-pytorch.

These questions are related to the Pytorch framework more than this particular implementation, you can ask to a pytorch forum if you have problems doing inference of one single sample. The model is kept in ram, it is not saved on the disk and reloaded at least in the tutorial. You can easily find pytorch tutorials on how to save and load models.

from movinet-pytorch.