Issue regarding the last step of self attention (weighted sum step) about msva HOT 3 OPEN

Hi, could you please guide me on how to summarize my own video?

from msva.

Hi, could you please guide me on how to summarize my own video?

you have to extract features first for frames of the video and then based on trained model you can predict probability to be in a summary. For object features refer https://github.com/VideoAnalysis/EDUVSUM/tree/master/src

Motion feature code I will upload after refactoring.

from msva.

I fully agree with @pangzss. If my calculations are right, the used formula/command

y = torch.matmul(V.transpose(1,0), weights).transpose(1,0)

would be correct, only if the weights array was symmetric, but this isn't the case.
Oddly enough, the produced results doesn't change much when the corrected formula/command is used.

from msva.