Misalignment of audio-visual frames and labels about ego4d_talknet_asd HOT 3 OPEN

Hi, I think both are good enough if there is not any difference in performance, but it is hard to evaluate because the labels are not so accurate. Feel free to raise a pull request if you would like to help fix it. Thanks a lot.

from ego4d_talknet_asd.

Also, some clip frame numbers that are there in the bbox annotations (stored in data/ego4d/bbox) are missing in data/video_imgs. For e.g., there is frame number 8991 at index 291 of the list stored in data/ego4d/bbox/34bc808a-4b98-42c5-a1ea-6f565dd4aa20:track_0:29.json but there is no corresponding image frame for it, i.e., data/video_imgs/34bc808a-4b98-42c5-a1ea-6f565dd4aa20/img_08991.jpg doesn't exist. I looked at https://github.com/zcxu-eric/Ego4d_TalkNet_ASD/blob/main/scripts/extract_frame.sh and it looks like it dumps all clip frames in data/video_imgs. Any idea why this is the case? Thanks.

from ego4d_talknet_asd.

Hi, we didn't encounter this bug on our side. One quick solution is to check the data sanity before training and discard these labels if necessary.

from ego4d_talknet_asd.