zcxu-eric / ava-avd Goto Github PK

Would you mind introducing the content of each column of the label files?

@zcxu-eric Could you please take a look at this issue? I'm experiencing a problem with this error:

Traceback (most recent call last):
File "model/mmsc_exp/run.py", line 125, in
run()
File "model/mmsc_exp/run.py", line 81, in run
setup_imports()
File ".../AVA-AVD/model/mmsc/utils/env.py", line 187, in setup_imports
importlib.import_module(module)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File ".../AVA-AVD/model/mmsc/trainers/mmsc_trainer.py", line 17, in
from mmsc.trainers.core.evaluation_loop import TrainerEvaluationLoopMixin
File ".../AVA-AVD/model/mmsc/trainers/core/evaluation_loop.py", line 9, in
from caffe2.python.timeout_guard import CompleteInTimeOrDie
ModuleNotFoundError: No module named 'caffe2'

Since the 'caffe2' module is not mentioned in the requirements file, I guess the codebase should be updated to use PyTorch instead of Caffe2. So I don't know if the error message is real or outdated. I was wondering if you have any comment on this error.

Hi, I have a question about the baseline method in the paper, but I cannot find your email, that's why I am asking here, sorry for the inappropriateness.
I wonder how the Speech+Face baseline in the 5.1.1 section is implemented. I assume the audio similarity and video similarity are obtained using pre-trained model, and the average of the similarity is calculated? Or it goes the other way. It would be helpful for me if you can provide the details.

Hi,

This is an interesting framework that we are looking to apply at one of our project. However, I'm currently stuck on preprocessing stage as it require to load the activespeaker csv for the video which I don't have. Would you please explain how this file is generated? Or point me a way so I can use your framework to inference on some of the video files that we have.

Thanks!

Hi, this is indeed fantastic work!
I have some problems with training on the AVA-AVD dataset only.
I trained for 28000 iterations, getting training loss of about 3.18. However, the DER is as high as 53 in the validation and test set (I have tried different thresholds). While the DER reported in the paper should be 27.
Should I train more iterations or adjust the learning rate or something? It would be very kind of you to supply the configuration of training only on the AVA-AVD dataset.

Thanks for this great dataset.

If I understand correctly, 15 minutes of each video were annotated manually for speaker diarization.

However, those 15 minutes were split into three 5 minutes exceprt ([900 --> 1200], [1200 --> 1500], and [1500 --> 1800]).

My question relates to the consistency of the labels between those three excerpts.

Is spk1 in {uid}_c_01 guaranteed to be the same speaker as spk1 in {uid}_c_02 and {uid}_c_03?