ava-avd's People
ava-avd's Issues
about the content of the label files
Would you mind introducing the content of each column of the label files?
caffe2 dependency
@zcxu-eric Could you please take a look at this issue? I'm experiencing a problem with this error:
Traceback (most recent call last):
File "model/mmsc_exp/run.py", line 125, in
run()
File "model/mmsc_exp/run.py", line 81, in run
setup_imports()
File ".../AVA-AVD/model/mmsc/utils/env.py", line 187, in setup_imports
importlib.import_module(module)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File ".../AVA-AVD/model/mmsc/trainers/mmsc_trainer.py", line 17, in
from mmsc.trainers.core.evaluation_loop import TrainerEvaluationLoopMixin
File ".../AVA-AVD/model/mmsc/trainers/core/evaluation_loop.py", line 9, in
from caffe2.python.timeout_guard import CompleteInTimeOrDie
ModuleNotFoundError: No module named 'caffe2'
Since the 'caffe2' module is not mentioned in the requirements file, I guess the codebase should be updated to use PyTorch instead of Caffe2. So I don't know if the error message is real or outdated. I was wondering if you have any comment on this error.
Question about the baseline method
Hi, I have a question about the baseline method in the paper, but I cannot find your email, that's why I am asking here, sorry for the inappropriateness.
I wonder how the Speech+Face baseline in the 5.1.1 section is implemented. I assume the audio similarity and video similarity are obtained using pre-trained model, and the average of the similarity is calculated? Or it goes the other way. It would be helpful for me if you can provide the details.
Is this framework able to run on any video files?
Hi,
This is an interesting framework that we are looking to apply at one of our project. However, I'm currently stuck on preprocessing stage as it require to load the activespeaker csv for the video which I don't have. Would you please explain how this file is generated? Or point me a way so I can use your framework to inference on some of the video files that we have.
Thanks!
Question about training with only AVA-AVD
Hi, this is indeed fantastic work!
I have some problems with training on the AVA-AVD dataset only.
I trained for 28000 iterations, getting training loss of about 3.18. However, the DER is as high as 53 in the validation and test set (I have tried different thresholds). While the DER reported in the paper should be 27.
Should I train more iterations or adjust the learning rate or something? It would be very kind of you to supply the configuration of training only on the AVA-AVD dataset.
Label consistency between {uid}_c_01, {uid}_c_02, and {uid}_c_03
Thanks for this great dataset.
If I understand correctly, 15 minutes of each video were annotated manually for speaker diarization.
However, those 15 minutes were split into three 5 minutes exceprt ([900 --> 1200], [1200 --> 1500], and [1500 --> 1800]).
My question relates to the consistency of the labels between those three excerpts.
Is spk1
in {uid}_c_01
guaranteed to be the same speaker as spk1
in {uid}_c_02
and {uid}_c_03
?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.