Comments (5)
Thanks a lot for the question~ the default format of the video and audio data is [batch_size, sequence_length, feature_dim], because the feature here are preprocessed from the videos (using OpenFace or Facet), so the format didn't follow the default video data format~ And seems you are more familiar with the video data format, and the video data is here, you can deal with the video directly if you want to try some video models by yourself ~ It really rare for people to deal with it that way, so it may be a good direction~
from multibench.
Thanks a lot for the question~ the default format of the video and audio data is [batch_size, sequence_length, feature_dim], because the feature here are preprocessed from the videos (using OpenFace or Facet), so the format didn't follow the default video data format~ And seems you are more familiar with the video data data format, and the video data is here, you can deal with the video directly if you want to try some video models by yourself ~ It really rare for people to deal with it that way, so it may be a good direction~
It doesn't look like this dataset contains a list of mosei files, do you know where it is please, thanks! @Vanvan2017
from multibench.
Hi, @TianhaoFu, did you mean the videos? It should be in the zip file here. You can download and unzip it and then use a video processing tool to get the raw frames~ As for the labels, you can check the labels.
from multibench.
Thanks for your reply. I mean I want a file contains the video files name. such as
column1
7155.mp4
7156.mp4
7157.mp4
...
do you know where can I such file? thanks!
By the way, It seems that the labels file you provided is .csd file, could tell me how to open it?
Thanks very much!
from multibench.
Hi, @TianhaoFu, I think you can open it with h5py, and the names of the files are in it, you can add .mp4 after the names~ For more details of the .csd file, you can refer to the multimodal SDK, but in a word, .csd just means the .h5 file~ So h5py is okay for it.
from multibench.
Related Issues (20)
- It will be great to have some tests to make sure this benchmark is runnable. HOT 27
- [QUESTION] Availability of trained models HOT 1
- What's the meaning of modalities in MUJOCO PUSH dataset? HOT 2
- Questions about the video encodings of the mosi and mosei datasets HOT 1
- Code to obtain features from raw data HOT 1
- Leaderboard
- Did you forget switch model train/eval state? HOT 1
- Errors running mmimdb examples HOT 2
- Models, data used in get_data.py for mmimdb missing
- Labels for CMU MOSEI HOT 1
- can you make multibench a python package?
- Question about relative robustness
- Extracting info from the H5 files HOT 6
- Suggestion about PyTorch version HOT 1
- Problem with code: os.environ['CUDA_VISIBLE_DEVICES'] = '0'
- The link to the raw dataset is broken HOT 4
- Info regarding preprocessed MOSEI
- Questions about the im.pk data & MFM HOT 2
- Question about the mortality label on im.pk file HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multibench.