Comments (2)
chevalierNoir
commented on September 10, 2024
1
Hi,
Table 1 are the result of CTC decoding. Our best result is obtained with S2S after training all iterations. But we did not find S2S always outperforms CTC in the entire process. Specifically for CTC decoding, we use 4-gram language model for decoding, similar to the original HuBERT. Besides, we use phone as the target unit during fine-tuning. You may find more details about the hyperparameters for CTC decoding in section A.4 of the paper.
from av_hubert.
jxzhanggg
commented on September 10, 2024
Thanks for your quick reply! I'll keep following the rest iters to see if the final result is right.
from av_hubert.
Related Issues (20)
- ImportError: cannot import name 'metrics' from 'fairseq' (unknown location) HOT 5
- preparation-cnn_face_detector HOT 1
- Question about the use of CMUDict in CTC finetuning HOT 3
- issue during 1st iteration of pretraining HOT 1
- How to train a LM used for decoding HOT 2
- Error in step 2 of preprocessing, what values to put in ${rank} and ${nshard} HOT 5
- non-deterministic results when decoding with noises HOT 5
- Fixing Colab
- How to finetune on AVSR setting? HOT 3
- What's the difference between A/MFCC→A and A/MFCC→AV in the paper? HOT 1
- Value expected for ${layer}-th transformer layer of a trained AV-HuBERT model saved at ${ckpt_path} HOT 1
- Format of pretrain data in the LRS3 dataset HOT 1
- Cannot register duplicate model (av_hubert) HOT 1
- Extraction of features with AV HuBERT HOT 4
- A problem of traning a new model HOT 2
- Release of clustering models HOT 3
- How to decode without any label files HOT 2
- Request for Base model pre-trained on multi-lingual data HOT 1
- Error loading AVSR model HOT 3
- How to adapt or train AV-HuBERT for other languages? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from av_hubert.