Comments (10)
@quangnh-2761 Hi! Yes, we've tried several experiments with end2end pipeline, but for some reason training on raw waveforms with mean-shifted terminal distribution was not stable enough. The samples you've listened to at the demo page were synthesized with the end2end model, which generates audios from the pure Gaussian noise, meaning starting denoising process from
from speech-backbones.
Sorry but I can not share samples from my datasets because of privacy policy, but I am experimenting with LJSpeech. I will send some samples when finish training.
from speech-backbones.
Thank you, I will check your work on fast solver. For my dataset, its language (Vietnamese) is single syllable and has no connection between words when pronounce so I think it's easier to learn and harder to identify errors, IMO model's size is kind of data dependent.
from speech-backbones.
Thanks for your informative response. I will reproduce on my data and try other end2end architectures to check if they help.
from speech-backbones.
Wavegrad large indeed helps, but interestingly outputs on my dataset with wavegrad base architecture are not distorted, I will experiment with LJSpeech to find what the difference is. Can I ask which noise schedule you tried, I was struggle to find suitable
from speech-backbones.
@quangnh-2761 Great! We've used the same noise schedule as in original WaveGrad work: params.py
.
from speech-backbones.
Thank you. For some reason wavegrad base with
from speech-backbones.
@quangnh-2761 Good! Can you share some audio samples to listen to?
from speech-backbones.
https://drive.google.com/drive/folders/1OCK_CD6nFmQZGPd_4hSdJLEN_ME1PxIU here are some samples from base and large models, trained with ~1k epochs. I think they are acceptable but still not perfect, I keep training to see if they improve (wavegrad 2 with base model can reach nearly 3.9 MOS, maybe because of other small details). A problem is that I can not do batch inference, because in the training phase I did random segment waveform to fit into memory, so model did not know how to deal with padding when inference, output would soon explode to infinity if I multiply with mask. Otherwise, generated audio might be affected by noise from padding frames.
Another thing is that I have to sample up to 1k steps to obtain good output. I have tried few steps schedule in wavegrad but it cant converge (I have divided
from speech-backbones.
@quangnh-2761 I see, good! Seems like hard-increase of parameters number really helps. To improve the inference speed I can suggest to try noise schedule grid-search for needed number of solver steps. Or you can add to your existing model and use our novel reverse SDE solver: https://arxiv.org/abs/2109.13821. It requires much lower number of steps to produce good quality samples. It is easy to implement, you don't need to re-train the model.
from speech-backbones.
Related Issues (20)
- Diffusion loss not decreasing HOT 1
- Generated outputs sound robotic in some cases! HOT 3
- Attention layer in GradTTS HOT 2
- How is `out_size` in `params` determined HOT 2
- Possibly missing __dict__ in the Projector class' constructor
- About the prior loss and MAS algorithm HOT 2
- Why does the BNE-PPG-VC model in your demo perform better than the pre-trained model given in the original paper? HOT 1
- Finetuning a Grad-TTS model on a small dataset? HOT 2
- Not able to generate audio using libritts of as good quality as using ljspeech HOT 10
- about diffVC on Mandarin datasets HOT 4
- Typo in some equations in GradTTS paper HOT 4
- Model training question HOT 4
- support for bigvgan HOT 2
- A bug in model/tts.py HOT 1
- Two questions about DiffVC
- Not possible to build HOT 1
- GradTTS device compatibility
- Different Implementation of Diffusion Model
- mels_mode generation
- Generated Samples are noisy
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from speech-backbones.