[Feature request] Add text-to-speech with SpeechT5 about transformers.js HOT 6 CLOSED

Done: huggingface/transformers#22487

from transformers.js.

Woah 🤯 This definitely sounds do-able! I'll look into it (and hopefully add it quite soon 💪 )

from transformers.js.

Yeah expressiveness/non-roboticness is the main factor for me. And next is inference speed. Size is probably not a big issue for my use cases - anything under 500mb is fine.

from transformers.js.

I've been looking into it more today, but it seems as though HF does not support text-to-speech in the pipeline function? It also appears that optimum doesn't support text-to-speech as a task (needed to convert to ONNX format).

Fortunately, the spaces demo you sent above includes some code I can use for testing (https://huggingface.co/spaces/Matthijs/speecht5-tts-demo/blob/main/app.py).

If you want, you could even open up a feature request / PR on the main transformers branch to add this.

from transformers.js.

Just came across this:

• https://github.com/rhasspy/piper
• https://rhasspy.github.io/piper-samples/

The audio quality here seems quite good for the model size.

from transformers.js.

@josephrocca Do you have any favorite model or do you use different models for different tasks?

I am very much looking forward to this 🙈

I guess the decision matrix would contain:

• Number of available languages
• Ability to handle emotive text
• Size of the model
• Transformers support

Just tested Bark: huggingface/transformers#23036

I like the ability to add emotion, just funny that it suddenly changed the voice/gender too 😅

from transformers.js.