modphoenix / beyond-human Goto Github PK

The main problem I'm currently struggling with is getting real-time voice generation of sufficient quality. Unfortunately, there isn't a ready-to-use library in Rust that works on all platforms.

I don't have enough knowledge in this area. I tried to rewrite this implementation from Python to Rust, but couldn't find a replacement for https://github.com/bootphon/phonemizer.

I have been playing around with https://github.com/ndarilek/tts-rs and it works, but the quality is not good enough. I also ran into the same issue ndarilek/tts-rs#40 when trying to split the text into sentences and voice them to avoid waiting for the generation of large chunks of text.

Here are some examples in Python:

https://github.com/as-ideas/ForwardTacotron
https://github.com/coqui-ai/TTS
https://github.com/as-ideas/TransformerTTS
https://github.com/rendchevi/nix-tts

To me that would be valuable because language models can provide levels of nuance that is very difficult to achieve by humans.

For the reference, here analogous thing https://futurimedia.com/radiogpt/. Though who knows how much of it is "pre-rendered" marketing.