esnya / hf-rvc Goto Github PK

hi! we just want to do plain old conversion for RVC - take an input wav and spit out a converted file - using the command line - like we can in so-vits. is there any easy way to do this? happy to help!

Hello,

Maybe it's just me who can't figure out how to use this package, but would it be possible to get a little more detailed documentation?

I don't want to bother you with something that's surely not an "issue", but I can't use the models I trained with the web-ui version of rvc.

For example : in the basic usage, you need to specify the feature extractor that will be used. But I can't find any file named preprocess_config.json in the files of my models on the web-ui version, or in the files of the web-ui version at all.

Am I missing something?

Is it possible using this library to convert an audio file through a model into the desired vocal?

Hi!

First, thank you so much for this repo, @esnya

How would you recommend training with hf-rvc?

I tried a few different hubert checkpoints including this and this and some of these and an official fairseq base, and a few random RVC models from huggingface, I always get an error

Missing key(s) in state_dict: "hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original0", "hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original1". 
Unexpected key(s) in state_dict: "encoder.pos_conv.0.weight_g", "encoder.pos_conv.0.weight_v". 

Tried to fix it with edited mapping

r"encoder.pos_conv.0.weight_g": r"hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original0",
r"encoder.pos_conv.0.weight_v": r"hubert.encoder.pos_conv_embed.conv.parametrizations.weight.original1",

I faced another issue:

RuntimeError: Error(s) in loading state_dict for SynthesizerTrnMs256NSFsid:
	size mismatch for enc_p.emb_phone.weight: copying a param with shape torch.Size([192, 768]) from checkpoint, the shape in current model is torch.Size([192, 256]).

turns out it was a v2 model, tried a v1 model and it worked.
changed emb_phone size to 768 and it converted v2 models, but v2 fails on inference:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (112x256 and 768x192)