Training Logs about freevc HOT 13 CLOSED

Sure.
Red: freevc (using data_utils_old.py)
Pink: freevc-s (using data_utils_old.py)
Orange: freevc (using data_utils.py)

from freevc.

I will try the new code and report back, thank you for the update.

from freevc.

I will try the new code and report back, thank you for the update.

Hmmm I found I forgot to de comment this line just now... But it might not be a big problem I think. And thank you for your interest in our work!

from freevc.

This vits issue is a good reference for your question.

from freevc.

Thanks for posting these.
Why does your loss/g/total climb?
mine decrease over the training run.
does this suggest a problem on my end?

from freevc.

my loss/g/total climb mainly because loss/g/fm increases faster than loss/g/mel decreases. I don't think it a problem if loss/g/total doesn't climb as mine, better listen to the generated audio in the validation part to check whether it goes ok.

from freevc.

My generated audio is coming out ok on training eval. Cleaning up as it goes, but I have noticed that the the actual conversion output of the model seems to be getting worse, comparing 245, 345, and 480K the output quality is getting worse. So the 245k model is very similar to target, and the content is clean. The 480k in comparison, has a lot of distortion, the content is less intelligible, and the pitch is off.

from freevc.

The code had some problem, I fixed it yesterday. Sorry to have wasted your time on problematic code. 😥

from freevc.

I will try the new code and report back, thank you for the update.

Hmmm I found I forgot to de comment this line just now... But it might not be a big problem I think. And thank you for your interest in our work!

Show this line be commented out?

from freevc.

I will try the new code and report back, thank you for the update.

Hmmm I found I forgot to de comment this line just now... But it might not be a big problem I think. And thank you for your interest in our work!

Show this line be commented out?

Yes it need to be commented out. (I don't even remember why I commented this line before uploading code...)
As in pytorch documentation:

In distributed mode, calling the set_epoch() method at the beginning of each epoch before creating the DataLoader iterator is necessary to make shuffling work properly across multiple epochs. Otherwise, the same ordering will be always used.

from freevc.

Does distributed mode relate to the case when gpus > 1

from freevc.

Does distributed mode relate to the case when gpus > 1

This code uses distributed mode no matter how many gpus used. No sure if set_epoch only affects distributed mode when gpus > 1 or all distributed mode settings.

from freevc.

Trying to resume the original model with the code changes you made. On previous runs the loss trend matched your original logs as far as increases and decreases goes. With the new code I am seeing a drop in FM loss, is this to be expected or a sign of a problem?

from freevc.