Question regarding pitch models (Reflow vs DDPM) about diffsinger HOT 3 OPEN

In our experiments, Reflow outperforms DDPM a lot on all types of datasets, especially for expressive ones. Furtherly, Reflow can hold worse (automatic) labels and more data/speakers. Thus, your case seems unexpected, and there may be other cause before blaming Reflow itself.

There are many factors which influence the pitch performance, like your training steps, your labels, your combination of variance modules, your choice of speedup/steps, or even your method of testing. For research purposes, I recommend reading the accuracy metrics and validation plots on TensorBoard, or using the CLI inference script in this repository. (There were cases where someone put a multi-speaker pitch model into OpenUTAU with misconfigured YAML, and the software produced wrong results without any error reports.)

Therefore, if you still cannot figure out the reason, please provide more details, for example:

• Your configuration file
• Accuracy and plots on TensorBoard, respectively
• Have you really controlled all variables?
• how did you do the tests above?

from diffsinger.

Hello.

Thank you for your response, it is much appreciated.

After doing more experiments, and also comparing the result with inference via command, ReFlow outperforms DDPM a lot. For some reason, the result is very different when it is generated in OpenUTAU. I did make sure the config for OpenUTAU was configured correctly, though. I wonder why it is. My apologies for blaming Reflow at first, when the issue is most likely OpenUTAU, or onnx exporting wrongly.

Thank you in advance.

from diffsinger.

A possible debugging method is to freeze one speaker into the model and test it in OpenUTAU. OpenUTAU encountered problems in multi-speaker cases for many times before. There are possible bugs that the result seemed okay but actually the model did not run correctly at all.

Also, it is not likely an ONNX bug if you exported the model with PyTorch 1.13 successfully, because there are other people who are using multi-speaker pitch models in OpenUTAU and can get reasonable results.

Maybe you still need to check the configuration carefully. OpenUTAU has too many undefined behaviors that can break the results without any error reporting, and only if you do everything as it expects that you can get the right outputs.

from diffsinger.