inference 60 characters, cost 0.86 sencond on cpu, how to acceleration time about g2pw HOT 15 CLOSED

maybe you can convert model to onnx and use onnxruntime, check pr in #5

from g2pw.

谢谢，对g2pw代码进行了修改：dataloader也进行了相应修改，最后改成直接预测一整句话，速度大概在0.07～0.13s左右。也是在paddle下增加了g2pw选项。

from g2pw.

nice~ I have printed the time of G2PWOnnxConverter, the first time of using onnxruntime will be slow (it's onnxruntime's feature)

I will be appreciated if you can include your perfect improvement into paddlespeech after our onnxruntime version of g2pw merged, and I also looking forward to your pr if you are using paddlespeech TTS :)

input text:

我有长头发，我长高了，头发变得长长的，不想长大，你的头发很长

from g2pw.

谢谢，对g2pw代码进行了修改：dataloader也进行了相应修改，最后改成直接预测一整句话，速度大概在0.07～0.13s左右。也是在paddle下增加了g2pw选项。

@liuhuang31
Thanks for your response. The feature of "predicting whole sentence in one shot" sounds interesting.
Could I invite you to give a PR and become a contributor?
Or, if your time is not available, could you show a piece of codes to help us adding this feature by ourself?

from g2pw.

Thanks for your response, i want to give a PR, it is much convenient.

from g2pw.

@liuhuang31 Thank you! I am looking forward your PR.

from g2pw.

Hi, could you please tell me how to use G2PWOnnxConverter ? I didn't find it in the code.

from g2pw.

Hi, the newest code default use OnnxConverter model to predict, so just install the newest code and use it.

from g2pw.

Hi, could you please tell me how to use G2PWOnnxConverter ? I didn't find it in the code.

解决了嘛，没解决话是又遇到啥问题了呢～ @beyondguo

from g2pw.

@liuhuang31
Hi！
我是这几天直接pip安装的，应该就是最新版了。但感觉预测速度还是比较慢：

from g2pw import G2PWConverter
conv = G2PWConverter(style='pinyin', enable_non_tradional_chinese=True)

%timeit conv('然而，他红了20年以后，他竟退出了大家的视线。')

平均时长：

701 ms ± 30.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

不是初次加载模型，是每次跑都差不多这个时间。我现在的需求是对一个大语料库做注音，所以希望推理速度快一点。

from g2pw.

@beyondguo 你可以看看我的pull request, 我在旧的版本上，60个字能达到0.08-0.13秒。

from g2pw.

@liuhuang31
我刚刚下载了你的版本（https://github.com/liuhuang31/g2pW），测试了一下，一样的代码：

>>> %timeit conv('然而，他红了20年以后，他竟退出了大家的视线。')
<<< 1.44 s ± 39.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

然而更慢了，不知道我是哪里用错了吗[Lol]

from g2pw.

@beyondguo 应该没用对，等有时间再跟你讨论，我先忙工作事情，比较紧急

from g2pw.

确实慢，有办法做模型精简吗？onnx也慢。

from g2pw.

g2pw每次预测会先做句子分词，然后一句话可能会分成10次，那么就要调用10次去预测。
没有做模型精简，我是直接修改代码，直接一句话输入进去，只预测一次，所以速度很快。

可以参考之前的一个代码，大概逻辑是：循环生成数据变成只一次就生成预测数据；模型循环预测变成只一次调用模型。
去年写的代码了，有些细节有点忘了。
https://github.com/liuhuang31/g2pW

from g2pw.