请问nn-Meter只支持CNN模型吗？只支持端侧设备不支持计算机服务器设备吗？ about nn-meter HOT 3 CLOSED

Hi,

For the first question, nn-Meter only support CNN model by now, and the support of transformer is in our plan of next step.
For the second one, nn-Meter presupposes that the latency is additive, which means the model latency can be estimated by the sum of operator latency. We tested several server devices such as Intel VNNI, A100, V100, P100, and the results shows inference on these device follows the additivity. We didn't test other server devices by now.

Best regards,
Jiahang

from nn-meter.

好的十分感谢。对于第二个问题，是否有nn-Meter Builder对服务器设备定制predictor的教程或者文档，我目前只看到有对于端侧设备定制predictor的相关文档，如果有的话能否告诉一下在哪里，如果没有以后会出相关教程吗？

from nn-meter.

Hi,

I hope this message finds you well. I understand that there is no separate documentation available regarding the server latency predictor build, however, the pipeline for building both the server latency predictor and edge latency predictor are similar. Hence, you may refer to the documentation of the predictor builder for building the server latency predictor. The only difference lies in the construction of the backend on the server to obtain the latency number.

I hope this information is helpful and clarifies any confusion you may have had. If there are any further questions or concerns, please do not hesitate to reach out.

Best regards,
Jiahang

from nn-meter.