How to convert quantized ONNX model from Tensor-Oriented format to Operator-Oriented format? about onnxruntime HOT 5 CLOSED

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

from onnxruntime.

I have the same question. Hi @hoangtv2000 do you find the solution?

from onnxruntime.

@UsingtcNower and @hoangtv2000, you can run the onnxruntime offline-mode to optimize the model to Operator oriented: https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html#offline-mode. BTW, why do you want to do this?

from onnxruntime.

@UsingtcNower and @hoangtv2000, you can run the onnxruntime offline-mode to optimize the model to Operator oriented: https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html#offline-mode. BTW, why do you want to do this?

I want to use your method to run on our self-designed chip which runs efficiently by integer arithmetic-only format.

from onnxruntime.

I have the same question. Hi @hoangtv2000 do you find the solution?

You should use built-in static post-training quantization function of onnxruntime library quantize_static, remember to set quant_format to QuantFormat.QOperator if you want your ouput model has only QOperator, set per_channel to False if you want per-tensor quantization and activation_type & weight_type to QuantType.QUInt8 if you want your weight and activation tensors in uint8 format.

from onnxruntime.