Error in quantize vicuna-7b model from fp16 to int8 about onnxruntime HOT 4 OPEN

Did you try to see if it works with onnxruntime==1.18?

from onnxruntime.

Did you try to see if it works with onnxruntime==1.18?

I switch to onnxruntime==1.18, there it still return the same error when i try to pre-process

if i simply use quantize_dynamic, it works fine, but it fails to check_model

I set op_version as default(14) when export from PyTorch, my torch version is torch2.3-cu11.8.
Do you have any insights?

from onnxruntime.

Are you using the latest onnx package?

from onnxruntime.

Are you using the latest onnx package?

I have updated onnx to 1.16.1, onnxruntime to 1.18.0, than it succeed in quantization

howerver, when i tried to run it in onnxruntime, it report

from onnxruntime.