Model optimization fails with Protobuf serialization failed error about onnxruntime HOT 6 CLOSED

Hello, give the following commands a try:

cd onnxruntime/onnxruntime/python/tools/transformers/
python3 optimizer.py --input /path/to/<filename>.onnx --output /path/to/<filename>.onnx --model_type gpt2 --num_heads <number of attention heads> --hidden_size <attention hidden size> --use_external_data_format --opt_level 0

You can also try the convert_to_onnx tool for Llama, which will convert & optimize in the same script.

Thanks @kunal-vaishnavi for the suggestions :)

from onnxruntime.

Could you provide the stack trace for protobuf serialization error?

from onnxruntime.

Could you provide the stack trace for protobuf serialization error?

Traceback (most recent call last):
File "/scratch/tuhinp/onnxruntime/onnxruntime/python/tools/transformers/optimizer.py", line 610, in
main()
File "/scratch/tuhinp/onnxruntime/onnxruntime/python/tools/transformers/optimizer.py", line 573, in main
optimizer = optimize_model(
File "/scratch/tuhinp/onnxruntime/onnxruntime/python/tools/transformers/optimizer.py", line 379, in optimize_model
temp_model_path = optimize_by_onnxruntime(
File "/scratch/tuhinp/onnxruntime/onnxruntime/python/tools/transformers/optimizer.py", line 204, in optimize_by_onnxruntime
onnxruntime.InferenceSession(onnx_model, sess_options, providers=providers, **kwargs)
File "/scratch/tuhinp/miniconda3/envs/x/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/scratch/tuhinp/miniconda3/envs/x/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 463, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidProtobuf: [ONNXRuntimeError] : 7 : INVALID_PROTOBUF : Protobuf serialization failed.

from onnxruntime.

Thanks @carzh and @kunal-vaishnavi for the suggestion. This command works for codellama.

I tried to optimize Qwen/Qwen1.5-7B-Chat onnx model with the same optimizer.py script but getting "Segmentation fault".
I used the same command as mentioned above
python3 optimizer.py --input /path/to/<filename>.onnx --output /path/to/<filename>.onnx --model_type gpt2 --num_heads <number of attention heads> --hidden_size <attention hidden size> --use_external_data_format --opt_level 0

Can you please help me on this?

from onnxruntime.

Can you clone ORT from the main branch and try again? I can run the ORT transformer optimizer successfully with the following steps.

git clone https://github.com/microsoft/onnxruntime
cd onnxruntime/onnxruntime/python/tools/transformers
optimum-cli export onnx --model Qwen/Qwen1.5-7B-Chat ./qwen1.5 --no-post-process
mkdir -p ./qwen1.5-opt
python3 optimizer.py --input ./qwen1.5/model.onnx --output ./qwen1.5-opt/model_opt.onnx --model_type gpt2 --num_heads 32 --hidden_size 4096 --use_external_data_format --opt_level 0

from onnxruntime.

You can also run onnx.check_model to get more information on the nature of protobuf issue.

from onnxruntime.