Giter Club home page Giter Club logo

optimizer's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

optimizer's Issues

CleanUnusedInitializersAndNodeArgs warning still issued after onnxoptimizer run

I converted a model from PyTorch to ONNX. The ONNX model produces > 100 warnings of the type:

2023-01-10 20:26:41.949523885 [W:onnxruntime:, graph.cc:3487 CleanUnusedInitializersAndNodeArgs] Removing initializer '1026'. It is not used by any node and should be removed from the model.

I used the following code to run onnxoptimizer on the model:

import onnx
import onnxoptimizer

model = onnx.load('model.onnx')
passes = onnxoptimizer.get_fuse_and_elimination_passes()
model = onnxoptimizer.optimize(model, passes)
onnx.save(model, 'model-opt.onnx')

I'm still seeing the same warnings when I load the optimized model. Here are the package versions I'm using:

>>> onnx.__version__, onnxoptimizer.__version__
('1.13.0', '0.3.6')

Need more explicit instruction on installing the Protobuf and ONNX optimizer.

Hi,

I tried installinig the Protobuf via both building from source and apt. When I tried to build ONNX optimizer from source, I got problems related to Protobuf. How do I specify Protobuf for installing ONNX optimizer? Thank you.

root@3235dac98ab6:/mnt# pip install -e .
Obtaining file:///mnt
Requirement already satisfied: onnx in /tmp/onnx (from onnxoptimizer==0.2.6) (1.9.0)
Requirement already satisfied: protobuf in /usr/local/lib/python3.8/dist-packages (from onnx->onnxoptimizer==0.2.6) (3.17.3)
Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.8/dist-packages (from onnx->onnxoptimizer==0.2.6) (1.21.0)
Requirement already satisfied: six in /usr/local/lib/python3.8/dist-packages (from onnx->onnxoptimizer==0.2.6) (1.16.0)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.8/dist-packages (from onnx->onnxoptimizer==0.2.6) (3.10.0.0)
Installing collected packages: onnxoptimizer
  Running setup.py develop for onnxoptimizer
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/mnt/setup.py'"'"'; __file__='"'"'/mnt/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /mnt/
    Complete output (105 lines):
    running develop
    running build_py
    running create_version
    running cmake_build
    CMake Error at /usr/share/cmake-3.16/Modules/FindProtobuf.cmake:469 (file):
      file STRINGS file "/opt/conda/include/google/protobuf/stubs/common.h"
      cannot be read.
    Call Stack (most recent call first):
      third_party/onnx/CMakeLists.txt:172 (find_package)
    
    
    CMake Error at /usr/share/cmake-3.16/Modules/FindProtobuf.cmake:475 (math):
      math cannot parse the expression: " / 1000000": syntax error, unexpected
      exp_DIVIDE (2).
    Call Stack (most recent call first):
      third_party/onnx/CMakeLists.txt:172 (find_package)
    
    
    CMake Error at /usr/share/cmake-3.16/Modules/FindProtobuf.cmake:476 (math):
      math cannot parse the expression: " / 1000 % 1000": syntax error,
      unexpected exp_DIVIDE (2).
    Call Stack (most recent call first):
      third_party/onnx/CMakeLists.txt:172 (find_package)
    
    
    CMake Error at /usr/share/cmake-3.16/Modules/FindProtobuf.cmake:477 (math):
      math cannot parse the expression: " % 1000": syntax error, unexpected
      exp_MOD (2).
    Call Stack (most recent call first):
      third_party/onnx/CMakeLists.txt:172 (find_package)

fuse_matmul_add_bias_into_gemm: not working for just MatMul

I'm coming across an issue with the fuse_matmul_add_bias_into_gemm when there is just a matmul node and no add or bias node after. I've written a quick example of what the issue is below. What I am expecting (correct me if I'm wrong) is the MatMul node to be converted to a GEMM node instead.

from onnx import helper
from onnx import checker, helper, ModelProto, TensorProto, GraphProto, NodeProto, shape_inference
import onnxoptimizer

matmul = helper.make_node("MatMul", ["X", "Y"], ["Z"])
add = helper.make_node("Add", ["Z", "B"], ["A"])
graph = helper.make_graph(
    [matmul],
    "test",
    [helper.make_tensor_value_info("X", TensorProto.FLOAT, (32, 10)),
     helper.make_tensor_value_info("Y", TensorProto.FLOAT, (10, 16))],
    [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (32, 16))]
)

model = helper.make_model(graph)
optimized_model = onnxoptimizer.optimize(
    model, passes=["fuse_matmul_add_bias_into_gemm"])

print(optimized_model)

assert len(list(optimized_model.graph.node)) == 1
assert optimized_model.graph.node[0].op_type == "Gemm"

Feature Request: Add optimizations for ConvTranspose

Hi

Currently onnx optimizer has several passes for Conv op fusion like fuse_add_bias_into_conv. However, it seems like their support is only limited to Conv nodes. Would be nice if ConvTranspose is also supported.

Install issue on Nvidia Xavier NX

Hi,

I tried to install onnxoptimizier on Nvidia Xavier NX via pip3 install onnxoptimizier, however it showed error "ERROR: Could not find a version that satisfies the requirement onnxoptimizier (from versions: none) ERROR: No matching distribution found for onnxoptimizier".

I also checked on Pypi, but there is no supported wheel for Nvidia Xavier NX.

Is there any ways for me to install it on my Xavier NX?

Thanks!

Multiple ONNX optimizer tests fail on big-endian

These tests were run on s390x. s390x is big-endian architecture.

Example failure log from pytest:

_________________________________________________ TestOptimizer.test_fuse_pad_into_conv_1d _________________________________________________

self = <optimizer_test.TestOptimizer testMethod=test_fuse_pad_into_conv_1d>

    def test_fuse_pad_into_conv_1d(self):  # type: () -> None
        pad = helper.make_node(
            "Pad",
            ["X", "Pads"],
            ["P"],
            mode="constant"
        )
        conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
        graph = helper.make_graph(
            [pad, conv],
            "test",
            [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 30)),
             helper.make_tensor_value_info("Pads", TensorProto.INT64, (6,)),
             helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 32))],
            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1))],
            [helper.make_tensor("Pads", TensorProto.INT64,
             dims=(6,),
             vals=np.array([0, 0, 1, 0, 0, 1]).astype(np.int64).tobytes(),
             raw=True)])
        optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
    
        assert len(list(optimized_model.graph.node)) == 1
        assert optimized_model.graph.node[0].op_type == "Conv"
        assert optimized_model.graph.node[0].attribute[0].name == "pads"
>       assert list(optimized_model.graph.node[0].attribute[0].ints) == [1, 1]
E       AssertionError: assert [720575940379...7594037927936] == [1, 1]
E         At index 0 diff: 72057594037927936 != 1
E         Use -v to get the full diff

optimizer_test.py:1038: AssertionError

Notice the big integers: 720575940379. Viewing the number in hex shows an endian swapping issue.

Attached is the patch file for fixing the issue in the testcase helper.py code. The helper.py code needs to account for generating the raw data on a big endian architecture while the ONNX internals always expects little endian values.

A pull request can be submitted if needed.

Attaching corrected patch below.

How to add empty input to a node

hi~Now I am adding a fuse pass

Now I have encountered a problem, how to add an empty input to a node, Node->addInput seems to be unable to add an empty input

Call the following api in python

node_def = helper.make_node(
    'Attention', # node name
    ['p2o.Add.6', 'qkv_weights', 'qkv_bias', '', '', 'extra_add_qk'],
    # ['p2o.Add.6', 'qkv_weights', 'qkv_bias', 'mask_index'],
    ['Y'], # outputs
    name='Attention',
    domain="com.microsoft",
    # attributes
    num_heads=9,
    unidirectional=0,
    )

You can create an Attention node with empty fourth and fifth inputs:
image

How to create it in C++,thanks a lot~

Problem using onnx.optimizer on model converted through tf2onnx with --opset 10

I am converting frozen tensorflow models using "tf2onnx --fold_const". The resulting onnx model can be optimized using onn.optimizer.optimize method with different optimization passes.

However when I use "--opset 10" option in the tf2onnx conversion tool, the resulting onnx model fails optimization with the following message (default is opset 8):

optimized_model_str = C.optimize(model_str, passes)
IndexError: invalid unordered_map<K, T> key

ONNX version is 1.6.0
tf2onnx versions (tried 1.5.4 and 1.5.6)

The issue is the same regardless of whether I run it in CPU or GPU.

Can somebody help me figure out why the optimization fails with the opset 10 model?

optimizer support for python 3.11

It seems like the optimizer currently does not support python 3.11.
I was wondering if you plan on supporting that since there are quite some performance improvements compared to prior python versions.

Optimization on valid ONNX models fail with obscure message

Bug Report

Some valid ONNX models fail during optimization phase. The checker is not reporting any problem on the model though.

System information

  • OS Platform: Linux Ubuntu 18.04
  • ONNX version 1.6.0
  • Python version 3.7.6
  • CMake version: 3.14.0
  • Protobuf version: 3.9.1

Reproduction instructions

import onnx
import argparse
from onnx import optimizer
import onnx.utils
import os

# Parse arguments
parser = argparse.ArgumentParser(description='')
parser.add_argument("--input", required=True, help="Input ONNX model")
args = parser.parse_args()

# Load ONNX model
print("Processing model:", args.input)
model = onnx.load(args.input)
onnx.checker.check_model(model)

# Remove dangling nodes and save new model
optimized_model = optimizer.optimize(model, ['eliminate_deadend'])
outputFile = os.path.splitext(args.input)[0] + "-opt.onnx"
onnx.save(optimized_model, outputFile)

Example ONNX model triggering the the issue is attached.
posenet_mobilenet_float_075_1_default_1.onnx.gz

Expected behavior

An optimized model is created

Actual output

python optimize_model.py   --input posenet_mobilenet_float_075_1_default_1.onnx 
Processing model: posenet_mobilenet_float_075_1_default_1.onnx
Traceback (most recent call last):
  File "optimize_model.py", line 26, in <module>
    optimized_model = optimizer.optimize(model, ['eliminate_deadend'])
  File "/home/daz/anaconda3/envs/atonn/lib/python3.7/site-packages/onnx/optimizer.py", line 55, in optimize
    optimized_model_str = C.optimize(model_str, passes)
IndexError: _Map_base::at

fuse_consecutive_squeezes issue: consecutive negtive axes

issue:

// Before:
// X is a tensor with shape=[1, 5000, 1, 1]
// Y = Squeeze(X, axes=[-1]) -> shape=[1, 5000, 1]
// Z = Squeeze(Y, axes=[-1]) -> shape=[1, 5000]
// After:
// Z = Squeeze(X, axes=[-1, 0]) -> shape=[5000, 1]

it should be
Z = Squeeze(X, axes=[-2, -1]) -> shape=[1, 5000]
or
Z = Squeeze(X, axes=[2, 3]) -> shape=[1, 5000]

codes resulted in line 93:
ret.push_back(i + prev_num + 1);
it's only right for positive axes values

Support for python3.6

请问能让v0.3.0+支持python3.6吗?现在要python3.7+才能使用v0.3.0+,谢谢。
期待回复!

Add cross-platform C API

It would be really cool if this had a C API and hence was easily available in other languages. E. g. C#.

onnx merge bn to fc?

@
Now have merge bn and conv to simplify, but don't have bn to fc,

Could you support it in the future ?

Unable to install on Mac M1

Since this package is not available through pip, I tried to build onnxoptimizer and unfortunately it fails with this error error.log

Is there any plan to add support for Apple M1?

Some tests are broken by the new squeeze op

==================================================================================== short test summary info =====================================================================================
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_add_bias_into_conv_use_conv_shape - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for ope...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_add_bias_into_conv_use_move_constant - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for ...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_add_bias_into_conv_use_weight_shape - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for o...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_add_bias_into_conv_with_scalar_bias - onnx.onnx_cpp2py_export.checker.ValidationError: Node () has input size 1 not in ra...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_consecutive_squeezes - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for operator Squeeze
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_consecutive_squeezes_default - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for operator...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_consecutive_squeezes_multi_uses - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for opera...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_consecutive_squeezes_random - onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: axes for operator ...
FAILED onnxoptimizer/test/optimizer_test.py::TestOptimizer::test_fuse_reduction_unsqueeze - onnx.onnx_cpp2py_export.checker.ValidationError: Node () has input size 1 not in range [min=2, max=2].
================================================================================== 9 failed, 68 passed in 1.97s ==================================================================================

The cause is the new squeeze op makes "axes" as an input instead of an attribute

skip custom ops

Can I skip custom ops in optimization?
My onnx model is export by pytorch, and I do not have custom_lib, so the simplification always fails.
Is there a solution?

[Feature] Separate graph rewriting and constant folding

For op fusion (like the fusion of conv and bn), we have implemented a "small onnxruntime" in tensor.h. It increases the workload (the more fusion we want to do, the more op we need to implement), and brings many problems (#6, #8, onnx/onnx#2677). However, as we know, onnx itself is not designed to infer onnx ops. It is unwise to take the effort to maintain an "embedded runtime" in the presence of onnxruntime.

In my opinion, we should drop the "embedded runtime". Instead, we should only rewrite the graph, and then call onnxruntime library to fold the constants. In this way, we will not need tensor.h or another tensor library in optimizer anymore.

For example, to fuse Add(Add(x, 1), 2), instead of calculating the result of Add(1, 2) in onnx-optimizer itself, we can just rewrite the graph to Add(x, Add(1, 2)), and call onnxruntime to fold Add(1, 2) to 3.

It is also the way of tensorflow built-in optimizer.

Regression: eraseOutput: Assertion `outputs_[i]->uses().empty()` failed.

This is a regression from running onnxoptimizer. Calling optimize from onnx 1.7 with the same model and same passes did not throw any error.

Traceback (most recent call last):
  File "repro.py", line 21, in <module>
    onnxoptimizer.optimize(m, passes)
  File "onnxoptimizer/onnxoptimizer/__init__.py", line 50, in optimize
    optimized_model_str = C.optimize(model_str, passes)
RuntimeError: onnxoptimizer/third_party/onnx/onnx/common/ir.h:1291: eraseOutput: Assertion `outputs_[i]->uses().empty()` failed.

Script to repro:

import onnx

model_str = b'\x08\x06\x12\x07pytorch\x1a\x031.9:\xe5\x02\n\'\x12\x011"\x08Constant*\x18\n\x05value*\x0c\x10\x07J\x08\x05\x00\x00\x00\x00\x00\x00\x00\xa0\x01\x04\n \x12\x012"\x08Constant*\x11\n\x05value*\x05\x10\tJ\x01\x01\xa0\x01\x04\n\xd1\x01\n\x011\n\x012\n\x03x.1\x12\x013"\x04Loop*\xba\x01\n\x04body2\xae\x01\n\x1a\n\x04x.11\n\x03i.1\x12\x017\x1a\x05Add_0"\x03Add\n\x1c\n\x012\x12\x018\x1a\nIdentity_1"\x08Identity\x12\x11torch-jit-export1Z\r\n\x03i.1\x12\x06\n\x04\x08\x07\x12\x00Z\x0e\n\x04cond\x12\x06\n\x04\x08\t\x12\x00Z\x1a\n\x04x.11\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03b\x0b\n\x018\x12\x06\n\x04\x08\t\x12\x00b\x17\n\x017\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03\xa0\x01\x05\x12\x10torch-jit-exportZ\x19\n\x03x.1\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03b\x17\n\x013\x12\x12\n\x10\x08\x07\x12\x0c\n\x02\x08\x01\n\x02\x08\x02\n\x02\x08\x03B\x02\x10\t'
m = onnx.load_from_string(model_str)

import onnxoptimizer

passes =  ['fuse_consecutive_transposes',
            'eliminate_nop_transpose',
            'fuse_transpose_into_gemm',
            'lift_lexical_references',
            'split_predict']
from onnx import optimizer
onnx.optimizer.optimize(m, passes)  # pass
onnxoptimizer.optimize(m, passes)  # fail

simplify can't find graph

example:
`
torch.onnx.export(torch_model, (batch1,{}), my_location, opset_version=13, input_names=my_input_names,
output_names = ['output'],dynamic_axes=dynamic_axes)
model_loaded = onnx.load(my_location)
model_loaded, check = simplify(model) # raises error 'MyModel' object has no attribute 'graph'

`

IndexError: Input 2728 is undefined!

I'm trying to run a onnx model on Caffe2 runtime, and I get a error when I try to use caffe2 backend to prepare model with this code

import caffe2.python.onnx.backend as backend
model = onnx.load("model.onnx") 
rep = backend.prepare(model)

The error I get is

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-21-4393df7eef2b> in <module>()
      1 # run onnx inference
----> 2 rep = backend.prepare(model)
      3 # raw_onnx_outputs = rep.run(W)

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/caffe2/python/onnx/backend.py in prepare(cls, model, device, raw_values_dict, **kwargs)
    711         device_option = get_device_option(Device(device))
    712 
--> 713         init_net, predict_net = cls._onnx_model_to_caffe2_net(model, device, opset_version, False)
    714 
    715         if raw_values_dict:

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/caffe2/python/onnx/backend.py in _onnx_model_to_caffe2_net(cls, onnx_model, device, opset_version, include_initializers)
    874         device_option = get_device_option(Device(device))
    875 
--> 876         onnx_model = onnx.utils.polish_model(onnx_model)
    877         init_model = cls.optimize_onnx(onnx_model, init=True)
    878         pred_model = cls.optimize_onnx(onnx_model, predict=True)

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/onnx/utils.py in polish_model(model)
     19     onnx.helper.strip_doc_string(model)
     20     model = onnx.shape_inference.infer_shapes(model)
---> 21     model = onnx.optimizer.optimize(model)
     22     onnx.checker.check_model(model)
     23     return model

~/.pyenv/versions/anaconda3-5.3.1/lib/python3.7/site-packages/onnx/optimizer.py in optimize(model, passes, fixed_point)
     53         optimized_model_str = C.optimize_fixedpoint(model_str, passes)
     54     else:
---> 55         optimized_model_str = C.optimize(model_str, passes)
     56 
     57     return onnx.load_from_string(optimized_model_str)

IndexError: Input 2728 is undefined!

Also, I don't get any errors when I check the model using the checker

onnx.checker.check_model(model)

Does anyone has any idea on what might be the issue?

optimize passes of mean ?

Can knowledge optimize the meaning of each parameter
eliminate_deadend eliminate_duplicate_initializer eliminate_identity eliminate_if_with_const_cond eliminate_nop_cast eliminate_nop_dropout eliminate_nop_flatten eliminate_nop_monotone_argmax eliminate_nop_pad eliminate_nop_transpose eliminate_unused_initializer extract_constant_to_initializer fuse_add_bias_into_conv fuse_bn_into_conv fuse_consecutive_concats fuse_consecutive_log_softmax fuse_consecutive_reduce_unsqueeze fuse_consecutive_squeezes fuse_consecutive_transposes fuse_matmul_add_bias_into_gemm fuse_pad_into_conv fuse_transpose_into_gemm lift_lexical_references nop split_init split_predict

ONNX optimizer - fuse_consecutive_reduce_unsqueeze crashes

Bug Report (copy from onnx/onnx#3229)

Is the issue related to model conversion?

No

Describe the bug

ONNX optimizer - fuse_consecutive_reduce_unsqueeze crashes

System information

  • OS Platform and Distribution (e.g. Linux Ubuntu 16.04): Windows
  • ONNX version (e.g. 1.7): 1.8.0
  • Python version: 3.6.8
  • GCC/Compiler version (if compiling from source): N/A
  • CMake version: N/A
  • Protobuf version: 3.8.0
  • Visual Studio version (if applicable): N/A

Reproduction instructions

  • Describe the code to reproduce the behavior.
import onnx
model = onnx.load('model.onnx')
model_optimized = onnx.optimizer.optimize(model, ["fuse_consecutive_reduce_unsqueeze"])
...

Expected behavior

Except 1. no crash and 2. fuse for consecutive reduce and unsqueeze op in the attached model

Notes

Any additional information

fuse_matmul_add_bias_into_gemm not working with batch size

Hi,

When using fuse_matmul_add_bias_into_gemm I expect that even with batch size the layers will fuse. Apparently, this is not supported. I can't see what is the reason for this. If there is a problem with more then one batch, the fuse can happen at least when the batch size dim is 1.

Here is the example code to create this issue (heavily based on #58):

    from onnx import helper
    from onnx import checker, helper, ModelProto, TensorProto, GraphProto, NodeProto, shape_inference
    import onnxoptimizer

    matmul = helper.make_node("MatMul", ["X", "Y"], ["Z"])
    add = helper.make_node("Add", ["Z", "B"], ["A"])
    graph = helper.make_graph(
        [matmul, add],
        "test",
        [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 32, 10)),

         ],
        [helper.make_tensor_value_info("A", TensorProto.FLOAT, (1, 32, 16))],
        [helper.make_tensor("B", TensorProto.FLOAT, tuple([16]), np.ones([1, 16])),
         helper.make_tensor("Y", TensorProto.FLOAT, (1, 10, 16), np.ones([1, 10, 16])), ]
    )

    model = helper.make_model(graph)
    onnx.save(model, "gg.onnx")
    optimized_model = onnxoptimizer.optimize(
        model, passes=["fuse_matmul_add_bias_into_gemm"])
    onnx.save(optimized_model, "gg1.onnx")

    print(optimized_model)

    assert len(list(optimized_model.graph.node)) == 1
    assert optimized_model.graph.node[0].op_type == "Gemm"

Lost 'external_data' field of tensor after doing optimize

Describe the bug

For a model saved with save_as_external_data=True and load the model into memory with load_external_data=False, if do optimize on this model, tensors in the resulting model will lose 'external_data' and 'data_location' fields (therefore in latter processes could not find the data of the tensor).

Reproduction instructions

  • Example code:
import onnxoptimizer

import numpy as np
import onnx
from onnx import  helper, TensorProto, numpy_helper

#======== 1. create an onnx model
dtype = np.float32
data_Z1 = np.ones((3, 3), dtype)
Z1 = helper.make_tensor(name = 'Z1', data_type = TensorProto.FLOAT, dims = data_Z1.shape, vals = data_Z1.flatten().astype(dtype).tobytes(), raw=True)

V = helper.make_tensor_value_info('V', TensorProto.FLOAT, [3, 3])
Z = helper.make_tensor_value_info('Z', TensorProto.FLOAT, [3, 3])

node_sum = helper.make_node(
    'Sum',                  # name
    ['Z1', 'V'], # inputs
    ['Z'],                  # outputs
)
# Create the graph (GraphProto)
graph_def = helper.make_graph(
    [node_sum],        # nodes
    'test-model',      # name
    [V],  # inputs
    [Z],  # outputs
    initializer = [Z1],
)
model_def = helper.make_model(graph_def, producer_name='onnx-example')


#======== 2. save
model_path = 'model_sep.onnx'
onnx.save_model(model_def, model_path
                , save_as_external_data=True
                , size_threshold=0)

#======== 3. do optimize

passes = [
    "eliminate_deadend",
    "eliminate_identity",
    "eliminate_nop_dropout",
    "eliminate_nop_monotone_argmax",
    "eliminate_nop_pad",
    "eliminate_nop_transpose",
    "eliminate_unused_initializer",
    "fuse_add_bias_into_conv",
    "fuse_bn_into_conv",
    "fuse_consecutive_concats",
    "fuse_consecutive_log_softmax",
    "fuse_consecutive_reduce_unsqueeze",
    "fuse_consecutive_squeezes",
    "fuse_matmul_add_bias_into_gemm",
    "fuse_pad_into_conv",
    "fuse_transpose_into_gemm",
    "fuse_consecutive_transposes"]
# passes=[]  # if the passes be empty, the issue will still appear

# (1). if load_external_data=False, after optimize, will lost 'external_data' and 'data_location' fields
print('for load_external_data=False:')
model_cf = onnx.load(model_path, load_external_data=False)

for tensor in model_cf.graph.initializer:
    tensor_field_names = [field[0].full_name for field in tensor.ListFields()]
    print('fields before opt:', tensor_field_names) # for observe, will get 5 filed names, including 'external_data' and 'data_location'

model = onnxoptimizer.optimize(model_cf, passes)

for tensor in model.graph.initializer:
    tensor_field_names = [field[0].full_name for field in tensor.ListFields()]
    print('fields after opt:', tensor_field_names) # for observe, will get only 3 filed names, data related fields are missing


# (2). if load_external_data=True, after optimize, will lost 'data_location' field
print('for load_external_data=True:')
model_cf = onnx.load(model_path, load_external_data=True)

for tensor in model_cf.graph.initializer:
    tensor_field_names = [field[0].full_name for field in tensor.ListFields()]
    print('fields before opt:', tensor_field_names) # for observe, will get 5 filed names

model = onnxoptimizer.optimize(model_cf, passes)  # run passes

for tensor in model.graph.initializer:
    tensor_field_names = [field[0].full_name for field in tensor.ListFields()]
    print('fields after opt:', tensor_field_names) # for observe, will get 4 filed names
  • output: can observe that when load_external_data=False, tensors in the model could not link to its data since the field 'external_data' is missing.
for load_external_data=False:
fields before opt: ['onnx.TensorProto.dims', 'onnx.TensorProto.data_type', 'onnx.TensorProto.name', 'onnx.TensorProto.external_data', 'onnx.TensorProto.data_location']
fields after opt: ['onnx.TensorProto.dims', 'onnx.TensorProto.data_type', 'onnx.TensorProto.name']
for load_external_data=True:
fields before opt: ['onnx.TensorProto.dims', 'onnx.TensorProto.data_type', 'onnx.TensorProto.name', 'onnx.TensorProto.raw_data', 'onnx.TensorProto.data_location']
fields after opt: ['onnx.TensorProto.dims', 'onnx.TensorProto.data_type', 'onnx.TensorProto.name', 'onnx.TensorProto.raw_data']

System information

  • OS Platform and Distribution (e.g. Linux Ubuntu 16.04): win10
  • onnxoptimizer version (e.g. 1.7): 1.9.0
  • Python version: 3.8.0 and 3.6

[Pass request] Fuse Pad - Pooling (Avg/MaxPool)

This is not an issue but a feature request for adding a fuse_pad_into_pool optimizer pass.

Currently, 'nop' pad layer are removed by eliminate_nop_pad.
Also, pad layers before conv layers are fused by fuse_pad_into_conv.

It would be desirable to have the same behavior also for pooling layers (AveragePool, MaxPool):
i.e.: fusing pad layers before pool layers into the pool layer.

Aka adding a fuse_pad_into_pool optimizer pass.

If you prefer us to provide a pull request for this feature, let us know.

optimizer_test.py failure on litprotobuf ERROR

I installed onnx/optimizer from source as mentioned.
Then I run python onnxoptimizer/test/optimizer_test.py, I got the failure below:

[libprotobuf ERROR google/protobuf/descriptor_database.cc:644] File already exists in database: onnx/onnx-ml.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
Aborted (core dumped)

Any suggestions? Thanks!

GraphProto loses its sparse tensor initializer going through the optimizer.

Bug Report

Is the issue related to model conversion?

No.

Describe the bug

Start with an ONNX model with a sparse initializer. Run model optimizer with option "eliminate_unused_initializer". Now the returned model no longer has the original sparse initializer.

System information

  • OS Platform and Distribution (e.g. Linux Ubuntu 16.04): Linux RedHat5
  • ONNX version (e.g. 1.7): 1.6
  • Python version: 3.8.2
  • GCC/Compiler version (if compiling from source): gcc7
  • CMake version:
  • Protobuf version: 3
  • Visual Studio version (if applicable):

Reproduction instructions

  • Describe the code to reproduce the behavior.
import onnx
from onnx import optimizer
model = onnx.load('model.onnx')
model.graph.sparse_initializer
optimized_model = optimizer.optimize(model)
optimized_model.graph.sparse_initializer
  • Attach the ONNX model to the issue (where applicable)
    A text version of the ONNX model is attached.

Expected behavior

The sparse initializer should persist in the optimized model.

Notes

When the optimizer converts GraphProto to onnx::Graph in the following stack, the sparse initializer is not carried over:

#0 onnx::graphProtoToGraph (gp=..., nested=nested@entry=false)
at .../onnx/common/ir_pb_converter.cc:188
onnx/onnx#1 0x0000000000568d3f in onnx::ImportModelProto (mp=...)
at .../onnx/common/ir_pb_converter.cc:338
onnx/onnx#2 0x000000000054e39a in onnx::optimization::Optimizer::optimize (
this=this@entry=0x7fffffffcf00, mp_in=...)
at .../onnx/optimizer/optimize.h:26
onnx/onnx#3 0x0000000000534d3c in onnx::optimization::Optimize (mp_in=..., names=...)
at .../onnx/optimizer/optimize.cc:32
rank2-identity.txt

onnx optimizer 0.2.4 crashes

Bug Report

Is the issue related to model conversion?

No.

Describe the bug

ONNX optimizer crashes

System information

  • OS Platform and Distribution (Microsoft Windows 10 Enterprise):
  • ONNX version : 1.8.1
  • Python version: 3.8.8
  • Protobuf version: 3.14.0
    image

Reproduction instructions

  • Describe the code to reproduce the behavior.
import numpy as np
import onnx
import onnxoptimizer
import onnxruntime

original_model = onnx.load("dump.onnx")
onnx.checker.check_model(original_model)

batch_sz = 1
max_sent_len = 50
keep_going = np.array([True], dtype=np.bool)
max_trip_count = np.array([2], dtype=np.int64)
query_data = np.zeros((batch_sz, max_sent_len), dtype=np.int64)
sequence_lens = np.ones([1], dtype=np.int32)

sess = onnxruntime.InferenceSession("dump.onnx")
result = sess.run(None, {'max_trip_count': max_trip_count, 'keep_going_in': keep_going, 'query': query_data, 'sequence_lens':  sequence_lens})

# Crashed here!!!
onnxoptimizer.optimize(original_model)
  • Attach the ONNX model to the issue (where applicable)

https://github.com/tzhang-666/issues/blob/master/dump.zip

Expected behavior

Optimizer should not crash and give clues if the input is illegal.

Notes

N/A.

Can't install optimizer and simplifier on jetson agx xavier

The error had occured when i install the onnx-simplifier on my jetson agx xavier using the command pip install onnx-simplifier:

`(mmcv) aurora@aurora-desktop:~$ pip install onnx-simplifier

Collecting onnx-simplifier

Using cached onnx-simplifier-0.3.5.tar.gz (13 kB)

Collecting onnx

Using cached onnx-1.9.0.tar.gz (9.8 MB)

Installing build dependencies ... done

Getting requirements to build wheel ... done

Installing backend dependencies ... done

Preparing wheel metadata ... done

ERROR: Could not find a version that satisfies the requirement onnxoptimizer>=0.2.5 (from onnx-simplifier) (from versions: none)

ERROR: No matching distribution found for onnxoptimizer>=0.2.5 (from onnx-simplifier)`

I guess maybe i should build the onnxoptimizer>=0.2.5 from source code ,but the following error occurred:

`(mmcv) aurora@aurora-desktop:~/onnxoptimizer$ pip install -e .

Obtaining file:///home/aurora/onnxoptimizer

Collecting onnx

Using cached onnx-1.9.0.tar.gz (9.8 MB)

Installing build dependencies ... done

Getting requirements to build wheel ... done

Installing backend dependencies ... done

Preparing wheel metadata ... done

Collecting typing-extensions>=3.6.2.1

Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)

Requirement already satisfied: numpy>=1.16.6 in /home/aurora/.local/lib/python3.6/site-packages (from onnx->onnxoptimizer==0.2.6) (1.19.2)

Requirement already satisfied: protobuf in /home/aurora/.local/lib/python3.6/site-packages (from onnx->onnxoptimizer==0.2.6) (3.14.0)

Requirement already satisfied: six in /home/aurora/.local/lib/python3.6/site-packages (from onnx->onnxoptimizer==0.2.6) (1.15.0)

Building wheels for collected packages: onnx

Building wheel for onnx (PEP 517) ... error

ERROR: Command errored out with exit status 1:

command: /home/aurora/archiconda3/envs/mmcv/bin/python3.6 /home/aurora/archiconda3/envs/mmcv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpi5sfy_km

cwd: /tmp/pip-install-0dnb9thf/onnx

Complete output (83 lines):

10:03:54.151479 git.c:344 trace: built-in: git rev-parse HEAD

fatal: 不是一个 git 仓库(或者任何父目录):.git

running bdist_wheel

running build

running build_py

running create_version

running cmake_build

Using cmake args: ['/usr/bin/cmake', '-DPYTHON_INCLUDE_DIR=/home/aurora/archiconda3/envs/mmcv/include/python3.6m', '-DPYTHON_EXECUTABLE=/home/aurora/archiconda3/envs/mmcv/bin/python3.6', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-36m-aarch64-linux-gnu.so', '-DCMAKE_BUILD_TYPE=Release', '-DONNX_ML=1', '/tmp/pip-install-0dnb9thf/onnx']

-- The C compiler identification is GNU 7.5.0

-- The CXX compiler identification is GNU 7.5.0

-- Check for working C compiler: /usr/bin/cc

-- Check for working C compiler: /usr/bin/cc -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Detecting C compile features

-- Detecting C compile features - done

-- Check for working CXX compiler: /usr/bin/c++

-- Check for working CXX compiler: /usr/bin/c++ -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Detecting CXX compile features

-- Detecting CXX compile features - done

-- Found PythonInterp: /home/aurora/archiconda3/envs/mmcv/bin/python3.6 (found version "3.6.11")

-- Found PythonLibs: /home/aurora/archiconda3/envs/mmcv/lib/libpython3.6m.so (found version "3.6.11")

Generated: /tmp/pip-install-0dnb9thf/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto

CMake Error at CMakeLists.txt:292 (message):

Protobuf compiler not found

Call Stack (most recent call first):

CMakeLists.txt:323 (relative_protobuf_generate_cpp)

-- Configuring incomplete, errors occurred!

See also "/tmp/pip-install-0dnb9thf/onnx/.setuptools-cmake-build/CMakeFiles/CMakeOutput.log".

/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/dist.py:645: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead

% (opt, underscore_opt))

Traceback (most recent call last):

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in

main()

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main

json_out['return_val'] = hook(**hook_input['kwargs'])

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 205, in build_wheel

metadata_directory)

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 222, in build_wheel

wheel_directory, config_settings)

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir

self.run_setup()

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 259, in run_setup

self).run_setup(setup_script=setup_script)

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 150, in run_setup

exec(compile(code, file, 'exec'), locals())

File "setup.py", line 359, in

'backend-test-tools = onnx.backend.test.cmd_tools:main',

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/setuptools/init.py", line 153, in setup

return distutils.core.setup(**attrs)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/core.py", line 148, in setup

dist.run_commands()

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/dist.py", line 955, in run_commands

self.run_command(cmd)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/dist.py", line 974, in run_command

cmd_obj.run()

File "/tmp/pip-build-env-yprluil8/overlay/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 299, in run

self.run_command('build')

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/cmd.py", line 313, in run_command

self.distribution.run_command(command)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/dist.py", line 974, in run_command

cmd_obj.run()

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/command/build.py", line 135, in run

self.run_command(cmd_name)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/cmd.py", line 313, in run_command

self.distribution.run_command(command)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/dist.py", line 974, in run_command

cmd_obj.run()

File "setup.py", line 233, in run

self.run_command('cmake_build')

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/cmd.py", line 313, in run_command

self.distribution.run_command(command)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/distutils/dist.py", line 974, in run_command

cmd_obj.run()

File "setup.py", line 219, in run

subprocess.check_call(cmake_args)

File "/home/aurora/archiconda3/envs/mmcv/lib/python3.6/subprocess.py", line 311, in check_call

raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command '['/usr/bin/cmake', '-DPYTHON_INCLUDE_DIR=/home/aurora/archiconda3/envs/mmcv/include/python3.6m', '-DPYTHON_EXECUTABLE=/home/aurora/archiconda3/envs/mmcv/bin/python3.6', '-DBUILD_ONNX_PYTHON=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-36m-aarch64-linux-gnu.so', '-DCMAKE_BUILD_TYPE=Release', '-DONNX_ML=1', '/tmp/pip-install-0dnb9thf/onnx']' returned non-zero exit status 1.


ERROR: Failed building wheel for onnx

Failed to build onnx

ERROR: Could not build wheels for onnx which use PEP 517 and cannot be installed directly`

Are the optimizer and simplifier supported on the jetson platform?What shold i do,if they are supported.

C++ Install Optimizer and ONNX respectively?

It's a pure installation amateur issue:

When I tried to install optimizer, it automatically install the third-party onnx. How can I install optimizer and onnx respectively as 2 packages? instead of when I install optimizer, it will overwrite onnx by default, namely, 1 single package is installed in the end?

fuse_add_bias_into_conv optimzier needs a unsqueeze op

Related issue: daquexian/onnx-simplifier#75

The "bias" can be a scalar. A unsqueeze op is needed to match the requirement of the tile op.

The following patch fixes this bug. The patch will be submitted to this repo once the moving completes.

diff --git a/onnx/optimizer/passes/fuse_add_bias_into_conv.h b/onnx/optimizer/passes/fuse_add_bias_into_conv.h
index 0af10cd7..067fe5cc 100644
--- a/onnx/optimizer/passes/fuse_add_bias_into_conv.h
+++ b/onnx/optimizer/passes/fuse_add_bias_into_conv.h
@@ -98,6 +98,12 @@ struct FuseAddBiasIntoConv final : public PredicateBasedPass {
         squeeze->addInput(conv_3rd_input);
         conv_3rd_input = squeeze->output();
         squeeze->insertBefore(orig_conv->node());
+      } else if (bias_shape.size() == 0) {
+        Node* unsqueeze = graph.create(kUnsqueeze, 1);
+        unsqueeze->is_(kaxes, {0});
+        unsqueeze->addInput(conv_3rd_input);
+        conv_3rd_input = unsqueeze->output();
+        unsqueeze->insertBefore(orig_conv->node());
       }
       if (M > 1) {
         Node* constant = graph.create(kConstant, 1);

why default pass only have 'eliminate_nop_transpose', 'eliminate_nop_pad','fuse_consecutive_transposes', 'fuse_transpose_into_gemm'?

In optimze func:

def optimize(model, passes=None, fixed_point=False):  # type: (ModelProto, Optional[Sequence[Text]], bool) -> ModelProto
    """Apply the optimization on the serialized ModelProto.
    Arguments:
        input (ModelProto): model
        names (list of string): list of optimization names
    Return:
        return (ModelProto) optimized model
    """
    if passes is None:
        print('WARNING: defualt optimization passes will be enlarged to all fuse and elimination passes in the next version')
        passes = ['eliminate_nop_transpose',
                  'eliminate_nop_pad',
                  'fuse_consecutive_transposes',
                  'fuse_transpose_into_gemm']
    if not isinstance(model, ModelProto):
        raise ValueError(
            'Optimizer only accepts ModelProto, incorrect type: {}'.format(type(model)))

    model_str = model.SerializeToString()
    if fixed_point:
        optimized_model_str = C.optimize_fixedpoint(model_str, passes)
    else:
        optimized_model_str = C.optimize(model_str, passes)

    return onnx.load_from_string(optimized_model_str)

default pass:
passes = ['eliminate_nop_transpose',
'eliminate_nop_pad',
'fuse_consecutive_transposes',
'fuse_transpose_into_gemm']

Why are there only these types of default passes? Can all passes be used together? Are there any restrictions between various passes?

fuse_bn_into_conv optimization fails on big-endian architecture

Bug Report

This test was run on s390x. s390x is big-endian architecture.

Failure log from pytest:

_______________________________________________ TestOptimizer.test_fuse_bn_into_conv_simple ________________________________________________

self = <optimizer_test.TestOptimizer testMethod=test_fuse_bn_into_conv_simple>

    def test_fuse_bn_into_conv_simple(self):  # type: () -> None
        for (tensor_type, np_type) in [(TensorProto.FLOAT, np.float32), (TensorProto.DOUBLE, np.float64)]:
            conv = helper.make_node("Conv", ["X", "W", "B"], ["Y"])
            bn = helper.make_node("BatchNormalization", [
                                  "Y", "scale", "b", "mean", "var"], ["Z"])
    
            W = np.random.randn(3, 2, 5, 5).astype(np_type) + 2
            B = np.random.randn(3,).astype(np_type) + 2
            scale = np.random.randn(3,).astype(np_type) + 2
            b = np.random.randn(3,).astype(np_type) + 2
            mean = np.random.randn(3,).astype(np_type) + 2
            var = np.abs(np.random.randn(3,).astype(np_type)) + 2
    
            initializers = [
                helper.make_tensor(name, tensor_type,
                                   npa.shape, npa.tobytes(), raw=True)
                for name, npa in [('W', W), ('B', B), ('scale', scale), ('b', b), ('mean', mean), ('var', var)]
            ]
            graph = helper.make_graph(
                [conv, bn],
                "test",
                [helper.make_tensor_value_info("X", tensor_type, (5, 2, 28, 28)),
                 helper.make_tensor_value_info("W", tensor_type, (3, 2, 5, 5)),
                 helper.make_tensor_value_info("B", tensor_type, (3,)),
                 helper.make_tensor_value_info("scale", tensor_type, (3,)),
                 helper.make_tensor_value_info("b", tensor_type, (3,)),
                 helper.make_tensor_value_info("mean", tensor_type, (3,)),
                 helper.make_tensor_value_info("var", tensor_type, (3,))],
                [helper.make_tensor_value_info(
                    "Z", tensor_type, (5, 3, 24, 24))],
                initializer=initializers,
                value_info=[
                    helper.make_tensor_value_info(
                        "Y", tensor_type, (5, 3, 24, 24))
                ]
            )
            optimized_model = self._optimized(graph, ["fuse_bn_into_conv"])
    
            self.assertEqual(len(optimized_model.graph.node), 1)
            self.assertEqual(optimized_model.graph.node[0].op_type, 'Conv')
            self.assertEqual(len(optimized_model.graph.initializer), 2)
            new_W = numpy_helper.to_array(optimized_model.graph.initializer[0])
            new_b = numpy_helper.to_array(optimized_model.graph.initializer[1])
    
            f = scale / np.sqrt(var + 1e-5)
>           np.testing.assert_almost_equal((B - mean) * f + b, new_b)
E           AssertionError: 
E           Arrays are not almost equal to 7 decimals
E           
E           Mismatched elements: 3 / 3 (100%)
E           Max absolute difference: 2.7824624e+14
E           Max relative difference: 1.0692023e+32
E            x: array([-1.3619891,  2.4206262,  2.2576501], dtype=float32)
E            y: array([-4.7447069e-14, -2.2639551e-32,  2.7824624e+14], dtype=float32)

optimizer_test.py:1509: AssertionError

The fuse_bn_into_conv.h code does not account for potentially running on a big-endian architecture machine when running the optimization routines.

Attached is a patch file with updated source files with a proposed fix for this optimization issue.

onnx-opt.patch.zip

A pull request can be submitted if needed.

Optimizer reports "Unresolved value references" since v0.3.0

Via onnxcli zhenhuaw-me/onnxcli#28

The model: https://github.com/zhenhuaw-me/onnxcli/blob/v0.2.0/assets/tests/conv.float32.onnx

Last pass version: v0.2.7

Log

2022-11-13 12:15:29,988 D [testing][test_dispatcher.py:34] Running optimize ./assets/tests/conv.float32.onnx optimized.onnx
2022-11-13 12:15:29,988 D [onnxcli][dispatcher.py:25] Running ['optimize', './assets/tests/conv.float32.onnx', 'optimized.onnx']
2022-11-13 12:15:29,994 I [onnxcli][optimize.py:27] Running <Optimization> on model ./assets/tests/conv.float32.onnx
2022-11-13 12:15:30,483 W [onnxcli][optimize.py:38] No optimization passes specified, running all available passes: ['rename_input_output', 'set_unique_name_for_nodes', 'nop', 'eliminate_nop_cast', 'eliminate_nop_dropout', 'eliminate_nop_flatten', 'extract_constant_to_initializer', 'eliminate_if_with_const_cond', 'eliminate_nop_monotone_argmax', 'eliminate_nop_pad', 'eliminate_nop_expand', 'eliminate_shape_gather', 'eliminate_slice_after_shape', 'eliminate_nop_transpose', 'fuse_add_bias_into_conv', 'fuse_bn_into_conv', 'fuse_consecutive_concats', 'fuse_consecutive_log_softmax', 'fuse_consecutive_reduce_unsqueeze', 'fuse_consecutive_squeezes', 'fuse_consecutive_transposes', 'fuse_matmul_add_bias_into_gemm', 'fuse_pad_into_conv', 'fuse_pad_into_pool', 'fuse_transpose_into_gemm', 'replace_einsum_with_matmul', 'lift_lexical_references', 'split_init', 'split_predict', 'fuse_concat_into_reshape', 'eliminate_nop_reshape', 'eliminate_deadend', 'eliminate_identity', 'eliminate_shape_op', 'eliminate_unused_initializer', 'eliminate_duplicate_initializer']
Traceback (most recent call last):
  File "/Users/zhenhuaw/workspace/ONNX/onnxcli/tests/test_dispatcher.py", line 55, in <module>
    test_dispatch_core()
  File "/Users/zhenhuaw/workspace/ONNX/onnxcli/tests/test_dispatcher.py", line 36, in test_dispatch_core
    dispatch_core(shlex.split(cmd))
  File "/Users/zhenhuaw/workspace/ONNX/onnxcli/onnxcli/dispatcher.py", line 40, in dispatch_core
    args.func(args)
  File "/Users/zhenhuaw/workspace/ONNX/onnxcli/onnxcli/optimize.py", line 44, in run
    optimized = onnxoptimizer.optimize(model, passes)
  File "/Users/zhenhuaw/workspace/toolchain/runtime.git/python/venv/onnx/lib/python3.10/site-packages/onnxoptimizer/__init__.py", line 50, in optimize
    optimized_model_str = C.optimize(model_str, passes)
RuntimeError: Unresolved value references: Conv2D_bias,Variable/read,

`onnxoptimizer` is missing support for python 3.10

Summary

It looks onnxoptimizer is missing support for python 3.10. Notably, I don't see 3.10 files in https://pypi.org/project/onnxoptimizer/#files

Details

On my Mac:

% python3.10 -m venv .venv
% source .venv/bin/activate
% python3.10 --version
Python 3.10.6

% python3.10 -m pip install onnxoptimizer

Collecting onnxoptimizer
  Using cached onnxoptimizer-0.3.1.tar.gz (17.4 MB)
  Preparing metadata (setup.py) ... done
Collecting onnx
  Downloading onnx-1.12.0-cp310-cp310-macosx_10_12_x86_64.whl (12.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.4/12.4 MB 32.2 MB/s eta 0:00:00
Collecting typing-extensions>=3.6.2.1
  Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Collecting protobuf<=3.20.1,>=3.12.2
  Downloading protobuf-3.20.1-cp310-cp310-macosx_10_9_universal2.whl (962 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 962.3/962.3 kB 20.2 MB/s eta 0:00:00
Collecting numpy>=1.16.6
  Using cached numpy-1.23.2-cp310-cp310-macosx_10_9_x86_64.whl (18.1 MB)
Using legacy 'setup.py install' for onnxoptimizer, since package 'wheel' is not installed.
Installing collected packages: typing-extensions, protobuf, numpy, onnx, onnxoptimizer
  Running setup.py install for onnxoptimizer ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for onnxoptimizer did not run successfully.
  │ exit code: 1
  ╰─> [70 lines of output]
      fatal: not a git repository (or any of the parent directories): .git
     [REDACTED]/.venv/lib/python3.10/site-packages/setuptools/dist.py:771: UserWarning: Usage of dash-separated 'license-file' will not be supported in future versions. Please use the underscore name 'license_file' instead
        warnings.warn(
      [REDACTED]/.venv/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      [REDACTED]/.venv/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      WARNING: The wheel package is not available.
      running install
      [REDACTED].venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      running create_version
      running cmake_build
      CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
        CMake 3.22 or higher is required.  You are running version 3.18.2
      
      
      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/zf/w2q9lrs922d1d515bg7hkqmw0000gn/T/pip-install-g2drcre2/onnxoptimizer_db0e761224f949edbbec832c6f400ae5/setup.py", line 320, in <module>
          setuptools.setup(
        File "[REDACTED].venv/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
          self.run_command(cmd)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/command/install.py", line 698, in run
          self.run_command('build')
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/command/build.py", line 24, in run
          super().run()
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/private/var/folders/zf/w2q9lrs922d1d515bg7hkqmw0000gn/T/pip-install-g2drcre2/onnxoptimizer_db0e761224f949edbbec832c6f400ae5/setup.py", line 216, in run
          self.run_command('cmake_build')
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "[REDACTED]/.venv/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "/private/var/folders/zf/w2q9lrs922d1d515bg7hkqmw0000gn/T/pip-install-g2drcre2/onnxoptimizer_db0e761224f949edbbec832c6f400ae5/setup.py", line 202, in run
          subprocess.check_call(cmake_args)
        File "/usr/local/Cellar/[email protected]/3.10.6_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '-DPython_INCLUDE_DIR=/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/include/python3.10', '-DPython_EXECUTABLE=[REDACTED]/.venv/bin/python3.10', '-DBUILD_ONNX_PYTHON=ON', '-DONNX_USE_LITE_PROTO=ON', '-DCMAKE_EXPORT_COMPILE_COMMANDS=ON', '-DONNX_NAMESPACE=onnx', '-DPY_EXT_SUFFIX=.cpython-310-darwin.so', '-DONNX_OPT_USE_SYSTEM_PROTOBUF=OFF', '-DCMAKE_BUILD_TYPE=Release', '-DONNX_ML=1', '/private/var/folders/zf/w2q9lrs922d1d515bg7hkqmw0000gn/T/pip-install-g2drcre2/onnxoptimizer_db0e761224f949edbbec832c6f400ae5']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> onnxoptimizer

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

-1 mean dynamic shape or Static shape?

-1 mean what? dynamic shape or Static shape? Using the latest version(v0.3.1), -1 dim will be constant fold. In old version, It doesn't fold. So that cause some conflict.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.