nervanasystems / ngraph-onnx Goto Github PK

nGraph™ Backend for ONNX

License: Apache License 2.0

Python 85.97% Shell 7.10% Dockerfile 1.51% Groovy 5.41%

ngraph-onnx's Introduction

DISCONTINUATION OF PROJECT

This project will no longer be maintained by Intel. This project has been identified as having known security escapes. Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project. Intel no longer accepts patches to this project.

ngraph-onnx

nGraph Backend for ONNX.

This repository contains tools to run ONNX models using the Intel nGraph library as a backend.

Installation

Follow our build instructions to install nGraph-ONNX from sources.

Usage example

Importing an ONNX model

You can download models from the ONNX model zoo. For example ResNet-50:

$ wget https://s3.amazonaws.com/download.onnx/models/opset_8/resnet50.tar.gz
$ tar -xzvf resnet50.tar.gz

Use the following Python commands to convert the downloaded model to an nGraph model:

# Import ONNX and load an ONNX file from disk
>>> import onnx
>>> onnx_protobuf = onnx.load('resnet50/model.onnx')

# Convert ONNX model to an ngraph model
>>> from ngraph_onnx.onnx_importer.importer import import_onnx_model
>>> ng_function = import_onnx_model(onnx_protobuf)

# The importer returns a list of ngraph models for every ONNX graph output:
>>> print(ng_function)
<Function: 'resnet50' ([1, 1000])>

This creates an nGraph Function object, which can be used to execute a computation on a chosen backend.

Running a computation

After importing an ONNX model, you will have an nGraph Function object. Now you can create an nGraph Runtime backend and use it to compile your Function to a backend-specific Computation object. Finally, you can execute your model by calling the created Computation object with input data.

# Using an ngraph runtime (CPU backend) create a callable computation object
>>> import ngraph as ng
>>> runtime = ng.runtime(backend_name='CPU')
>>> resnet_on_cpu = runtime.computation(ng_function)

# Load an image (or create a mock as in this example)
>>> import numpy as np
>>> picture = np.ones([1, 3, 224, 224], dtype=np.float32)

# Run computation on the picture:
>>> resnet_on_cpu(picture)
[array([[2.16105007e-04, 5.58412226e-04, 9.70510227e-05, 5.76671446e-05,
         7.45318757e-05, 4.80892748e-04, 5.67404088e-04, 9.48728994e-05,
         ...

ngraph-onnx's People

Contributors

Stargazers

Watchers

Forkers

ddokupil bergtholdt tuboss opencici2006 calvinlcchen arnaudwald stjordanis c00lrain postrational evefg4 rsketine evelynfay psfoley lnguyen-nvn jiapei100 silee2 vishalbelsare diyessi

ngraph-onnx's Issues

extend the example: validate the model

Could you extend your example that shows how users can validate the model like the following shown at onnx model github page ?
It is not clear to beginners how we can do that.

Thanks

import numpy as np
import onnx
import os
import glob
import onnx_backend as backend

from onnx import numpy_helper

model = onnx.load('model.onnx')
test_data_dir = 'test_data_set_0'

Load inputs

inputs = []
inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
for i in range(inputs_num):
input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
tensor = onnx.TensorProto()
with open(input_file, 'rb') as f:
tensor.ParseFromString(f.read())
inputs.append(numpy_helper.to_array(tensor))

Load reference outputs

ref_outputs = []
ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
for i in range(ref_outputs_num):
output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
tensor = onnx.TensorProto()
with open(output_file, 'rb') as f:
tensor.ParseFromString(f.read())
ref_outputs.append(numpy_helper.to_array(tensor))

Run the model on the backend

outputs = list(backend.run_model(model, inputs))

Compare the results with reference outputs.

for ref_o, o in zip(ref_outputs, outputs):
np.testing.assert_almost_equal(ref_o, o)

Serialized Numpy archives, stored in files with the naming convention test_data_*.npz. Each file contains one set of test inputs and outputs.
import numpy as np
import onnx
import onnx_backend as backend

Load the model and sample inputs and outputs

model = onnx.load(model_pb_path)
sample = np.load(npz_path, encoding='bytes')
inputs = list(sample['inputs'])
outputs = list(sample['outputs'])

Run the model with an onnx backend and verify the results

np.testing.assert_almost_equal(outputs, backend.run_model(model, inputs))

RecursionError: maximum recursion depth exceeded while calling a Python object

Seeing errors when running ie-transformer using ONNX front end.

https://aipg-rancher.intel.com/jenkins/algo/job/ie-tranformer-unittests/285/consoleText

  File "resnet50.py", line 90, in <module>
    net = compile_onnx('model.onnx', target)
  File "resnet50.py", line 33, in compile_onnx
    ng_models = import_onnx_model(onnx_protobuf)
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/importer.py", line 53, in import_onnx_model
    model = ModelWrapper(onnx_protobuf)
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/model_wrappers.py", line 62, in __init__
    self.graph = GraphWrapper(model_proto.graph, self)
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/model_wrappers.py", line 84, in __init__
    self._initialize_ng_nodes()
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/model_wrappers.py", line 133, in _initialize_ng_nodes
    node.get_ng_nodes_dict()
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/model_wrappers.py", line 310, in get_ng_nodes_dict
    output_nodes = make_ng_nodes(self._graph.model.node_factory, self)
  File "/home/dockuser/ie-transformer/build/onnx-venv/lib/python3.5/site-packages/ngraph_onnx/onnx_importer/model_wrappers.py", line 41, in __getattr__
    return getattr(self._proto, name)

The getattr call is repeated recursively until failure.

To reproduce follow ie-trasnformer readme to run resnet50.
https://github.com/NervanaSystems/ie-transformer

RuntimeError: Node (): unknown attribute 'starts'

I converted a pytorch model to ONNX using these settings:

input = torch.tensor(torch.ones(1,3,32,32))

torch.onnx.export(model.module,
input,
"test.onnx",
export_params=True,
opset_version=11,
do_constant_folding=True,
input_names=['input'],
output_names=['output'],
verbose=True)

I imported that model to ngraph using these commands:

import onnx
from ngraph_onnx.onnx_importer.importer import import_onnx_model
onnx_protobuf = onnx.load('test.onnx')
ng_function = import_onnx_model(onnx_protobuf)

On import_onnx_model, it throws this error:

File "/home/justin/PycharmProjects/beverage-inference/venv/lib/python3.6/site-packages/ngraph_onnx/onnx_importer/importer.py", line 36, in import_onnx_model
return onnx_import.import_onnx_model(onnx_protobuf.SerializeToString())
RuntimeError: Node (): unknown attribute 'starts'

Is there a way to see which node has the "starts" attribute? I have a feeling it's related to slice, but I can't be sure.

Attached is a dump of the nodes from the onnx protobuf, nodes.txt.

Versions:
pytorch 1.4
onnx 1.6
ngraph-core 0.26
ngraph-onnx 0.24

ONNX importer runtime error: unsupported data type: BOOL

From this code:
onnx_protobuf = onnx.load(model_path)
ng_function = import_onnx_model(onnx_protobuf)

bridge fails like that:
File "./test_onnx_model.py", line 22, in evaluate
ng_function = import_onnx_model(onnx_protobuf)
File "/media/dmitry/BigDisk/rnovikov/ngraph-onnx/venv_onnx_check/lib/python3.6/site-packages/ngraph_onnx/onnx_importer/importer.py", line 36, in import_onnx_model
return onnx_import.import_onnx_model(onnx_protobuf.SerializeToString())
RuntimeError: unsupported data type: BOOL

source onnx:
https://www.dropbox.com/s/84xnff3bec035dz/wavenet_body.zip?dl=0

Could not find a version that satisfies the requirement ngraph-core (from versions: )

How can I solve this issue?

Using Intel Distribution for python:

python --version
Python 3.6.5 :: Intel Corporation

pip install ngraph-core ngraph-onnx
Collecting ngraph-core
Could not find a version that satisfies the requirement ngraph-core (from versions: )
No matching distribution found for ngraph-core

cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

GPU backend question

I have a question about running the example on an Intel integrated GPU. I got the following error when the backend_name is GPU. I am not clear which libraries or tools need to be installed. Must I install ngraph-onnx from the source ? I installed it using pip.

Thanks for your instruction.

File "test_onnx_model.py", line 41, in
runtime = ng.runtime(backend_name='GPU')
File "/home/cc/.local/lib/python3.6/site-packages/ngraph/runtime.py", line 35, in runtime
return Runtime(backend_name)
File "/home/cc/.local/lib/python3.6/site-packages/ngraph/runtime.py", line 43, in init
self.backend = Backend.create(backend_name)
RuntimeError: Unable to find backend 'GPU' as file '/home/cc/.local/lib/python3.6/site-packages/../../libgpu_backend.so'
Open error message '/home/cc/.local/lib/python3.6/site-packages/../../libgpu_backend.so: cannot open shared object file: No such file or directory'

Get segmentation fault after running "import ngraph_onnx.onnx_importer.importer"

I have followed the installation guide and try to use ngraph-onnx to convert the Resnet50 model as below. But, I got a segmentation fault after running import ngraph_onnx.onnx_importer.importer.

>>> import onnx
>>> onnx_protobuf = onnx.load('resnet50/model.onnx')
>>> from ngraph_onnx.onnx_importer.importer import import_onnx_model
Segmentation fault (core dumped)

Got RuntimeError after running import ngraph with resnet example

Got this error after running the import_onnx_model command following the example for resnet50
Running onnx 1.3 and python 3.5.2

ng_models = import_onnx_model(onnx_pb)
ONNX ai.onnx opset version 8 is not supported. Falling back to latest supported version: 7
More than one different shape in input nodes [<Constant: 'Constant_289' ([])>, <BatchNormInference: 'gpu_0/res2_0_branch2c_bn_1' ([1, 256, 56, 56])>].
More than one different data type in input nodes [<Constant: 'Constant_289' ([])>, <BatchNormInference: 'gpu_0/res2_0_branch2c_bn_1' ([1, 256, 56, 56])>].
truncated traceback:
RuntimeError: While validating node 'Add[Add_292](Broadcast_290: int64_t{1,256,56,56}, Broadcast_291: float{1,256,56,56})' of type 'Add':
Assertion 'element::Type::merge(element_type, element_type, get_input_element_type(i))' failed at /root/ng/src/ngraph/node.cpp:451:
Argument element types are inconsistent.

Tried converting the model to opset v7 and still no luck.