Comments (9)
jxt1234
commented on June 12, 2024
Use testMNNFromOnnx.py to test firstly.
from mnn.
kushalpatil07
commented on June 12, 2024
I have ran testMNNFromOnnx.py. The test passes when given random input np.random.uniform(0, 12, shapes). When I edited the script to take np.zeros(shapes) as input, the test fails.
On a side note, this mosaic model is a standard model taken from https://github.com/onnx/models/tree/main/validated/vision/style_transfer/fast_neural_style
from mnn.
kushalpatil07
commented on June 12, 2024
hey @jxt1234 is someone looking into this? I would really appreciate the help :)
from mnn.
jxt1234
commented on June 12, 2024
testMNNFromOnnx.py TEST_SUCCESS means MNN is right.
You can convert MNN with --keepInputFormat
and don't call MNN.expr.convert(input,MNN.expr.NC4HW4).
from mnn.
kushalpatil07
commented on June 12, 2024
testMNNFromOnnx.py TEST_SUCCESS means MNN is right.
I feel we are having a misunderstanding, What do you mean by this exactly?
I am giving you an example mosaic.onnx and mosaic.mnn with a particular input not matching.
This does not work even after incorporating your suggestions You can convert MNN with --keepInputFormat and don't call MNN.expr.convert(input,MNN.expr.NC4HW4).
Steps to reproduce the issue:
- This script testMNNFromOnnx.py takes random input into the model, and that passes the test and gives TEST_SUCCESS.
- When we edit the script to take
np.zeros(shape)as input the test itself fails.
@jxt1234 could you try and reproduce this issue using the two steps mentioned above. Attaching the edited testMNNFromOnnx.py for easy reproducibility. Added a comment on line 153 # EDITED HERE so that the model takes zeros as input where I have edited the script to take zeros as input.
from mnn.
jxt1234
commented on June 12, 2024
For your case np.zeros / np.ones will cause input data is all same. So var is very small. It will cause sqrt function Disparity.
It's not common case. Please use real input data to test.
from mnn.
kushalpatil07
commented on June 12, 2024
There is also another disparity we are observing. @jxt1234
The testScript shared is giving very different results on Mac and Linux machines on np.ones and np.zeros.
Test Setup of Linux Machine
MNN version==2.8.1
onnxruntime version==1.17.0
x86_64 architecture Linux machine python3.10.12
Test Setup of Mac Machine
MNN version==2.8.1
onnxruntime version==1.17.0
Mac M1 chip (2020) python3.7.16
Output on Linux machine
######################## MNN Output##############################
array([[[[ 1.7789851e+02, 1.7789816e+02, 1.7789783e+02, ...,
1.7790385e+02, 1.7790039e+02, 1.7790286e+02],
[ 1.7789778e+02, 1.7789758e+02, 1.7789714e+02, ...,
1.7790488e+02, 1.7790128e+02, 1.7790410e+02],
[ 1.7789806e+02, 1.7789786e+02, 1.7789740e+02, ...,
1.7790530e+02, 1.7790167e+02, 1.7790437e+02],
...,
[ 1.7787030e+02, 1.7786987e+02, 1.7786964e+02, ...,
-1.8242943e+01, -3.4454234e+00, 9.5861053e+00],
[ 1.7786783e+02, 1.7786771e+02, 1.7786725e+02, ...,
-3.3775089e+01, -2.9153913e+01, -3.9820961e+01],
[ 1.7787071e+02, 1.7787071e+02, 1.7787039e+02, ...,
-7.8822029e+01, -7.9405579e+01, -9.0173950e+01]],
[[ 1.6287909e+02, 1.6287906e+02, 1.6287897e+02, ...,
1.6287405e+02, 1.6287401e+02, 1.6287613e+02],
[ 1.6287944e+02, 1.6287935e+02, 1.6287930e+02, ...,
1.6287408e+02, 1.6287430e+02, 1.6287608e+02],
[ 1.6287997e+02, 1.6288005e+02, 1.6287979e+02, ...,
1.6287399e+02, 1.6287418e+02, 1.6287605e+02],
...,
[ 1.6285625e+02, 1.6285625e+02, 1.6285588e+02, ...,
-4.3152927e+01, -2.9389851e+01, -3.3526914e+00],
[ 1.6285176e+02, 1.6285170e+02, 1.6285165e+02, ...,
2.2175953e+01, 2.8807076e+01, 3.5963924e+01],
[ 1.6285019e+02, 1.6284982e+02, 1.6284982e+02, ...,
2.5776017e+00, -6.1696470e-03, 5.3866558e+00]],
[[ 1.5353537e+02, 1.5353502e+02, 1.5353499e+02, ...,
1.5352097e+02, 1.5351585e+02, 1.5351753e+02],
[ 1.5353632e+02, 1.5353571e+02, 1.5353595e+02, ...,
1.5352025e+02, 1.5351535e+02, 1.5351712e+02],
[ 1.5353517e+02, 1.5353476e+02, 1.5353500e+02, ...,
1.5352109e+02, 1.5351575e+02, 1.5351735e+02],
...,
[ 1.5354117e+02, 1.5354056e+02, 1.5354077e+02, ...,
2.3128716e+01, 4.9280975e+01, 6.8965607e+01],
[ 1.5353847e+02, 1.5353783e+02, 1.5353815e+02, ...,
7.4308746e+01, 8.9900993e+01, 8.4358002e+01],
[ 1.5353415e+02, 1.5353368e+02, 1.5353375e+02, ...,
2.3612188e+01, 2.1621935e+01, 1.5537018e+01]]]],
dtype=float32)
#################################################################
######################## ONNX Output##############################
[array([[[[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421],
[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421],
[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421],
...,
[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421],
[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421],
[171.34421, 171.34421, 171.34421, ..., 171.34421, 171.34421,
171.34421]],
[[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192],
[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192],
[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192],
...,
[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192],
[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192],
[157.65192, 157.65192, 157.65192, ..., 157.65192, 157.65192,
157.65192]],
[[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988],
[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988],
[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988],
...,
[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988],
[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988],
[147.90988, 147.90988, 147.90988, ..., 147.90988, 147.90988,
147.90988]]]], dtype=float32)]
#################################################################
Output on Mac Machine
######################## MNN Output##############################
array([[[[250.21507 , 250.16147 , 249.76387 , ..., 58.233986,
59.66561 , 51.944107],
[249.54218 , 249.3817 , 248.97392 , ..., 58.83722 ,
64.62019 , 58.525745],
[249.11377 , 249.11868 , 248.49596 , ..., 53.54505 ,
62.351784, 56.883274],
...,
[ 90.75204 , 85.10422 , 77.92067 , ..., 58.6197 ,
88.46058 , 74.92694 ],
[139.21458 , 137.08766 , 132.7679 , ..., 123.57399 ,
149.02847 , 110.15517 ],
[135.13843 , 134.24225 , 134.49554 , ..., 114.151 ,
144.84058 , 116.4156 ]],
[[250.7712 , 251.45166 , 251.51967 , ..., 116.97559 ,
107.31926 , 111.61564 ],
[249.24142 , 249.67822 , 249.77678 , ..., 116.61871 ,
110.15776 , 116.610756],
[247.57831 , 248.08623 , 247.88712 , ..., 116.43434 ,
113.84082 , 120.039795],
...,
[129.85591 , 127.12337 , 119.98896 , ..., 83.45971 ,
100.48431 , 93.10752 ],
[130.522 , 124.07818 , 122.81633 , ..., 79.169044,
96.53803 , 79.26343 ],
[124.85134 , 121.22441 , 123.81856 , ..., 93.23835 ,
112.23832 , 103.228325]],
[[303.78763 , 303.88348 , 303.10953 , ..., 171.72421 ,
168.28552 , 164.99461 ],
[301.4203 , 301.25974 , 300.55664 , ..., 172.78957 ,
172.58192 , 170.90668 ],
[298.97623 , 298.9938 , 298.28656 , ..., 169.06041 ,
172.94447 , 174.88399 ],
...,
[120.82464 , 112.478584, 100.644356, ..., 90.14767 ,
119.52492 , 128.43115 ],
[144.41287 , 137.94812 , 133.33185 , ..., 128.82146 ,
163.61607 , 151.649 ],
[153.434 , 149.53357 , 152.06706 , ..., 156.95384 ,
189.02585 , 174.62895 ]]]], dtype=float32)
#################################################################
######################## ONNX Output##############################
[array([[[[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201],
[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201],
[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201],
...,
[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201],
[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201],
[170.95201, 170.95201, 170.95201, ..., 170.95201, 170.95201,
170.95201]],
[[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027],
[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027],
[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027],
...,
[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027],
[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027],
[157.96027, 157.96027, 157.96027, ..., 157.96027, 157.96027,
157.96027]],
[[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ],
[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ],
[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ],
...,
[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ],
[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ],
[151.1252 , 151.1252 , 151.1252 , ..., 151.1252 , 151.1252 ,
151.1252 ]]]], dtype=float32)]
#################################################################
The difference in outputs of the onnx models between two devices could be attributed to the sqrt function disparity. But the difference in the result of MNN models is huge, do we attribute this large change to the sqrt function?
Is the implementation of MNN runtime so different for Arm and x86 that it is expected to see such behaviour? Because the numbers being so different feels like that I am not using it correctly, or there is a bug?
Attaching the testScript again for reference
import MNN
mnn_model = 'mosaic.mnn'
net = MNN.nn.load_module_from_file(mnn_model,[],[]);
input=MNN.numpy.zeros([1,3,224,224],dtype=MNN.numpy.float32)
#input=MNN.expr.convert(input,MNN.expr.NC4HW4)
output=net.forward(input)
output = MNN.expr.convert(output,MNN.expr.NCHW)
print("######################## MNN Output##############################")
print(output)
print('#################################################################')
print("")
import onnxruntime as ort
import numpy as np
onnx_model='mosaic.onnx'
model= ort.InferenceSession('mosaic.onnx')
output=model.run(None,{'input1':np.zeros([1,3,224,224],dtype=np.float32)})
print("######################## ONNX Output##############################")
print(output)
print('#################################################################')
from mnn.
jxt1234
commented on June 12, 2024
np.zeros() and np.ones() isn't valid input for this model because of instancenorm. It will result to compute sqrt(0.0 + 0.000000000001) , thus cause [Disparity]
from mnn.
github-actions
commented on June 12, 2024
Marking as stale. No activity in 60 days.
from mnn.
Related Issues (20)
- 编译示例工程代码报错
- 编译测试工具报错阻塞 HOT 3
- 问一下, 在armv8.2 cpu 手机上, 是否已支持fp16 进行卷积的推理阿? HOT 1
- c++ 链接MNN后，编译报错，无法解析的外部符号 _std_init_once_link_alternate_names_and_abort和_std_min_element_4 HOT 6
- mnncompress量化模型以后大小没有变化 HOT 3
- MNN.cv has the rotation issue in loading some photos from the phone HOT 1
- 求教 expr 中_Fill 的正确用法 HOT 3
- 鸿蒙系统编译失败 HOT 1
- Facing issue with version incapitability between protobuffer library and MNN library while performing mnnquant quantization process using python API of mnn HOT 4
- 多输入模型转换时添加--optimizePrefer 2选项 并基于此使用MNNPythonOfflineQuant离线量化，在arm cpu后端推理输出异常，与x86 cpu推理不对齐 HOT 1
- 请问MNN如何实现使用同一模型做并行推理？ HOT 5
- Invalidate buffer to create MNN Module HOT 5
- MNNConvert onnx转mnn 错误，nn.ConvTranspose1d forward 中 pad 会出问题 HOT 7
- How to get SD mnn model used in transformers/diffussion/diffussion_demo
- Android 编译DEMO 出现 Target "MNN_CL_WRAP" of type OBJECT_LIBRARY may not be linked into another target
- 问问MNN的兼容方式 HOT 1
- ios平台 没有getTensor方法 HOT 2
- linux x86_64 平台， 多线程情况下， mnn 和推理速度不如 onnx，有可能是什么原因？ HOT 4
- 手机端vulkan特性 image_write_without_format这个特性影响大吗?
- mnn 模型，Android 平台 benchmark 测试，推理使用 openCL, vulkan 都比 CPU 慢，可能是什么原因？
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mnn.