Comments (11)
jywu-msft
commented on July 17, 2024
the QNN docs don't mention aarch64 Linux support, but you're right, the QNN SDK does seem to have some aarch64--linux directories under lib
cross compiling on Windows isn't going to work in your case if you are targeting a Linux system.
maybe it's possible to modify the CMakeLists.txt
onnxruntime/cmake/CMakeLists.txt
Line 832 in 7e93cd7
to target one of the aarch64--linux , currently it assumes non x86_64 Linux is aarch64-android but that's not the case for you.
from onnxruntime.
Misaka17032
commented on July 17, 2024
Hi, thank you for the advise.
so I did try to do the compiling on a x86_64 ubuntu machine, and edit the CMakeLists.txt as you suggested:
else()
if (${CMAKE_SYSTEM_NAME} STREQUAL "Android")
set(QNN_ARCH_ABI aarch64-android-clang6.0)
elseif (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
if (${GEN_PLATFORM} STREQUAL "x86_64")
set(QNN_ARCH_ABI aarch64-linux-clang) # edited
else()
set(QNN_ARCH_ABI aarch64-linux-clang) # edited
endif()
endif()
endif()
endif()
and here is the command I use:
bash build.sh --arm64 --use_qnn --qnn_home=/opt/qcom/aistack/qnn/2.16.4.231110 --parallel --build_wheel --skip_submodule_sync --config Release
The building process is all passed until the step Linking CXX shared module onnxruntime_pybind11_state.so:
[100%] Building CXX object CMakeFiles/onnxruntime_test_all.dir/mnt/user/onnxruntime-qnn/onnxruntime/onnxruntime/test/flatbuffers/flatbuffer_utils_test.cc.o
[100%] Building CXX object CMakeFiles/onnxruntime_test_all.dir/mnt/user/onnxruntime-qnn/onnxruntime/onnxruntime/test/unittest_main/test_main.cc.o
[100%] Linking CXX shared module onnxruntime_pybind11_state.so
CMake Error: cmake version 3.29.6
Usage: /mnt/user/anaconda3/envs/onnxqnn/lib/python3.10/site-packages/cmake/data/bin/cmake -E <command> [arguments...]
Available commands:
# just the usage of cmake, omited
make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:407: onnxruntime_pybind11_state.so] Error 1
make[2]: *** Deleting file 'onnxruntime_pybind11_state.so'
make[1]: *** [CMakeFiles/Makefile2:2208: CMakeFiles/onnxruntime_pybind11_state.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Linking CXX executable onnxruntime_test_all
CMake Error: cmake version 3.29.6
Usage: /mnt/user/anaconda3/envs/onnxqnn/lib/python3.10/site-packages/cmake/data/bin/cmake -E <command> [arguments...]
Available commands:
# just the usage of cmake, omited
make[2]: *** [CMakeFiles/onnxruntime_test_all.dir/build.make:5660: onnxruntime_test_all] Error 1
make[2]: *** Deleting file 'onnxruntime_test_all'
make[1]: *** [CMakeFiles/Makefile2:2363: CMakeFiles/onnxruntime_test_all.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
from onnxruntime.
Weisonorz
commented on July 17, 2024
Do you have any update on solving this issue? (compiling onnx_qnn on aarch linux) Our team is having the same problem with setting up the cmake file. By the way, does this onnx_qnn allow dynamic model input?
from onnxruntime.
jywu-msft
commented on July 17, 2024
are you all trying to cross compile or build natively on aarch64 Linux?
remove --arm64 from build option and try to build natively?
from onnxruntime.
Misaka17032
commented on July 17, 2024
For me, I tried to build natively on aarch64 Linux first but then realized that I can't setup the QNN SDK on that environment.
from onnxruntime.
jywu-msft
commented on July 17, 2024
For me, I tried to build natively on aarch64 Linux first but then realized that I can't setup the QNN SDK on that environment.
what setup are you missing. i think you can try copying the qnn sdk installation directory (e.g. /path/to/qnn/2.23.0.240531) from an installation to different platform (e.g. Linux x64) to the aarch64 linux environment , make the cmake modification mentioned above and give it another try.
from onnxruntime.
Misaka17032
commented on July 17, 2024
yes, I did try that before and it seems to failed. Thank you for the advice though, I'll still try to work on that.
from onnxruntime.
jywu-msft
commented on July 17, 2024
yes, I did try that before and it seems to failed. Thank you for the advice though, I'll still try to work on that.
what kind of failure do you get. we can reach out to qualcomm and ask for guidance.
from onnxruntime.
Misaka17032
commented on July 17, 2024
[edited]
So I did some debugging and resolve the previous error, and the failure now is listed below:
/home/user/ort_build/onnxruntime/onnxruntime/core/providers/qnn/builder/qnn_utils.cc: In function ‘std::pair<float, float> onnxruntime::qnn::utils::CheckMinMax(float, float)’:
/home/user/ort_build/onnxruntime/onnxruntime/core/providers/qnn/builder/qnn_utils.cc:466:59: note: parameter passing for argument of type ‘std::pair<float, float>’ when C++17 is enabled changed to match C++14 in GCC 10.1
466 | std::pair<float, float> CheckMinMax(float rmin, float rmax) {
| ^
/home/user/ort_build/onnxruntime/onnxruntime/core/providers/qnn/qnn_execution_provider.cc: In function ‘void onnxruntime::ParseHtpArchitecture(const string&, QnnHtpDevice_Arch_t&)’:
/home/user/ort_build/onnxruntime/onnxruntime/core/providers/qnn/qnn_execution_provider.cc:158:20: error: ‘QNN_HTP_DEVICE_ARCH_V75’ was not declared in this scope; did you mean ‘QNN_HTP_DEVICE_ARCH_V73’?
158 | qnn_htp_arch = QNN_HTP_DEVICE_ARCH_V75;
| ^~~~~~~~~~~~~~~~~~~~~~~
| QNN_HTP_DEVICE_ARCH_V73
gmake[2]: *** [CMakeFiles/onnxruntime_providers_qnn.dir/build.make:552: CMakeFiles/onnxruntime_providers_qnn.dir/home/user/ort_build/onnxruntime/onnxruntime/core/providers/qnn/qnn_execution_provider.cc.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2052: CMakeFiles/onnxruntime_providers_qnn.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
from onnxruntime.
Misaka17032
commented on July 17, 2024
Okay, so I've been trying to figure out the dependencies and configurations, and finally I did build a python wheel with the following command:
bash build.sh --build --arm64 --use_qnn --qnn_home=/path/to/qnn/2.22.6.240515/ --parallel --build_wheel --skip_submodule_sync --config ReleaseHowever, when I tried to follow the instruction and run a basic mobilenet_v3_small, the following error raised:
python I 8330 8330 BufferAllocator.cpp:100] Using DMA-BUF heap named: system
/prj/qct/webtech_hyd18/mlg_user_admin/qaisw_source_repo/qaisw_repo_point_release/snpe_src/avante-tools/prebuilt/dsp/hexagon-sdk-5.4.0/ipc/fastrpc/rpcmem/src/rpcmem_android.c:38:dummy call to rpcmem_init, rpcmem APIs will be used from libxdsprpc
2024-07-05 17:52:03.839963468 [E:onnxruntime:, qnn_execution_provider.cc:597 GetCapability] QNN SetupBackend failed Failed to create device. Error: 14001
The script I run is:
import onnxruntime
import numpy as np
options = onnxruntime.SessionOptions()
# (Optional) Enable configuration that raises an exception if the model can't be
# run entirely on the QNN HTP backend.
# options.add_session_config_entry("session.disable_cpu_ep_fallback", "1")
# Create an ONNX Runtime session.
session = onnxruntime.InferenceSession("mobnet.qdq.onnx",
sess_options=options,
providers=["QNNExecutionProvider"],
provider_options=[{"backend_path": "./lib/libQnnHtp.so"}])
# Run the model with your input.
# TODO: Use numpy to load your actual input from a file or generate random input.
input0 = np.ones((1,3,224,224), dtype=np.float32)
while True:
result = session.run(None, {"input.1": input0})
# Print output.
print(result)from onnxruntime.
Lucashien
commented on July 17, 2024
Did you solve the issue with "QNN SetupBackend failed Failed to create device"?
I'm facing the same problem on Windows AArch64 (Snapdragon 8cx Gen3).
Thanks!
from onnxruntime.
Related Issues (20)
- Model saved with offline basic optimizations will not load - ShapeInferenceError HOT 1
- [Training] [ShapeInferenceError] Dimension could not be inferred: incompatible shapes
- [Build] How can I quantize the llama3 model activation to int4 ?
- [Feature Request] ORT-Profiler: Include timestamps for tensor allocations and deallocations. HOT 2
- header files path not recognized or unable to read header file HOT 1
- [Build] AllocatorTest.CUDAAllocatorFallbackTest failed HOT 1
- [Performance] Get nan value when I block all the node in fp16 conversion HOT 6
- [Bug] The per_tensor quantized weight type of matmul is wrong HOT 1
- ONNX Runtime 1.18.1 CUDA 12.4 cuDNN 9.2 breaks inference with repeated inputs when enable_mem_reuse is enabled
- Latest Release(1.18.1) Java Artifacts Unavailable HOT 1
- [Build] C++ API cannot be reliably linked with an program using CMake
- [BUG] CANN: onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError]
- [Build] Cross compilation of the library for ARMv7 32bit target with gcc 8.3 HOT 2
- CUDA 12 and session.get_providers() not showing CUDAExecutionProvider HOT 7
- [Web] Memory access out of bounds / alignment fault
- An error occurred when I installed onnxruntime-qnn in an Arm environment
- [Performance] Multiple Sessions on Same GPU is very slow
- [Models larger than 2GB :(] Specify mid-graph.output after initializing InferenceSession
- [Error] [ONNXRuntimeError] : 1 : FAIL : CUDA failure 3: initialization error HOT 1
- [Build] long paths in NuGet package breaking build on Windows
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime.