Comments (5)
possibly related to #7573
from onnxruntime.
I think I might be missing onnxruntime-gpu
package
from onnxruntime.
Since I'm coming from the dotnet background and aforementioned package is not yet available for nixos I decided to explore running phi3 in a dotnet project with Microsoft.ML.OnnxRuntimeGenAI.Cuda
, but if throws the same error:
> dotnet run
-------------
Hello, Phi-3!
-------------
Unhandled exception. Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException: Load model from /home/a/projects/phi3/Phi-3-mini-128k-instruct-onnx/cuda/cuda-fp16/phi3-mini-128k-instruct-cuda-fp16.onnx failed:This is an invalid model. In Node, ("/model/layers.0/input_layernorm/LayerNorm", SimplifiedLayerNormalization, "", -1) : ("/model/embed_tokens/Gather/output_0": tensor(float16),"model.layers.0.input_layernorm.weight": tensor(float16),) -> ("/model/layers.0/input_layernorm/output_0": tensor(float16),) , Error No Op registered for SimplifiedLayerNormalization with domain_version of 14
at Microsoft.ML.OnnxRuntimeGenAI.Model..ctor(String modelPath)
at Program.main(String[] argv) in /home/a/projects/phi3/Program.fs:line 20
Now I'm wondering if I opened the issue in the right repo or if should be moved over to onnxruntime-genai.
from onnxruntime.
On nixos onnxruntime
withcuda
is packaged using both onnxruntime_USE_CUDA
and onnxruntime_DISABLE_CONTRIB_OPS
, which could be the reason. Trying to override that.
from onnxruntime.
indeed removing onnxruntime_DISABLE_CONTRIB_OPS
from the nixos derivation resolved the issue
from onnxruntime.
Related Issues (20)
- [Build] Openvino EP Build Error Linux
- [Mobile] Test Application Reveals Multiple Failures in QnnHTPBackendTests Suite on Device HOT 4
- [Performance] OrtValue created form OrtValue.ortvalue_from_numpy() has the same ptr same as the previous one, it cause the value of the previous variable to be modified. HOT 2
- [Build] “onnxruntime_cxx_api.h”: No such file or directory
- Get wrong results occasionally when run yolov9 onnx model using OpenVINOExecutionProvider
- NMS Operator Output Different From Torchvision Implementation
- [Feature Request] May the Calibration Cache in the roadmap?
- [Build] I want to ask if centos 7 does not support 1.18.0 and higher HOT 1
- [Performance] High thread contention in BFCArena HOT 1
- [Feature Request] Add vulkan execution provider
- [Training] linux ondevice training with onnxruntim failed by loading checkpoint: Segmentation fault (core dumped)
- Corrupted value for model outputs that are also model inputs HOT 1
- Llama2 RMS Norm : SimplifiedLayerNormalization
- Llama2 RMS Norm : SimplifiedLayerNormalization
- [Performance]
- CPU LayerNormalization Produces NaN for Inputs with Small Variances
- Why C++ cannot modify the enable_mem_reuse option in Ort::SessionOptions... HOT 2
- [Mobile] Number of elements in buffer is changed when running InferenceSession HOT 1
- CreateSessionFromArray doesn't work HOT 2
- Snapdragon X processor is unsupported HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnxruntime.