Comments (2)
Hi @sujikarNStarx, to load LLM model, OpenVINO models can be run locally using the HuggingFacePipeline class by specifying the backend;="openvino" parameter to trigger the backend inference framework. However, as of Intel OpenVINO version 2024.0, Large Language models (LLMs) are not supported by the Intel NPU plugin.
You may refer to Create a RAG system using OpenVINO and Langchain.
from openvino_notebooks.
Closing issue, feel free to re-open or start a new issue if additional information assistance is needed.
from openvino_notebooks.
Related Issues (20)
- Installed openvino notebook using docker. But check_install.py failed HOT 8
- Unable to compile Llama3 8B model for NPU using Openvino (we have latest NPU drivers - April 2024 version) HOT 2
- [Issue]: KeyError: "phi3 is not supported yet for transformers. HOT 1
- There is a problem with openvino's yolov8 tutorial. HOT 5
- Pip install intel-extension-for-pytorch does not work. HOT 2
- LLM-chatbot llama3 model issue HOT 1
- setuptools version error
- [GH Pages] Any buttons on notebook cards will open its document anyway instead of doing it's expected action
- llm-rag-langchain answer wrongly when using llama-3-8b-instruct HOT 5
- notebooks\hello-npu return error HOT 25
- Issue running Yolov8 segmentation example HOT 4
- Why not python sample code instead of Colab? HOT 2
- iGPU memory usage problem HOT 8
- supplementary_materials /qwen2 have api HOT 2
- Cannot create StringTensorUnpack layer StringTensorUnpack_220719 id:5 from unsupported opset: extension HOT 2
- Custom AI Assistant recipe fails to convert and optimize model HOT 2
- llm-chatbot-generate-api notebook fails during pip / git install routine HOT 3
- openVINO2024.3.0 gave RuntimeError:[GPU] ProgramBuilder build failed! at notebook:3D-segmentation-point-clouds, when using Integrated GPU HOT 2
- Problem with openvino's yolov8-keypoint-detection tutorial HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openvino_notebooks.