Comments (4)
The error was thrown from here:
https://github.com/intel/intel-extension-for-pytorch/blob/e4a645649744bf03fa2c3d90b771d48a8dc204fc/csrc/cpu/aten/kernels/RMSNormKrnl.cpp#L66C25-L66C47
Even though I have no idea why, inputs are neither float nor bfloat16? cc @jianan-gu
from intel-extension-for-pytorch.
@mozillazg Hi, please check the first line of this notes~
https://github.com/intel/intel-extension-for-pytorch/tree/main/examples/cpu/inference/python/llm#423-additional-configuration-for-specific-models
ChatGLM models have some hard coded init with torch.float16
dtype, we have to change it to torch.float32
in config.json
from intel-extension-for-pytorch.
@jianan-gu oh! sorry, my bad! Thanks for your reminder.
BTW,
- how about place these notes before 4.1 section?
- asking the user to change the config.json of the model may not be a good idea ?
from intel-extension-for-pytorch.
ChatGLM models have some hard coded init with
torch.float16
dtype, we have to change it totorch.float32
in config.json
Is it possible to reinit the RotaryEmbedding
inside IPEX for such a case? Then, we don't require users to change the script.
from intel-extension-for-pytorch.
Related Issues (20)
- Hi @qslia , thanks for reporting this issue. Yes we support Arc graphics.
- '_IPEXLinear' object has no attribute 'use_dnnl' HOT 5
- NotImplementedError: Could not run 'aten::empty_strided' HOT 3
- Speech Translation using Coqui.ai on Intel Arc GPU 770 takes 23 seconds compared to CPU [3 sec] why ? HOT 8
- Undefined Symbols with oneAPI 2024.0.0 HOT 4
- No Devices Visible with oneAPI 2024.1 and Intel GPU Max 1550 GPUs HOT 5
- NotImplementedError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript (most recent call last): RuntimeError: Could not run 'torch_ipex::mul_add' with arguments from the 'CPU' backend. HOT 3
- When is ipex available for pytorch 2.3.0 ? HOT 2
- no XPU device found with IPEX-v2.1.30-xpu and oneAPI2024.1 releases running with MAX-1550 HOT 2
- Memory usage is low HOT 10
- GPT-J benchmarking fails with IPEX 2.2.0/2.3.0 HOT 9
- [WSL]"XPU out of memory" error when using to("xpu") method in Intel PyTorch Extension (IPEX) HOT 5
- failed to apply concat_linear on unet HOT 3
- [PyTorch-XPU] NotImplementedError: No registered fallback function for aten::view HOT 5
- Error loading "{dll}" or one of its dependencies after following windows installation instructions HOT 3
- crash when out token length is > 64 HOT 4
- `ERRORLEVEL` not equal to 0 when moving an tensor to XPU on Windows HOT 1
- Can a model quantized using IPEX be run with torch.compile enabled? HOT 11
- Preview changes in ComfyUI causes segmentation fault and etc. with IPEX v2.1.30+xpu HOT 12
- bfloat16 is slower than float32 on Intel Xeon Platinum 8481C CPU HOT 13
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from intel-extension-for-pytorch.