Comments (4)
evshiron
commented on June 23, 2024
pip3 uninstall bitsandbytes should do the trick.
AutoGPTQ just adds support for ROCm days ago, you can do ROCM_VERSION=5.6 pip3 install auto-gptq and it should work (and quantization works too). I haven't used BitsAndBytes for a long time due to bad ROCm support and low performance.
If it doesn't compile, please report back, and try this:
Clone https://github.com/PanQiWei/AutoGPTQ, add these lines:
__host__ __forceinline__ hipblasStatus_t hipblasHgemm(hipblasHandle_t handle,
hipblasOperation_t transA,
hipblasOperation_t transB,
int m,
int n,
int k,
const half* alpha,
const half* AP,
int lda,
const half* BP,
int ldb,
const half* beta,
half* CP,
int ldc) {
return hipblasHgemm(handle, transA, transB, m, n, k,
reinterpret_cast<const hipblasHalf *>(alpha),
reinterpret_cast<const hipblasHalf *>(AP), lda,
reinterpret_cast<const hipblasHalf *>(BP), ldb,
reinterpret_cast<const hipblasHalf *>(beta),
reinterpret_cast<hipblasHalf *>(CP), ldc);
}to autogptq_cuda/exllama/hip_compat.cuh, and then cd AutoGPTQ && pip3 install -e ..
If there is a need, I might check and update BitsAndBytes builds in this repo for ROCm 5.6 compatibility, but no new feature will be supported.
Btw, I have a plan to update the article about text-generation-webui, but haven't had time to do it.
from rocm_lab.
ZhenyaPav
commented on June 23, 2024
Removing bitsandbytes removes the issue. ExLlama works fine. Haven't tried auto-gptq yet.
from rocm_lab.
ZhenyaPav
commented on June 23, 2024
Running pip3 install -e . fails with this error:
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/home/zhenyapav/Projects/text-generation-webui/repositories/AutoGPTQ/setup.py", line 52, in <module>
CUDA_VERSION = "".join(os.environ.get("CUDA_VERSION", default_cuda_version).split("."))
AttributeError: 'NoneType' object has no attribute 'split'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
It does work with ROCM_VERSION=5.6 pip install .
from rocm_lab.
evshiron
commented on June 23, 2024
Yeah. You have to add that ROCM_VERSION=5.6 to build it for ROCm, at least for now.
from rocm_lab.
Related Issues (15)
- Support for torchaudio wheels HOT 5
- Why are we using a different repo for automatic1111 build script? HOT 1
- how can i solve the problem in training lora with 7900xtx HOT 1
- Error running ghcr.io/evshiron/rocm_lab:rocm5.5-text-gen-webui 7dea7110f293 HOT 13
- ROCM5.7 build pytorch failed HOT 36
- what about ait HOT 1
- Roadmap HOT 5
- Generation not starting locally HOT 13
- Copy to VRAM hanging HOT 56
- 7900xtx get confused sdpa result HOT 2
- use wheels on artix? HOT 2
- stable diffusion segement faults HOT 3
- bitsandbytes 0.39.0? HOT 7
- Any progress on Rocm 5.6 ? HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rocm_lab.