Comments (4)
youkaichao
commented on August 25, 2024
Nvidia driver version: 470.57.02
OS: EulerOS 2.0 (SP2) (x86_64)
You have a very old driver, and an OS that I don't know.
from vllm.
XianmingJin08
commented on August 25, 2024
Nvidia driver version: 470.57.02
OS: EulerOS 2.0 (SP2) (x86_64)You have a very old driver, and an OS that I don't know.
@youkaichao Thank you for your reply. Sadly, i am not able to change the driver and the OS. Is it possible that there are other reasons that caused the problem?
from vllm.
youkaichao
commented on August 25, 2024
470.57.02 is a too old driver that I can answer nothing about. please talk with your admin about the machine.
from vllm.
m0g1cian
commented on August 25, 2024
Nvidia driver version: 470.57.02
OS: EulerOS 2.0 (SP2) (x86_64)You have a very old driver, and an OS that I don't know.
@youkaichao Thank you for your reply. Sadly, i am not able to change the driver and the OS. Is it possible that there are other reasons that caused the problem?
Try use cuda-compat to soft "upgrade" your driver version so you can run CUDA 12 on driver version 470.
from vllm.
Related Issues (20)
- [Installation]: container images - too big and need to publish also cpu versions HOT 3
- [Bug]: CUDA error: an illegal memory access was encountered when running autofp8 HOT 1
- [Performance]: Block manager v2 has low throughput with prefix caching warmup HOT 3
- [Bug]: vllm server 部署base和lora模型后，请求lora模型失败 HOT 3
- [Doc]: Has the offline chat inference function been updated? HOT 1
- [Bug]: AttributeError: Model BitsAndBytesModelLoader does not support BitsAndBytes quantization yet HOT 1
- [Bug]: The error is caused by: RuntimeError: out must have shape (total_q, num_heads, head_size_og), leading to the following error: vllm.engine.async_llm_engine.AsyncEngineDeadError: Background loop has errored already. HOT 1
- [Bug]: Speculative sampling does not excatly maintain the distribution HOT 3
- [Bug]: OpenGVLab/InternVL-Chat-V1-5 never stops properly HOT 8
- [Bug]: assert num_new_tokens > 0 crashes entire worker instead of just failing single API call HOT 1
- [Feature]: Exit on failures HOT 2
- [Misc]: TTFT profiling with respect to prompt length HOT 3
- [Usage]: The results of the model depend on the number of GPUs. HOT 2
- [Feature]: support chunked_prefill for llava
- [Bug]: Gemma2 models inference using vLLM 0.5.4 produces incorrect responses
- [Bug]: Error happened with Large scale requests based on 0.5.4 vllm HOT 6
- [Misc]: OOM (CUDA Out Of Memory) when running LLMs in WSL using vLLM HOT 4
- [Feature]: json_schema support in OpenAI compat server
- Virtual Office Hours: August 8 and August 21
- [Feature]: GGUF quantization with tensor parallelism HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vllm.