Comments (8)
+1
from vllm.
found the related docker commands
https://hub.docker.com/layers/vllm/vllm-openai/v0.4.3/images/sha256-d55c38328bfda7b049dd6c3695d60fff8d88a63b19db058791879dc6bf3188af?context=explore
what new features require cuda>=12.4 ?
from vllm.
we use 12.4 because the old versions of base container will be deleted soon, according to nv policy https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/support-policy.md .
from vllm.
It seems even v0.4.3 is now built with cuda 12.4 required.
from vllm.
we use 12.4 because the old versions of base container will be deleted soon, according to nv policy https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/support-policy.md .
I see. Thanks.
from vllm.
It seems even v0.4.3 is now built with cuda 12.4 required.
Yes. But the document didn't update with that. https://docs.vllm.ai/en/v0.4.3/getting_started/installation.html
from vllm.
hopefully this can be fixed when i fix #5646
from vllm.
hopefully this can be fixed when i fix #5646
great!
from vllm.
Related Issues (20)
- [Bug]: Mismatch in the number of image tokens and placeholders during batch inference HOT 2
- [Bug]: Streaming API: Abort functionality not working as expected
- [Usage]: how to use guided_decoding_backend? HOT 1
- [Feature]: Please Support FATRelu
- [Usage, bug]: vLLM Docker | ValueError: OpenTelemetry packages must be installed before configuring 'otlp_traces_endpoint' during vLLM startup
- [Performance]: vllm量化和非量化的性能对比 HOT 3
- [Bug]: when `echo=True`, vllm will append chat template(`assistant`) after the last message HOT 6
- [Misc]: how to add tests for new backends? HOT 1
- [Feature]: Pipeline Parallelism support for the Vision Language Models HOT 4
- [Usage]: Qwen2 GGUF model can't run successfully HOT 7
- [Feature]: Overlap model weight loading and model prefill HOT 2
- [RFC]: Add Ascend NPU as a new backend
- [Bug]: Inconsistent Output Behavior with and without tools and tool_choice Parameters
- [Usage]: Periodic snapshots for spot instances HOT 1
- [RFC]: Enable Memory Tiering for vLLM HOT 5
- [Bug]: vLLM inconsistently crashes on startup for multinode cluster HOT 7
- [Usage]: how to abort request? HOT 1
- [Usage]: alignment between trl and llm.generate
- [Bug]: errors when loading mixtral 8x7b
- [Bug]: The MixtralForCausalLM architecture and the mistralai/Mixtral-8x7B-Instruct-v0.1 model are stated to be supported by vLLM, but an error occurs during model loading. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vllm.