Related Issues (20)
- [Bug]: Does not work on MacOS HOT 2
- [Usage]: GPTQ quantization behavior HOT 2
- [Feature]: CI - Split up "Models Test" and "Vision Language Models Test"
- [Bug]: base_model.model.score.weight is unsupported LoRA weight HOT 1
- [Feature]: sliding window attention for odd layers
- [Feature][Encoder-Decoder]: Add support for cuda graph during decoding in encoder-decoder models
- [Feature]: Integrate `flash-infer` FP8 KV Cache Chunked-Prefill (Append Attention) HOT 3
- [Bug]: (VllmWorkerProcess pid=3253) WARNING 08-13 11:31:37 shm_broadcast.py:386] No available block found in 60 second HOT 1
- [Misc]: I want to run Llama 3.1 405B using speculative. Can you give me a guide? HOT 1
- [Usage]: release notes,best practice, abort-actively HOT 1
- [Feature]: early-stop in mooncake
- [Usage]: openai.APIStatusError: Error code: 405 - {'detail': 'Method Not Allowed'} HOT 2
- [Bug]: Gemma-2-2b-it load model hangs by vLLM==0.5.1 on Tesla T4 GPU HOT 5
- [Feature]: Use .env file to deploy vllm HOT 3
- [Usage]: Got nccl error when deploy vllm in k8s with multiple GPUs HOT 5
- [Feature]: ROCm 6.2 support & FP8 Support
- [Bug]: BlockManagerV2 allocate with Sliding Window Attention HOT 2
- [Bug]: FP8 Quantization support for AMD GPUs HOT 1
- [Bug]: a bug in CUDA capabilities test with different GPUs available HOT 11
- [Bug]: Model serving failed with these arguments --tensor-parallel-size 2 --pipeline-parallel-size 2 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vllm.