Comments (3)
youkaichao
commented on July 17, 2024
can you follow https://docs.vllm.ai/en/latest/getting_started/debugging.html to dig out the cause of the error?
from vllm.
eByteTheDust
commented on July 17, 2024
I solved the problem setting NCCL_SOCKET_IFNAME=eth0 on both ray head and nodes. Now it loads with no errors. P.S.: another error message I found in the log was: "Error executing method determine_num_available_blocks. This might cause deadlock in distributed execution"
but again, settting NCCL_SOCKET_IFNAME solved the problem and v0.5.0.post1 loads with no errors.
from vllm.
youkaichao
commented on July 17, 2024
glad it works. I just added NCCL_SOCKET_IFNAME in https://docs.vllm.ai/en/latest/getting_started/debugging.html
from vllm.
Related Issues (20)
- [Usage]: Multi-LoRA questions
- [Bug]: Inconsistent Output from OPT-x models
- [Feature]: Support in distributed speculative inference
- [Bug]: Neuron offline inferenc example assertion error
- [Installation]: ValueError: Quantization method specified in the model config (gptq) does not match the quantization method specified in the `quantization` argument (gptq_marlin). HOT 4
- [Bug]: VLM same chat different image results in serving_chat.py:238 Error in loading image data: HOT 5
- [Bug]: OutOfMemoryError when loading a small model with a huge context length HOT 3
- [CI] [Flaky test] distributed/test_shm_broadcast.py is flaky HOT 3
- [Bug]: server error when hosting TheBloke/Llama-2-7B-Chat-GPTQ with chunked-prefill HOT 1
- [Misc]: CUDAGraph captured generation stuck with custom_all_reduce and tensor_parallel=2 HOT 2
- [Feature]: Supported For Quanto Models
- [Feature]: support logging input and output
- [Bug]: ray version problem HOT 3
- [Usage]: Can I get the streaming output when using offline inference? HOT 1
- [Bug]: A bug when running examples/llava_example.py with image_features as input and multiple GPUs enabled HOT 14
- [Bug]: ValueError: Model architectures ['Phi3VForCausalLM'] are not supported for now. HOT 3
- [Bug]: The end_sync operation inside the cross_device_reduce_2stage kernel sometimes deadlocks because it can't wait for the end signal.
- IP Bind Error on v0.5.0.post1 HOT 14
- [Bug]: LLaVa Next Value Error - "Incorrect type of image sizes" when running in Docker HOT 5
- [Bug]: vllm stuck when using prompt_token_ids and setting prompt_logprobs HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vllm.