Running Vllm on ray cluster, logging stuck at loading about vllm HOT 3 OPEN

@simon-mo can i get some help ?

from vllm.

This setup is a bit hard to debug because while the multi-node setting should work out of the box, the hanging could be due to connectivity issue between two machines.

from vllm.

thanks for your response @simon-mo

• the ray cluster can see both machine ray status returns that the available resources are 2 GPUs.
• nvidia-smi seems shows that ray already allocated the resources need for loading the model.
• my setup for the firewall was to allow any request from each server to the other server.

what could make the setup not suitable for the application ?

from vllm.