[Bug]: which torchvision version required about vllm HOT 12 OPEN

I am using cpu

from vllm.

The latest version should be fine. By the way, Phi-3V is not well-supported on CPU yet.

from vllm.

for cpu required torch == 2.3.1+cpu and torchvision==0.18.1 required torch == 2.3.1

torchvision==0.18.1 not working

from vllm.

@Isotr0py can you provide more insight on this?

from vllm.

@tusharraskar torchvision==0.18.1 should work fine. Maybe you can try to reinstall torch and torchvision:

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu

from vllm.

The CI failure here may be related to this issue. It seems that torchvision cannot be installed correctly alongside PyTorch for CPU.

from vllm.

@DarkLight1337 We only added torchvision to requirements-test.txt and .buildkite/run-cpu-test.sh didn't install package from it. So torchvision wouldn't be installed by default in intel-test.

from vllm.

@DarkLight1337 We only added torchvision to requirements-test.txt and .buildkite/run-cpu-test.sh didn't install package from it. So torchvision wouldn't be installed by default in intel-test.

Hmm ok, let me update that then.

from vllm.

@tusharraskar torchvision==0.18.1 should work fine. Maybe you can try to reinstall torch and torchvision:

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu

Now it's working, but the model is using only one core, which makes the process slow. I also added a screenshot.

from vllm.

@tusharraskar torchvision==0.18.1 should work fine. Maybe you can try to reinstall torch and torchvision:

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cpu

Now it's working, but the model is using only one core, which makes the process slow. I also added a screenshot

The latest version should be fine. By the way, Phi-3V is not well-supported on CPU yet.

That is exactly what I meant by this.

from vllm.

You can check #4194 for the progress on CPU support.

from vllm.

@tusharraskar I think adding dtype="bfloat16" or dtype="float" to LLM should be able to use multi-core.

from vllm.