"Qwen2-VL FP16 inference results in errors or gibberish output." about transformers HOT 3 OPEN

The language model of the Qwen2 series does not support float16 reasoning. The official document mentioned this. Please refer to https://qwen.readthedocs.io/en/latest/inference/chat.html and check Troubleshooting section

from transformers.

Qwen2系列的语言模型不支持float16推理，官方文档提到了这一点，请参考https://qwen.readthedocs.io/en/latest/inference/chat.html 并查看Troubleshooting部分

For NVIDIA GPUs, being able to perform inference with float32 and bfloat16 but not with float16 is highly unusual and goes against common sense.
I have resolved this issue and will submit a PR for everyone's reference after this PR is merged.

from transformers.

Qwen2系列的语言模型不支持float16推理，官方文档提到了这一点，请参考https://qwen.readthedocs.io/en/latest/inference/chat.html 并查看Troubleshooting部分

Thank you. I've taken a look at the Qwen team's solution, and while it's different from mine, it's quite valuable for reference. I'll test it out tomorrow.

from transformers.