[Bug]: AutoGen can't work with vLLM v0.5.1 about autogen HOT 7 OPEN

we integrated tools into vllm with function calling models. might be relevant: https://docs.rubra.ai/inference/vllm

from autogen.

I can suggest a couple of approaches:

• A vLLM client class that can handle this empty tool_calls
• A change to the existing AutoGen codebase that ignores empty tool_calls

If someone wants to work on a PR that would help.

from autogen.

Hey @tonyaw, this is a bit tricky in my opinion. I feel that it should return None if there are no tool calls to be made rather than an empty list, []. The finish_reason being stop indicates that it is not suggesting tool calls in this response.

What do you think?

from autogen.

@marklysze , I'm OK with both None and "[]" as long as it is aligned between agent framework(autogen) and LLM inference framework(vllm). :-)
As it follows OpenAI API schema, may I ask if there is some detail requirement from API schema perspective?

from autogen.

I also opened a same ticket to vllm. Let's align with vllm team for an agreement. :-)

from autogen.

@sanjay920,
Thanks for your info!

1. Is it possible to contribute your code change to vllm git repo? :-)
2. Have you tried your vllm with autogen?

from autogen.

Is there any possible way to bypass this?... This really gives me headache...

from autogen.