Function Calling with LM Studio Server Model about local-llm-function-calling HOT 2 CLOSED

The whole idea of this project is to restrict the generation token-by-token. Therefore, the short answer is no, it's not possible. This project is something you would integrate into a tool like LM Studio on the server side, to make it more suitable for function calling. Unless you want to change the server configuration, what you're trying to do just boils down to careful prompt engineering, and at most validation of the result. I have another project that does the validation and parsing with OpenAI. If you're receiving the full completion, though, there is no way to enforce the actually valid JSON being generated.

from local-llm-function-calling.

The whole idea of this project is to restrict the generation token-by-token. Therefore, the short answer is no, it's not possible. This project is something you would integrate into a tool like LM Studio on the server side, to make it more suitable for function calling. Unless you want to change the server configuration, what you're trying to do just boils down to careful prompt engineering, and at most validation of the result. I have another project that does the validation and parsing with OpenAI. If you're receiving the full completion, though, there is no way to enforce the actually valid JSON being generated.

LM studio just added "{} Structured Output (enforce JSON)"; I am not really sure how to go about this. Do you have have any ideas? I am just trying to use function calling with the "tool params" that openAI uses so i do not have to convert my entire project when switching to a local LLM.

from local-llm-function-calling.