Comments (2)
The whole idea of this project is to restrict the generation token-by-token. Therefore, the short answer is no, it's not possible. This project is something you would integrate into a tool like LM Studio on the server side, to make it more suitable for function calling. Unless you want to change the server configuration, what you're trying to do just boils down to careful prompt engineering, and at most validation of the result. I have another project that does the validation and parsing with OpenAI. If you're receiving the full completion, though, there is no way to enforce the actually valid JSON being generated.
from local-llm-function-calling.
The whole idea of this project is to restrict the generation token-by-token. Therefore, the short answer is no, it's not possible. This project is something you would integrate into a tool like LM Studio on the server side, to make it more suitable for function calling. Unless you want to change the server configuration, what you're trying to do just boils down to careful prompt engineering, and at most validation of the result. I have another project that does the validation and parsing with OpenAI. If you're receiving the full completion, though, there is no way to enforce the actually valid JSON being generated.
LM studio just added "{} Structured Output (enforce JSON)"; I am not really sure how to go about this. Do you have have any ideas? I am just trying to use function calling with the "tool params" that openAI uses so i do not have to convert my entire project when switching to a local LLM.
from local-llm-function-calling.
Related Issues (14)
- About the parameter extraction? HOT 3
- Problems when trying to use Llama-2 models for function calling HOT 2
- Using CUDA for inference
- How does this project be compatible with BPE tokenizers? HOT 1
- Lm studio support?
- ERROR: No matching distribution found for local-llm-function-calling HOT 1
- is it possible for us to connect over mail? HOT 1
- Could not install via pip HOT 2
- can this be used for function calling with other opensource models? HOT 7
- Docs example doesn't seem correct HOT 2
- Error at installing HOT 1
- Benchmark data HOT 1
- Function calling Issue HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from local-llm-function-calling.