instinct.cpp
is a toolkit for developing LLM-powered applications.
🚨 This project is under active development and has not reached to GA stage of first major release. See more at Roadmap section.
What instinct.cpp
offer:
- Applications that are working out-of-box.
- Assistant API server: Agent service that is fully compatible with OpenAI's Assistant API.
- mini-assistant-api: Single binary for single node deployment with vector database and other dependencies bundled.
mighty-assistant-api
: (WIP) A cloud native implementation that is highly scalable with distributed components and multi-tenant support.
- chat-agent: A CLI application that create knowledge index with your docs (PDF,TXT,MD,...) and launch an HTTP server that is fully compatible with OpenAI
ChatCompletion
.
- Assistant API server: Agent service that is fully compatible with OpenAI's Assistant API.
- Frameworks to build LLM-based applications. Say it
langchain.cpp
.- Integration for privacy-first LLM providers: Built-in support for Ollama and other OpenAI compatible API services like vllm, llama.cpp server, nitro and more.
- Building blocks for common application patterns like Chatbot, RAG, LLM Agent.
- Functional chaining components for composable LLM pipelines.
- Agent patterns: ReACT, OpenAI-based tool agent, LLMCompiler, ...
For built-in applications:
For library itself:
Complete project plan is tracked at Project kanban.
Milestone | Features | DDL |
---|---|---|
v0.1.0 | Long-short memory, PDF/TXT/DOCX ingestor, Chain programing paradigm, RAG reference app doc-agent |
3.29 |
v0.1.1 | Performance tuning, RAG evaluation, Function calling agent | 4.16 |
v0.1.2 | OpenAI Assistant API initial implementation, single-binary reference app mini-assistant |
4.30 |
v0.1.3 | * mini-assistant : tool calls with opensourced LLMs |
5.17 |
v0.1.4 | * doc-agent : rerank model* mini-assistant : file-search tool support. |
|
v0.1.5 | Overall optimization | 6.30 |
v0.1.6 | code-interpreter in mini-assistant |
7.15 |
Contributions are welcomed! You can join discord server, or contact me via email.