A project for versatile AI agents that can run with proprietary models or completely open-source. The meta expert has two agents: a basic Meta Agent, and Jar3d, a more sophisticated and versatile agent.

1. Core Concepts
2. Prerequisites
   • Environment Setup
3. Repository Setup
4. API Key Configuration
5. Basic Meta Agent Setup
6. Jar3d Setup
   • Setting up the NLM-Ingestor Server
7. Working with Ollama
8. On the Roadmap for Jar3d

This project leverages three core concepts:

1. Meta prompting: For more information, refer to the paper on Meta-Prompting (source). Read our notes on Meta-Prompting Overview for a more concise overview.
2. Chain of Reasoning: For Jar3d, we also leverage an adaptation of Chain-of-Reasoning
3. Jar3d uses retrieval augmented generation, which isn't used within the Basic Meta Agent. Read our notes on Overview of Agentic RAG.

1. Install Anaconda:
   Download Anaconda from https://www.anaconda.com/.

2. Create a Virtual Environment:

   conda create -n agent_env python=3.11 pip

3. Activate the Virtual Environment:

   conda activate agent_env

1. Clone the Repository:

   git clone https://github.com/brainqub3/meta_expert.git

2. Navigate to the Repository:

   cd /path/to/your-repo/meta_expert

3. Install Requirements:

   pip install -r requirements.txt

1. Open the config.yaml file:

   nano config.yaml

2. Enter API Keys:

   • Serper API Key: Get it from https://serper.dev/
   • OpenAI API Key: Get it from https://openai.com/
   • Gemini API Key: Get it from https://ai.google.dev/gemini-api
   • Claude API Key: Get it from https://docs.anthropic.com/en/api/getting-started
   • Groq API Key: Get it from https://console.groq.com/keys

The basic meta agent is an early iteration of the project. It demonstrates meta prompting rather than being a useful tool for research. It uses a naive approach of scraping the entirety of a web page and feeding that into the context of the meta agent, who either continues the task or delivers a final answer.

python -m agents.meta_agent

Then enter your query.

Jar3d is a more sophisticated agent that uses RAG, Chain-of-Reasoning, and Meta-Prompting to complete long-running research tasks.

Try Jar3d with:

• Writing a newsletter - Example
• Writing a literature review
• As a holiday assistant

Jar3d is in active development, and its capabilities are expected to improve with better models. Feedback is greatly appreciated.

Jar3d is capable of running 100% locally. However, the setup itself is non-trivial. There are two components to set up:

1. The nlm-ingestor server
2. Running the application

Running Jar3d requires you to set up your own backend server to run the document parsing. We leverage the nlm-ingestor and llm-sherpa from NLMatics to do this.

The nlm-ingestor uses a modified version of Apache Tika for parsing documents. The server will be deployed locally on whatever machine you run the Docker image.

The server provides an easy way to parse and intelligently chunk a variety of documents including "HTML", "PDF", "Markdown", and "Text". There is an option to turn on OCR; check out the docs.

1. Ensure you have Docker installed on your machine. Once installed, ensure you have started the Docker daemon.

2. Next, pull the Docker image from nlmatics:

   docker pull ghcr.io/nlmatics/nlm-ingestor:v0.1.6

3. Once you have pulled the image, you can run the container:

   docker run -p 5010:5001 ghcr.io/nlmatics/nlm-ingestor:v0.1.6

4. Navigate to config/config.yaml and check that the LLM_SHERPA_SERVER variable is exactly like this:

   http://localhost:5010/api/parseDocument?renderFormat=all
   

   Note: You can change the port mapping from 5010 to whatever mapping you want. You must ensure that it is consistent with the mapping you select in the docker run command.

5. Once you're ready, you can start the RAG-agent by running the following from the meta_expert directory:

   python -m agents.jar3d

1. Download Ollama: Download from https://ollama.com/download

2. Download an Ollama Model:

   curl http://localhost:11434/api/pull -d "{\"name\": \"llama3\"}"

For more information, refer to the Ollama API documentation.

• Feedback to Jar3d so that final responses can be iterated on and amended.
• Long term memory.
• Frontend.
• Integrations to RAG platforms for more intelligent document processing and faster RAG.