Applying Evaluation Driven Development (EDD) to aid in the design decision of RAG pipelines. Specifically, this repo demonstrates how to use EDD to decide which of these two strategies perform best for a multi document RAG pipeline:

• Recursive retriever + document agent
• Metadata replacement + node sentence window

See Colab notebook edd_recursive_doc_agent_metadata_replacement for detailed implementation.

Check out my Medium blog post for details. Evaluation Driven Development, the Swiss Army Knife for RAG Pipelines.

This notebook demonstrates how to use EDD to decide which of the two LLMs perform best for a multi document RAG pipeline for Metadata replacement + node sentence window:

• gpt-3.5-turbo
• zephyr-7b-alpha

See Colab notebook edd_zephyr_7b_gpt3_5_metadata_replacement_multi_doc for detailed implementation.

Check out my Medium blog post for details. Exploring zephyr-7b-alpha Through the Lens of Evaluation Driven Development.