This open-source project demonstrates Retrieval Augmented Generation (RAG) using Large Language Models (LLMs).

siosio-rag uses the following technologies:

• Orchestration: LangChain.
• Embedding Vectorstore: Weaviate.
• Text Embeddings: FlagEmbedding.
• LLMs: OpenAI ChatGPT and Google Gemini.
• Evaluation: Ragas.

Reference: chat-langchain

• Python (version 3.10 or higher)
• PyTorch (version 2.1.2 or higher)

Once you have these dependencies installed, clone this repository and set up the environment:

git clone https://github.com/your-username/siosio-rag.git

pip install -r requirements.txt

Run prepare_data.py to preprocess and save the data:

python prepare_data.py --out_path out_directory_path

Run build_index.py to convert data into embeddings and build an index:

python build_index.py --input_path data_directory_path --out_path out_directory_path --index_name name_for_index

Run rag.py to execute RAG:

python rag.py \
    --input_path csv_path \
    --output_path out_directory_path \
    --index_path index_path \
    --index_name index_name \
    --model_name llm_model_name \
    --api_key api_key_for_llm

Run evaluate.py to evaluate the RAG results:

python evaluate.py \
    --input_path result_csv_path \
    --output_path out_directory_path \
    --api_key api_key_for_llm

Note:

1. Data used in the experiment can be found here.
2. The results were obtained with Ragas v1.7.
3. For information about the metrics, please refer to this doc.

This project is licensed under the MIT license. Please refer to the LICENSE file for details.