This project implements a local QA system by combining RAG with LangChain. LangChain's library assists in building the RAG pipeline, which leverages a powerful LLM hosted on OLLAMA. Notably, this system operates entirely on your local machine, offering privacy and control over your data. A user-friendly Streamlit interface visualizes the process and results.
Dependencies:
- langchain
- streamlit
- streamlit-chat
- pypdf
- chromadb
- fastembed
pip install langchain streamlit streamlit_chat chromadb pypdf fastembed
This project creates a local Question Answering system for PDFs, similar to a simpler version of ChatPDF. It leverages Langchain, Ollama, and Streamlit for a user-friendly experience.
Tech Stack:
Ollama: Provides a robust LLM server that runs locally on your machine.
Langchain: A powerful library used to build the Retrieval-Augmented Generation (RAG) pipeline.
Chroma: A vector storage solution that integrates well with Langchain.
Streamlit: Creates a straightforward user interface for uploading PDFs and asking questions.
Building the App:
Setting Up Ollama: Download and run the Ollama application suitable for your OS. Install the "Mistral-7B" LLM model within Ollama for its compact size and good performance.
RAG Pipeline Construction: This core component handles ingesting and processing user queries.
ingest(pdf_file_path):
Loads the PDF using PyPDFLoader.
Splits the document into manageable chunks with RecursiveCharacterTextSplitter.
Uses Chroma for vector storage with FastEmbedEmbeddings for lightweight processing.
Creates a retriever to find relevant passages based on user queries using vector similarity search.
Builds a Langchain conversation chain to connect these functionalities.
ask(query):
Takes the user's question and feeds it through the predefined chain, retrieving the answer from the LLM server.
Drafting a Simple UI with Streamlit:
The code utilizes Streamlit to create a user-friendly interface.
Users can upload a PDF document.
A chat window displays the conversation history.
Users can type questions related to the uploaded document and receive answers generated by the LLM system.
Running the Project:
Save the code as app.py and execute streamlit run app.py to launch the application.