Run document Q&A on Wikipedia articles. Use Wikipedia-API to search/retrieve/beautify Wikipedia articles, LangChain for the Q&A framework, and OpenAI & HuggingFace models for embeddings and LLMs. The meat of the code is in WikipediaQA.py.

For the accompanying blog post, see https://georgesung.github.io/ai/llm-qa-eval-wikipedia/

For a batch run over different LLMs and embedding models, you can run the notebook WikipediaQA_batch_runs.ipynb in your own compute instance, or run the same notebook on Colab:

To run an interactive Gradio app, do the following:

• pip install -r requirements.txt
• If you're using OpenAI ada embeddings and/or GPT 3.5, then cp template.env .env, and edit .env to include your OpenAI API key
• python gradio_app.py

For detailed results and analysis, see the full blog post here