Discover the latest LLM implementations in production. 🚀

Learn how big tech companies and startups implement and leverage LLMs in 2024:

• How LLMs are deployed and integrated into large-scale applications 🔎
• What architectures and techniques worked when implementing LLM in the SWD cycle ✅ (Data Quality, Data Engineering, Serving, Monitoring 📈 etc)
• What real-world results were achieved (so you can better assess ROI ⏰💰)
• Why it works and what is the science behind it with research, literature, and references 📂

Feel free to contribute!

1. Training and Fine-tuning Techniques
2. Data Quality for LLMs
3. Data Engineering with LLM
4. LLM Deployment
5. Evaluation and Metrics for LLMs
6. Prompt Engineering
7. Vector Stores
8. Tools and Frameworks
9. Retrieval Augmented Generation (RAG)
10. Graph and LLMs
11. Multimodal with LLMs
12. Scaling and Optimization
13. Ethical Considerations and Limitations

14. LLM Seminal Papers
15. Courses and Tutorials
16. GitHub Repositories
17. LLM Tools for Developers
18. Team Structure and Strategy
19. Newsletters to follow

• Imbue: Training a 70B Model from Scratch
• Google: PaLM 2 Technical Report
• Anthropic: Training language models to follow instructions with human feedback
• EleutherAI: The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Hugging Face: How we trained BLOOM, the world's largest open multilingual language model
• DeepMind: Training Compute-Optimal Large Language Models
• NVIDIA Megatron-Turing NLG

• Nvidia: Scale and Curate High-Quality Datasets for LLMs
• Data Quality Error Detection with LLMs
• IBM: How to ensure Data Quality and Reliability
• NVIDIA: Curating custom datasets for LLM training

• Harnessing the power of LLMs in Data Engineering
• Data Engineer 2.0
• Data Collection Magic
• Data Engineers, here is how LLMs can make your life easier

• Efficient Large Language Model serving with BentoML
• Databricks: Model Serving
• TensorRT-LLM: Toolkit to optimize inference of LLMs
• Datyabricks: Deploying Large Language Models in Production
• Patterns for Building LLM-based Systems & Products
• LLM Engineering Guide

• OpenAI: Evaluating Large Language Models Trained on Code
• EleutherAI: HELM - Holistic Evaluation of Language Models
• Google: Beyond the Imitation Game - Measuring and extrapolating the capabilities of language models
• DeepMind: Measuring Massive Multitask Language Understanding
• Best Practices for LLM Evaluation
• LLM Evaluation Framework

• Google Cloud: Best practices for prompt engineering with LLMs
• Prompt Engineering Guide
• OpenAI Cookbook: Techniques to improve reliability
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models

• Building a Vector Search Engine with Faiss
• Introducing ScaNN: Efficient Vector Similarity Search
• Milvus: An Open Source Vector Database for Scalable Similarity Search
• Vector Similarity Search: From Basics to Production
• Weaviate: The Open Source Vector Database
• Qdrant: Vector Database for the Next Generation of AI Applications

• Hugging Face Transformers
• LangChain: Building applications with LLMs through composability
• OpenAI API
• Anthropic Claude API
• Llama.cpp: Inference of LLaMA model in pure C/C++

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Improving language models by retrieving from trillions of tokens
• Retrieval-Augmented Generation: A Survey
• REALM: Retrieval-Augmented Language Model Pre-Training
• Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
• Atlas: Few-shot Learning with Retrieval Augmented Language Models
• In-Context Retrieval-Augmented Language Models

• Combining Knowledge Graphs and Large Language Models
• Graph Neural Networks and Language Models: A Powerful Combination
• Knowledge Graphs and Language Models: Bridging the Gap
• Enhancing Language Models with Knowledge Graph Embeddings
• Graph-augmented Learning for Language Understanding
• Integrating Knowledge Graphs with Large Language Models
• Graph-based Neural Language Models

• DALL·E 2: Extending Language Models to Images
• PaLM-E: An Embodied Multimodal Language Model
• Flamingo: a Visual Language Model for Few-Shot Learning
• CLIP: Connecting Text and Images
• Multimodal Few-Shot Learning with Frozen Language Models
• VisualBERT: A Simple and Performant Baseline for Vision and Language

• DeepSpeed: Deep learning optimization library
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
• Challenges and Applications of Large Language Models
• AI Ethics Guidelines Global Inventory
• Ethical and social risks of harm from Language Models

1. Attention Is All You Need (Transformer Paper)
2. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
3. GPT-3: Language Models are Few-Shot Learners
4. LLaMA: Open and Efficient Foundation Language Models
5. A Survey of Large Language Models
6. Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
7. Awesome-LLM: A curated list of Large Language Model resources
8. WebGPT: Browser-assisted question-answering with human feedback
9. InstructGPT: Training language models to follow instructions with human feedback

1. DeepLearning.AI: ChatGPT Prompt Engineering for Developers
2. Coursera: Natural Language Processing Specialization
3. Fast.ai: Practical Deep Learning for Coders
4. Stanford CS224N: Natural Language Processing with Deep Learning
5. Hugging Face: NLP Course
6. Creme de la Creme of Free AI courses

1. Awesome Machine Learning
2. Applied ML
3. Awesome Scalability
4. Made with ML
5. The Algorithms
6. TensorFlow Models
7. Transformers Examples
8. Sentence Transformers

1. OpenAI Playground
2. Hugging Face Spaces
3. Gradio: Build Machine Learning Web Apps
4. Streamlit: The fastest way to build data apps
5. LangChain: Building applications with LLMs through composability

1. Google: Machine Learning: The High Interest Credit Card of Technical Debt
2. Spotify: How We Structure Our ML Teams
3. Uber: Scaling Machine Learning at Uber with Michelangelo
4. Netflix: Human-Centric Machine Learning Infrastructure at Netflix
5. Airbnb: Scaling Knowledge Access and Retrieval at Airbnb

1. Meri Nova's
2. Ahead of AI
3. Breaking into Data
4. Underfitted
5. Marvelous MLOps
6. SemiAnalysis