Navigating RAG Optimization with an Evaluation Driven Compass: Atita Arora and Deanna Emery
Nov 12, 2024 · 18:14
Atita Arora (Qdrant) and Deanna Emery (Quotient AI) demonstrate how evaluation-driven optimization improves Retrieval Augmented Generation (RAG) systems, using Qdrant's vector database and Quotient's evaluation platform. They detail a 10-experiment pipeline on Qdrant documentation, starting with naive RAG and incrementally testing chunk sizes, embedding models, LLMs (Mistral, GPT-3.5), re-rankers (MixedBread, Cohere, ColBERT), and hybrid search. Key metrics—faithfulness (hallucination reduction), context relevance, and chunk relevance—guide decisions: increasing chunk size hurt faithfulness (0.76 to 0.72), while smaller chunks with larger retrieval windows improved it. Switching to GPT-3.5 lifted all metrics, but poor context relevance revealed retrieval issues. Adding Cohere re-ranking boosted faithfulness to 0.82, and hybrid search with re-ranking achieved 0.85, proving domain-specific terminology demands tailored search strategies. The episode stresses avoiding over-engineering without metrics, keeping evaluation datasets current, and using domain understanding for systematic RAG improvement.
