RAG in 2025: State of the Art and the Road Forward — Tengyu Ma, MongoDB (acq. Voyage AI)
Jun 27, 2025 · 18:48
Tengyu Ma, Chief AI Scientist at MongoDB (acquired Voyage AI), argues that retrieval-augmented generation (RAG) will outlast fine-tuning and long-context models for enterprise AI because it mirrors how humans use libraries—retrieving only relevant information rather than memorizing or reprocessing entire corpora. He reports that Voyage's embedding models now achieve ~80% average accuracy across 100 datasets, with half exceeding 90%, and that 100x storage compression via matryoshka learning and quantization loses only 5–10% accuracy. Ma advocates hybrid search with rerankers and domain-specific embeddings (e.g., for code) to further improve retrieval. He predicts the model layer will absorb current 'tricks' like query decomposition and contextual chunking, simplifying RAG pipelines. New products like multimodal embeddings (accepting screenshots of PDFs, tables, videos) and auto-chunking embeddings that incorporate cross-chunk metadata are designed to shift complexity from users to model providers.
