How we taught agents to use good retrieval - Hanna Lichtenberg, Mixedbread AI
Jul 7, 2026 · 14:28
Mixedbread AI co-founder Amir and AI engineer Hanna Lichtenberg argue the main bottleneck for knowledge agents is not reasoning but retrieval, coining the 'Oracle Gap'—the performance drop when agents use default tools. On BrowsComPlus, Oracle accuracy is 93% but Codex drops 9 points; with Mixedbread's search they recover to within 3 points. They show models write keyword gibberish (e.g., 'Senator, women, questions, billionaires, not a company') because they were trained on code tools like grep. To fix this, they built a search agent with 4 tools (overview, semantic, filter, grep) using a harness that encourages natural query sentences. Training involved supervised fine-tuning from a larger teacher and reinforcement learning with a reward combining retrieval metrics and trajectory quality (e.g., judging if queries are natural sentences). Their agent achieved NDCG@10 of 0.4 on Oblique Congress (vs. 0.18 for GPT multihop) and topped Snowflake's MedQA benchmark with 93.4% accuracy using Gemini 3.5 Flash.
