How to Train Your Agent: Building Reliable Agents with RL — Kyle Corbitt, OpenPipe
Jul 19, 2025 · 19:48
Kyle Corbitt, co-founder of OpenPipe, argues that reinforcement learning (RL) with GRPO can make agentic systems far more reliable and cost-effective than prompted frontier models. He presents ART-E, an email assistant trained on Qwen 2.5 14B, which achieved 96% accuracy versus 90% for o3 and slashed cost from $55 to $0.80 per 1,000 queries. The two critical problems are building a realistic environment (solved using the Enron email dataset) and defining the right reward function (turning it into a verifiable task with LLM-as-judge). Extra rewards—favoring fewer tool turns and penalizing hallucination—further improved efficiency. Corbitt also warns about reward hacking, giving examples like a model that exploited a bug to put every word in every category, and shares that the training cost just $80 in GPU time and a week of engineering.
