Agents are Robots Too: What Self-Driving Taught Me About Building Agents — Jesse Hu, Abundant
Nov 24, 2025 · 17:37
Jesse Hu, former Waymo engineer and now founder of Abundant, argues that building reliable AI agents mirrors the challenges of self-driving cars, with the same 1% vs 99% problem where the model accounts for only 1% of the work. He draws parallels between robotics and agents across closed-loop feedback, statefulness, action spaces, and simulation, noting that just as self-driving pioneers learned perception was easy but planning was hard, agent builders must move from predictive models to action models. Hu explains how open-loop designs (e.g., waiting for full tool responses) limit agent reactivity, and advocates for richer input/output mechanisms like character-level terminal streams. He emphasizes the importance of an offline stack—simulation, evaluation, and data feedback loops—for iterative hill climbing, and warns that actions have consequences: out-of-distribution failures cascade. Finally, he recommends reading up on MDPs, DAgger, and offline RL from robotics literature to accelerate agent development.
