Why should anyone care about Evals? — Manu Goyal, Braintrust
Jun 27, 2025 · 5:41
Manu Goyal, founding engineer at Braintrust, argues that evals are not just unit tests for AI but a critical tool for building a laboratory that enables 90% of product iteration before shipping to production. Drawing from his personal journey from a disappointed Nintendo-playing child to a self-driving car engineer at Nuro, he explains how evals provide the necessary signal to contextualize model improvements and reduce risk. He describes Braintrust's platform as a dev environment that combines tweaking prompts, logging data, and observability to create a data flywheel for AI development. Citing tech luminaries like Kevin Weil, Gary Tan, Mike Krieger, and Greg Brockman, he asserts that evals are the key to industry transformation and successful AI deployment. The talk concludes with an invitation to the Evals Track at the conference, emphasizing his message of 'evals, evals, evals'.
