2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI
Aug 6, 2025 · 19:14
John Dickerson, CEO of Mozilla AI and former co-founder of Arthur AI, argues that 2025 is finally the year for AI evaluation (evals) due to three converging forces: ChatGPT making AI tangible to the C-suite, enterprise budget freezes that funnelled funding into GenAI pet projects, and the rise of agentic systems acting autonomously. He traces how ML monitoring existed pre-2022 but lacked C-suite attention, with Jamie Dimon’s JPMC spending only $100 million on AI from 2017 to 2021. The ChatGPT launch on November 30, 2022, unlocked CEO discretionary budgets for GenAI, leading to 2023 science projects, 2024 production deployments, and 2025 scaling. Now, as agents reason and act, connecting evals to downstream business KPIs—risk mitigation, revenue gains—is a first-class discussion for CEOs, CFOs, CISOs, and CTOs. Dickerson notes that evaluation companies have shifted to multi-agent system monitoring, and while LLM-as-judge is popular, biases require human validation, with companies like Mercor hiring experts at $50–200/hour to lockstep multi-agent outputs.
