Air Canada

Taming Rogue AI Agents with Observability-Driven Evaluation — Jim Bennett, Galileo

Jun 27, 2025 · 16:14

Jim Bennett, Principal Developer Advocate at Galileo, argues that AI agents must be tamed using observability-driven evaluation, where LLMs evaluate other LLMs to detect failures like hallucinations and tool misuse. He cites real-world examples: the Chicago Sun-Times publishing a hallucinated summer reading list, and a lawyer citing false AI-generated case law. Bennett demonstrates a fintech chatbot that fails to answer 'what is my account balance' directly, requiring three turns; metrics like 'action completion' and 'action advancement' reveal the agent advances but does not complete. He stresses granular evaluation at every step—LLM calls, tool use, RAG retrieval—and using a better, custom-trained LLM as the evaluator. Human feedback is essential to correct mis-scored metrics and continuously retrain. Bennett urges adding evaluations from day one, even before production, and maintaining them in CI/CD and production with alerting for rogue behavior.

Judging LLMs: Alex Volkov

Sep 9, 2024 · 18:39

Alex Volkov, as an LLM judge from 2034, humorously judges AI engineers on their development practices, emphasizing the importance of tracing, iterative prompt engineering, and robust evaluation pipelines. He finds Daniel guilty of deploying without logging, Sasha guilty of premature fine-tuning without prompt iteration, and Morgan guilty of ignoring AI news—commuted to attending ThursdAI. Francisco’s overreliance on programmatic evals leads to a legal loss, while Maxime is 'awesome' for using Weights & Biases Weave. Alex concludes with a primer on evaluation methods: programmatic, human-in-the-loop, and LLM-as-judge, stressing the need to validate validators and create custom criteria. He promotes Weave for tracing and evals, and ThursdAI for staying updated.