LLM Observability, Evaluation, Experimentation Platform — Dat Ngo, Arize
Jun 7, 2026 · 16:32
Dat Ngo from Arize AI explains that observability and evaluation are essential for debugging nondeterministic LLM agents, where code no longer audits behavior—telemetry does. He outlines five flavors of eval signal—LLM as judge, human feedback, golden datasets, deterministic checks, and business metrics—and describes running them at different scopes: single span, multispan, trajectory, and session. Arize’s open-source Phoenix runs as a single container without Kubernetes, while the enterprise Arize AX adds Alex, an AI that scans traces for latency and errors and creates evals automatically. The goal is to automate the entire observability loop, letting developers focus on improvement rather than manual monitoring.
