Ara Khan

Evals Are Broken, Use Them Anyway — Ara Khan, Cline

Jun 6, 2026 · 19:04

Ara Khan from Cline argues that evals are broken—people either treat benchmark numbers as gospel or dismiss them for vibes—but that the truth lies in between, and they should still be used. He presents three heuristics: don't believe model vendor eval numbers, stay current but not an earliest adopter, and look for new precise evals like Terminal Bench. Khan details Cline's journey from ignoring evals to building their own, then adopting Terminal Bench (89 real-world coding tasks). He explains the process: get a score (Cline started at 43%), portfolio allocate failures by sending another agent through traces to identify small levers, then hill climb by fixing zone1 bugs, zone2 nuanced prompt engineering (e.g., Anthropic-specific techniques that don't transfer to Codex or Gemini), and avoid zone3 overfitting. The episode offers a practical framework for using evals to improve agent performance while staying grounded in real-world usefulness.

Don't Build Slop (4 Levels of AI Agent Maturity) - Ara Khan, Cline

May 19, 2026 · 18:52

Ara Khan of Cline presents four levels of AI agent maturity, arguing that most builders suffer from mass psychosis and should focus on thoughtful architecture. Level one is using existing frameworks for rapid prototyping. Level two introduces five rules: treat every agent as a state machine, keep system prompts minimal (e.g., GPT-5.3's prompt is one-third the size of GPT-5's to avoid sensory overload), integrate with a CLI for pseudo-RL pipelines, avoid building slop by designing architecture manually, and master frontier model APIs to leverage reasoning traces correctly. Level three advocates Kanban boards as the ideal UX form factor for managing parallel inference-bound agents, a prediction he made on March 26 that Claude Code shipped ten hours before this talk. Level four involves shipping agents to the cloud for scalability, enabling long-running tasks and parallel execution without local dependencies.