Droid

Evals Are Broken, Use Them Anyway — Ara Khan, Cline

Jun 6, 2026 · 19:04

Ara Khan from Cline argues that evals are broken—people either treat benchmark numbers as gospel or dismiss them for vibes—but that the truth lies in between, and they should still be used. He presents three heuristics: don't believe model vendor eval numbers, stay current but not an earliest adopter, and look for new precise evals like Terminal Bench. Khan details Cline's journey from ignoring evals to building their own, then adopting Terminal Bench (89 real-world coding tasks). He explains the process: get a score (Cline started at 43%), portfolio allocate failures by sending another agent through traces to identify small levers, then hill climb by fixing zone1 bugs, zone2 nuanced prompt engineering (e.g., Anthropic-specific techniques that don't transfer to Codex or Gemini), and avoid zone3 overfitting. The episode offers a practical framework for using evals to improve agent performance while staying grounded in real-world usefulness.

How Claude Code Works - Jared Zoneraich, PromptLayer

Dec 26, 2025 · 1:05:43

Jared Zoneraich from PromptLayer dissects how Claude Code works, arguing its success stems from a radical simplification: a single-threaded master loop with tool calls, relying on capable models rather than complex DAG-based architectures. He breaks down the core internal tools—Bash, FileEdit, Grep—and the to-do planning mechanism that is purely prompt-based and not deterministically enforced. Zoneraich contrasts Claude Code with other coding agents: Codex (Rust core, kernel-level sandboxing), Amp (handoff context management, model-agnostic), and Cursor Composer (UI-first, distilled model). He emphasizes that Bash is the most important tool, acting as a universal adapter, and that context management via sub-agents and summarization is critical. Key takeaways include trusting the model, simple design, rigorous tool testing, and that different agent philosophies (not one winner) will serve different use cases.

Making Codebases Agent Ready – Eno Reyes, Factory AI

Dec 22, 2025 · 15:33

Eno Reyes of Factory AI argues that the primary bottleneck for AI agents in software engineering is not model capability but the 'agent readiness' of the codebase, measured by eight categories of automated verification including linters, tests, and documentation. He describes a shift from specification-based to verification-based development (Software 2.0), where agents succeed only when environments have fast feedback loops and explicit constraints. Reyes presents a flywheel: better validation enables better agents, which in turn improve validation, and claims this investment can yield 5x, 6x, 7x engineering velocity. He advises organizations to measure their readiness across pillars like style validation, build systems, and observability, and to remediate gaps using agents themselves. The talk emphasizes that organizational practices, not tool selection, determine agent success.

Dispatch from the Future: building an AI-native Company – Dan Shipper, Every, AI & I

Dec 18, 2025 · 17:58

In this episode, Dan Shipper of Every argues that there is a 10x difference between an organization where 90% of engineers use AI and one where 100% do. At Every, 99% of code is written by AI agents, enabling 15 people to run four software products with 7,000 paying subscribers, each built by a single developer. Shipper introduces 'Compounding Engineering', where each feature makes the next easier to build through a loop of plan, delegate, assess, and codify. He describes how this approach allows managers to commit code, enables tacit code sharing across products, and lets new hires be productive on day one. The episode details how Every's shift to agentic workflows (Claude Code, Codex) and a 'demo culture' over 'memo culture' has transformed their engineering velocity and collaboration.