InstructGPT

Real AI Agents Need Planning, Not Just Prompting - Yuval Belfer

Jun 3, 2025 · 7:58

Yuval Belfer of AI21 Labs argues that LLMs alone still fail at instruction following, as shown by GPT-4.1's struggles in 2025, and that true AI agents require dynamic planning, not just prompting. He critiques ReAct for lacking look-ahead, contrasting it with AI21 Maestro's planner and smart execution engine, which uses best-of-n sampling, candidate pruning, and replanning. On AIfeval, Maestro pushes GPT-4.0, Claude Sonnet 3.5, and R3 Mini to near-perfect scores; on internal requirement satisfaction benchmarks, it improves over single LLM calls despite higher runtime and cost. Belfer advises starting simple (SLMs, ReAct) and escalating to planning only for complex tasks, inviting listeners to join the Maestro waitlist.

Building Blocks for LLM Systems & Products: Eugene Yan

Nov 2, 2023 · 17:24

Eugene Yan presents practical patterns for building LLM systems and products, emphasizing evals as the foundation. He argues that automated evals—starting with as few as 40 domain-specific questions—enable faster iteration and safer deployment, while academic benchmarks like MMLU may not fit real tasks. For retrieval-augmented generation, he warns that LLMs perform worse when the answer is in the middle of retrieved documents, and even with perfect retrieval, accuracy tops out at 75%. Guardrails against hallucination can be built via natural language inference at the sentence level or by sampling multiple summaries to check consistency. Finally, he highlights UX design—like Copilot’s accept/reject or Midjourney’s upscale/vary—as a way to collect implicit feedback that builds a data flywheel for evals and fine-tuning.