A product discussed on AI Engineer.

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind
Jul 14, 2026 · 21:46
Philipp Schmid, Staff Engineer at Google DeepMind, argues that shipping agent skills without evals is like merging code without tests, leading to unreliable performance. He explains that most of the 50,000+ skills on Skill Bench lack evals and are often AI-generated, which can negatively impact performance. The talk distinguishes between capability skills (teaching models new tasks) and preference skills (encoding team-specific workflows), with evals enabling retirement of skills as models improve. Schmid shares practical advice: keep skill descriptions concise (under 500 lines), include negative tests, and write directives over passive information. He details a case study creating 117 test cases for the Gemini interactions API, achieving 90% valid code generation using regex-based asserts and LLM-as-judge. The key takeaway: always run ablation tests with and without skills to measure their true impact, and start small with as few as 10-20 eval samples.

Should AI Engineers Still Read Code in 2026? The Z/L Continuum — Alex Volkov, ThursdAI
Jul 10, 2026 · 21:35
Alex Volkov examines the debate sparked by Ryan Lopopolo's claim that 'code is free' and Mario Zechner's counter that engineers must 'read every fucking line' of critical code, arguing that the Z/L Continuum is about task-level proof rather than personality. Citing a Ferrous AI survey showing an 861% increase in code deletion per PR and a 242% rise in incidents, he notes Anthropic's recursive self-improvement essay admitting human code review is a new bottleneck. Volkov's routing table prescribes reading every line for authentication, money movement, and irreversible data, while letting agents handle less critical changes. He also introduces loops—cron-like agent systems that self-verify—as the next frontier, quoting Adi Osmani that automated loops don't remove judgment. Volkov concludes that capability drift moves where proof belongs, but every system still requires human judgment.

Amp Code: Next Generation AI Coding – Beyang Liu, Amp Code
Dec 22, 2025 · 18:31
Beyang Liu, co-founder and CTO of Amp Code, presents Amp as an opinionated frontier coding agent that rejects MCP in favor of custom tools to avoid context confusion and optimize feedback loops. Amp's architecture relies on specialized subagents—the finder for code search, the oracle for deep reasoning, the librarian for external context, and the kraken for large-scale refactors—rather than a model selector. The agent offers two top-level modes: a slower 'smart' agent for complex tasks and a 'rush' agent for quick, in-loop edits, recently updated to leverage Gemini 3. To address inference costs, Amp ships subtle ads in the terminal to sponsor free usage for students and side projects. Liu also highlights a shared-threads feature for team learning and a community called the Weird Builder Cohort, emphasizing a culture of experimentation over hype.

Defying Gravity - Kevin Hou, Google DeepMind
Dec 2, 2025 · 25:10
Kevin Hou from Google DeepMind introduces Antigravity, a new agent-first IDE that combines three surfaces: an AI editor, an agent-controlled Chrome browser, and an Agent Manager. The central claim is that agents should live outside the IDE, enabling longer-running tasks and multimodal interactions. Key features include Artifacts—dynamic representations like plans, screen recordings, and diagrams—that replace raw chain-of-thought with visual summaries. The browser enables context retrieval and verification via screen recordings, while image generation allows iterative design through comments. Hou explains the research-product flywheel: DeepMind researchers use Antigravity internally to identify model gaps, improving capabilities like computer use and instruction following, which then ship in the product. He also details four categories of model improvements: intelligence, tools, long-running tasks, and multimodal, all powered by Gemini 3 Pro and related models.
Powered by PodHood