Patrick Dougherty: How to Build AI Agents that Actually Work
Feb 22, 2025 · 17:44
Patrick Dougherty, co-founder and CTO of Rosco (acquired by Klarity in 2024), argues that building effective AI agents requires prioritizing reasoning over inherent model knowledge, iterating on the Agent-Computer Interface (ACI) rather than fine-tuning or relying on frameworks. He defines an agent as a system that takes directions, calls tools, and autonomously reasons—not a prompt chain. His team abandoned RAG-style knowledge injection for discrete tool calls (e.g., SEARCH TABLES, GET TABLE DETAIL), finding that GPT-4o often hallucinates impossible SQL queries while o1 correctly identifies missing data. Switching tool-response formats—Markdown to JSON for GPT-4o, JSON to XML for Claude—dramatically improved accuracy. For production, he warns against over-dependence on abstractions like LangGraph due to security needs (e.g., OAuth credential cascading). In multi-agent systems, a manager agent should delegate to 5–8 specialized worker agents, incentivized by the overall goal, to avoid loops. The true moat, he concludes, is the ecosystem—UX, connections, and security—not system prompts.
