Scaling Agents for Gen AI Products - Anju Kambadur, Bloomberg Head of AI Engineering
Apr 1, 2025 · 19:38
Bloomberg's Head of AI Engineering, Anju Kambadur, details the company's shift from building proprietary LLMs to leveraging open-source models for agentic products, emphasizing that scaling agents requires accepting inherent fragility and building resilient guardrails rather than seeking perfect upstream systems. Drawing on Bloomberg's daily scale—400 billion structured data ticks, over a billion unstructured messages, and 40+ years of history—Anju explains how their 400-person AI team organized into 50 teams across London, New York, Princeton, and Toronto. He argues that agents must be semi-autonomous with non-optional guardrails (e.g., preventing financial advice, ensuring factuality) because compounding errors from evolving APIs and LLMs demand self-contained safety checks. The talk uses the example of a research analyst agent that factors query understanding, answer generation, and guardrails into separate components, reflecting an org structure that collapses vertically for fast iteration on single agents then later introduces horizontal teams (like guardrails) for optimization and cost reduction. Anju stresses that readiness to build horizontal teams comes after multiple…
