Ornella Bahidika

Don't Let the LLM Drive - Ornella Bahidika & Joel Allou, Microsoft

Jul 20, 2026 · 6:08

Ornella Bahidika and Joel Allou of Microsoft present their voice tutor Ace, arguing that the LLM should never control the flow in multi-step agents. They built Ace with a state machine that confines the model to narrow contracts per step, letting the harness validate outputs and decide next actions. This design allows them to use a cheaper, faster model (Haiku 4.5) instead of a heavy reasoning model (Opus 4.7 Cloud). They identify three decisions the LLM must never own: when the lesson ends, whether the student answered correctly, and what comes next. By engineering these checks outside the model, Ace achieves reliability in production, avoiding loops and early terminations that plague prompted-only approaches. The pattern applies to any flow agent, including coding agents, runbooks, and onboarding flows.

Your Voice Agent Doesn't Need a Frontier Model - Joel Allou & Ornella Bahidika, Microsoft

Jul 20, 2026 · 5:45

Joel Allou and Ornella Bahidika from Microsoft present ACE, an AI voice tutor that deliberately uses a small model instead of a frontier model, arguing that latency—not intelligence—is the critical constraint in voice applications. They demonstrate that a frontier model's reasoning time of multiple seconds breaks conversational flow, while their system keeps model latency under 950 milliseconds by extracting all logic and planning into a deterministic state machine. This scaffolding handles lesson progression, student mastery tracking, and response generation, leaving the small model (Haiku 4.5) with only the task of speaking. The result: a 900-millisecond response time that feels instantaneous. They acknowledge that scaffolding requires strict rules to prevent drift, but call it a one-time code investment that unlocks cost-effective, real-time performance. The episode’s core lesson: pick the fastest model your latency budget allows, then invest in external scaffolding to make it smart.