AWS

Voice Agents That Handle Interrupts - Chintan Agrawal and Daniel Wirjo, AWS

Jul 20, 2026 · 32:57

AWS Solutions Architects Chintan Agrawal and Daniel Wirjo argue that the hardest problem in production voice agents is audio engineering—not AI—specifically turn-taking, the decision of when an agent should stop speaking or start responding. They present three levels of turn detection: Level 1 uses Silero VAD with a silence timeout (e.g., 300ms default), Level 2 delegates to STT providers like Cartesia or Deepgram for built-in endpointing (p50 ~250-300ms), and Level 3 combines Silero VAD with SmartTurn, an open-source 8MB model achieving 58.9% recall and 68.4% precision while falling back to VAD on low confidence. They show how interruption handling (barge-in) flushes TTS/LLM in ~15ms and distinguish real interruptions from backchannel acknowledgments. Latency budgets are tight: 40ms mic encoding, 52ms network/jitter, 300ms STT+endpointing, 500-650ms LLM time-to-first-byte (dominant bottleneck), and 120-190ms TTS playback, totaling 800-1300ms in standard cloud setups. Co-locating models in one GPU cluster can achieve ~500ms voice-to-voice. For LLMs, Nemotron 3 Ultra and GPT-4.1 achieve ~530ms p50 but GPT-4.1 spikes to 1.7s p95, and multi-turn drift (>15 turns) can break prompt…

Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS

Jul 11, 2026 · 55:19

Elizabeth Fuentes (AWS) presents five code-based techniques to stop AI agent hallucinations, each with measurable before/after metrics. Semantic Tool Selection filters 29 tools to the 3 most relevant per query, cutting token usage from thousands to under 300 per call. Graph-RAG replaces vector similarity with structured graph queries (using Neo4j), enabling precise aggregation and multi-hop reasoning that vanilla RAG fabricates. Multi-Agent Validation uses an Executor-Validator-Critic swarm to catch fabrications, achieving a 92% detection rate. Neurosymbolic Guardrails enforce business rules in Python hooks that the agent cannot skip, achieving zero rule violations. Agent Steering guides agents to self-correct when soft rules fire, completing tasks without hard failures—demonstrated by booking 50 guests by intelligently splitting into two rooms.

Everything we knew about software has changed — Theo Browne, @t3dotgg ​

Jul 8, 2026 · 16:02

Theo Browne argues that AI model evolution—from Sonnet 3.5’s tool-calling to Opus 4.5's long-running tasks and Mythos's orchestration—requires engineers to think bigger and wider. He compares current developer habits to skeuomorphism in iOS 7, urging rejection of legacy constraints like Git's inability to commit environment files and terminal-centric workflows. Browne introduces a shifted tier system: what was a startup is now a side project; a Markdown file running on a cron job can replace a company's product. His own PR triage service became a Markdown file updated daily via cron. He advocates building breadth over depth—architecting products so users can extend features, enabling small teams to compete with AWS or Salesforce. 'If your idea doesn't feel stupid, it's because your idea is not big enough,' he concludes.

Your Agent Is Wasting Tokens and You Don't Know It - Erik Hanchett, AWS

Jun 28, 2026 · 5:55

Erik Hanchett, a Senior Developer Advocate at AWS, presents five specific techniques to reduce token costs in agent workflows without changing prompts or models. He demonstrates caching system prompts with 'cachePrompt' to avoid resending full context, and routing tasks by difficulty using cheaper models like Claude Haiku for simple requests. He advises offloading large tool results by storing them externally and sending only summaries, capping tool loops with a max iterations setting to prevent infinite loops, and trimming conversation history using a sliding window that keeps only the last 10 messages while optionally summarizing older context. These methods directly address common cost pitfalls, such as using expensive models for trivial tasks and letting context windows grow unchecked.

Using Spec-Driven Development for Production Workflows - Erik Hanchett, AWS

Jun 28, 2026 · 17:47

Erik Hanchett (AWS) argues that spec-driven development—writing markdown specification files before any code—produces higher quality code by guiding AI coding assistants, which he compares to 'AI interns' that need explicit direction. He introduces Kiro, AWS’s new AI IDE and CLI, which offers a spec mode that generates requirements documents, design documents with mermaid diagrams, property-based tests using Fastcheck, and a task list. In a demo building a movie website, Kiro created an MVP-focused implementation verified by property tests. Hanchett also highlights integrating external data via the Model Context Protocol (MCP) and stresses the importance of human review. He advises balancing context in steering docs and using on-demand skills to control the workflow.

Fast Models Need Slow Developers — Sarah Chieng, Cerebras

May 22, 2026 · 18:02

Sarah Chieng from Cerebras argues that the 20x speed increase of models like Codex Spark (1,200 tokens/sec vs. 40-60 for Sonnet/Opus) forces developers to rethink workflows, or risk generating technical debt at unprecedented scale. She presents a practical playbook: validation and linting become free at every step, so must run continuously; developers can generate 75 component variations across five sub-agents and cherry-pick the best; and with context filling in 30 seconds instead of ten minutes, a four-file external memory system (agents, plan, progress, verify) maintains continuity between sessions. Chieng emphasizes real-time collaboration with the model rather than spawning agents and walking away, and advocates using a slower planner model with a fast executor to orchestrate agents effectively.

AI Didn’t Kill the Web, It Moved in! — Olivier Leplus (AWS) & Yohan Lasorsa (Microsoft)

Apr 10, 2026 · 52:55

Yohan Lasorsa (Microsoft) and Olivier Leplus (AWS) argue that AI isn't replacing the web—it's becoming its native layer, embedded in every stage of development. They demo coding agents that use lightweight 'skills' to automate workflows like fetching GitHub issues and testing via Playwright, then show Chrome's MCP server letting agents debug performance across different network conditions. New on-device Web AI APIs (summarizer, proofreader, prompt API) run models locally in the browser, enabling features like auto-writing reviews from an uploaded product image without cloud calls. Finally, they introduce LLMs.txt for guiding AI agents to relevant documentation and Web MCP—a proposal to register native tools on websites (e.g., 'add to cart') so agentic browsers can execute actions without mimicking human clicks. The episode delivers concrete implementations, from coding to deploying AI-ready web apps.

From Chaos to Choreography: Multi-Agent Orchestration Patterns That Actually Work — Sandipan Bhaumik

Apr 8, 2026 · 26:29

Sandipan Bhaumik, Data & AI Tech Lead at Databricks, argues that scaling AI agents from one to five transforms an AI problem into a distributed systems problem, exposing teams to race conditions, stale reads, and cascading failures. He recounts a production deployment where a credit decisioning system incorrectly approved 20% of applicants because a caching layer wasn't invalidated between agents. Bhaumik presents two coordination patterns: choreography (event-driven, decentralized) for simple workflows with high autonomy, and orchestration (central coordinator) for complex workflows needing rollback and observability. He advocates for immutable state snapshots (append-only versions) and data contracts (schema validation at handoffs) to eliminate race conditions, and demonstrates failure recovery using circuit breakers and compensation (saga) patterns. The episode delivers a production-grade architecture on Databricks using LangGraph, Unity Catalog, Delta Lake, and MLflow for tracing, with code examples for state handoff and circuit breaker logic.

Building durable Agents with Workflow DevKit & AI SDK - Peter Wielander, Vercel

Jan 6, 2026 · 1:09:49

Peter Wielander from Vercel introduces the Workflow DevKit, an open-source library that adds durability, observability, and resumability to AI agents by wrapping them in a workflow pattern. He demonstrates converting a coding agent into a workflow-supported agent, showing how steps like LLM calls and tool executions become isolated, retryable units. The toolkit enables long-running agents that can sleep for days, resume streams after disconnection, and integrate human-in-the-loop via webhooks. Deployable on Vercel or any cloud, it provides built-in observability through a local UI. The Workflow DevKit is currently in beta with general availability targeted for January.

VoiceVision RAG - Integrating Visual Document Intelligence with Voice Response — Suman Debnath, AWS

Dec 6, 2025 · 1:23:52

Suman Debnath, a Principal ML Advocate at AWS, demonstrates how Colpali—a vision-based retrieval model that treats each document page as an image and generates multi-vector embeddings via patch-based late interaction—can be combined with voice synthesis for a more intuitive RAG system. He explains that Colpali bypasses traditional OCR and preprocessing by directly embedding document images, then scoring query-page relevance through dot-product similarity across patches. The workshop shows how to embed pages, store them in Qdrant with multivector support, and retrieve top pages. Debnath then wraps this retrieval pipeline using the Strands Agent framework, adding a speak tool to output answers in natural voice. A live demo answers a textbook question about trophic levels, first using Bedrock to generate text and then speaking the answer in a female voice, all without a predefined system prompt. The talk positions Colpali as a complementary technique for complex visual documents like IKEA instructions or scanned forms, not a full replacement for traditional RAG.

Full Workshop: Realtime Voice AI — Mark Backman, Daily

Aug 3, 2025 · 1:09:41

In this hands-on workshop, Mark Backman and Aleś from Daily demonstrate how to build real-time voice AI agents using Pipecat, their open-source Python framework, and Google's Gemini Multimodal Live API. They explain that Pipecat's modular pipeline orchestrates audio transport, speech-to-text, LLM, and text-to-speech services, allowing developers to plug and play vendors like Deepgram, OpenAI, or Cartesia. The session highlights how speech-to-speech models like Gemini Live simplify architecture by handling transcription, LLM logic, and voice generation in one step, reducing latency to under 800 milliseconds. Key topics include the critical role of voice activity detection (VAD) for natural turn-taking, strategies for managing context windows to maintain accuracy, and the trade-offs between speed and reliability when using tool calls in real-time. Despite conference Wi-Fi issues, the team live-codes a functional bot and demonstrates interruption handling and the Word Wrangler game, showcasing Pipecat's production readiness with hundreds of thousands of daily calls.

How we hacked YC Spring 2025 batch’s AI agents — Rene Brandel, Casco

Jul 30, 2025 · 17:33

Rene Brandel, CEO of Casco, explains how his team hacked seven of sixteen YC Spring 2025 batch AI agents within 30 minutes each, revealing three critical security flaws: cross-user data access via IDOR, arbitrary code execution through code tools, and server-side request forgery (SSRF) from tool endpoints. They extracted personal data by exploiting missing authorization checks, overwrote security controls by writing malicious files into code sandboxes, and stole Git credentials by manipulating a database-schema tool. Brandel emphasizes that agent security extends beyond LLM prompt injection, advises treating agents as users with proper authentication and authorization, and warns against rolling custom code sandboxes, recommending out-of-the-box solutions like E2B. The talk concludes with a Q&A on extracting system prompts and the dangers of local coding agents.

How agents will unlock the $500B promise of AI - Donald Hruska, Retool

Jul 23, 2025 · 16:22

Donald Hruska, engineering lead for Retool's Agents product, argues that AI agents will unlock the $500B promise of AI by moving enterprises beyond toy chatbots into production-grade systems. He explains that building a basic agent is easy (e.g., 100 lines of code using React) but getting it into production requires solving security, cost overruns, observability, and compliance. Hruska outlines four options—build from scratch, use a framework like LangGraph, a managed platform like Retool Agents, or verticalized agents—and advises building for core differentiators and buying for commodity workflows. He cites Retool customers like ClickUp saving over $200,000 in vendor costs and Descript saving hundreds of hours weekly, while noting inference costs dropped 99.7% from 2022-2024. Retool charges $3 per hour for its cheapest agent and supports on-prem deployment.

How Intuit uses LLMs to explain taxes to millions of taxpayers - Jaspreet Singh, Intuit

Jul 23, 2025 · 18:59

Jaspreet Singh, Senior Staff Engineer at Intuit, explains how TurboTax uses Anthropic's Claude and OpenAI models to generate personalized tax explanations for 44 million customers. The system, built on Intuit's GenOS platform, combines prompt-engineered static explanations with dynamic question-answering using RAG and graphRAG for tax-specific queries. Singh details their fine-tuning of Claude Haiku on AWS Bedrock, which improved quality but proved too specialized. A key focus is evaluation: manual reviews by tax analysts followed by automated LLM-as-a-judge scoring for accuracy, relevancy, and coherence, with a golden dataset and safety guardrails to prevent hallucinated numbers. He highlights challenges like latency spikes on tax day (3–10 seconds), vendor lock-in through expensive contracts, and the difficulty of upgrading models even within the same vendor. Singh emphasizes that evaluations are essential for launching any GenAI feature in a regulated domain like tax.

2025 in LLMs so far, illustrated by Pelicans on Bicycles — Simon Willison

Jul 9, 2025 · 18:30

Simon Willison reviews the past six months of LLM releases — including AWS Nova, Llama 3.3 70B, DeepSeek R1, Mistral Small 3, Claude 3.7 Sonnet, GPT 4.5, Gemini 2.5 Pro, GPT-4o, Llama 4, GPT 4.1, O3/O4 Mini, and Claude 4 — using his 'pelican on bicycle' SVG benchmark to argue that local models have become good enough to run GPT-4 class models on a laptop and that combining tools with reasoning is the most powerful technique in AI engineering, while noting risks like prompt injection and the 'lethal trifecta'. He tracks 30 significant model releases, highlighting that Mistral Small 3 (24B) matches Llama 3 70B's performance, which itself matched the 405B model, enabling local inference. DeepSeek's R1 caused a $500B+ Nvidia stock drop on January 27. GPT 4.1 Nano is the cheapest model yet at a fraction of a cent per pelican. He also examines bugs: ChatGPT's sycophantic 'shit-on-a-stick' incident and Claude 4's tendency to snitch to authorities when given ethical instructions and email tools. Willison concludes that while the pace is accelerating, control over context and security remain critical.

Conquering Agent Chaos — Rick Blalock, Agentuity

Jul 1, 2025 · 14:40

Rick Blalock, founder of Agentuity, argues that deploying AI agents remains the number one headache for developers, citing common issues like serverless timeouts (agents running 15–30 minutes), statelessness, and networking complexities. He demonstrates Agentuity’s platform, which treats agents as first-class infrastructure citizens: users scaffold projects via a CLI (supporting Bun, Python/UV, Node.js), write a simple request handler, and deploy with automatic routing, tunneling, and a built-in AI gateway that tracks costs per run. The platform decouples inputs and outputs—agents can be triggered via email, SMS, webhooks, or cron—and provides human and agent-facing telemetry via OTel tracing. Blalock notes Agentuity already hosts 50–60 internal agents and is building infrastructure agents to replace tools like PagerDuty, with plans to add Slack/Discord integrations and service-level reasoning capabilities.

Serving Voice AI at Scale — Arjun Desai (Cartesia) & Rohit Talluri (AWS)

Jun 27, 2025 · 17:05

Arjun Desai of Cartesia AI and AWS's Rohit Talluri discuss scaling voice AI for enterprise, arguing that latency and controllability are critical, with Cartesia's state-space model Sonic 2 achieving 40ms model latency for real-time applications. Desai explains that traditional transformer models scale quadratically, while Cartesia's SSMs maintain O(1) generation, enabling 2.5x faster inference than their earlier models. He emphasizes that edge deployment is 5x faster than cloud round-trips, making local models essential for interactive use cases. On quality, Desai notes that voice AI must handle interruptions, accents, and background noise, and that Cartesia's voice marketplace amplifies human voice actors rather than replacing them. Looking to 2030, he predicts voice AI will become the default interface across healthcare, customer support, and gaming, with interactive models extending beyond audio to full world models.

Ship it! Building Production Ready Agents — Mike Chambers, AWS

Jun 27, 2025 · 19:37

Mike Chambers, a developer advocate at AWS, demonstrates how to take a simple local agent—built with a Llama 3.1 8B model and a dice-rolling tool—and ship it to production at cloud scale using Amazon Bedrock Agents. He breaks down the essential components of an agent: model, prompt, loop, history, and tools. Then he live-deploys the agent with Bedrock Agents, configuring instructions and an action group wired to an AWS Lambda function that handles the dice roll. The fully managed service automates scaling, infrastructure, and the agentic loop. Chambers also highlights free courses on DeepLearning.AI and invites attendees to discuss MCP servers and the new open-source SDK for model-first agents.

Building Agents with Amazon Nova Act and MCP - Du'An Lightfoot, Amazon (Full Workshop)

Jun 21, 2025 · 1:26:20

Du'An Lightfoot and Benjiro Byami present a workshop on building AI agents with Amazon Nova Act, Model Context Protocol (MCP), and Strands Agents, demonstrating autonomous web navigation and multi-step task execution. Lightfoot explains agentic AI as plan, action, and reasoning, with components like LLM, knowledge base, guardrails, and tools. Nova Act, a research preview, enables browser automation: one demo searches Amazon for a coffee maker and retrieves the product title using natural language commands instead of explicit HTML tags. Byami builds an MCP server for Nova Act, allowing clients like Claude Desktop to control the browser via spoken instructions, then integrates it with Strands Agents, an open-source framework that combines a prompt, LLM, and tools in minimal code. The workshop shows parallel execution (e.g., comparing three monitors) and multi-agent collaboration for tasks like generating a PowerPoint presentation for cloud migration. Limitations include inability to bypass CAPTCHAs and occasional loops in complex websites.

Supercharging developer workflow with Amazon Q Developer - Vikash Agrawal

Jun 10, 2025 · 13:22

Vikash Agrawal and Linda show how Amazon Q Developer can drive every stage of the software development lifecycle. They build a 2048 game with FastAPI and Poetry using Q's /dev command in the CLI, then add unit tests with /test, generate documentation with /doc, and integrate with GitHub for automated pull requests. After deploying to AWS, they use Q in CloudWatch to debug a runtime import error by analyzing logs and resource topology. The episode emphasizes that planning infrastructure and prompts upfront, along with security scans, are critical for production-ready AI-assisted development.

7 Habits of Highly Effective Generative AI Evaluations - Justin Muller

Jun 3, 2025 · 25:39

Principal Applied AI Architect Justin Muller argues that generative AI evaluations are the missing piece to scaling, offering seven habits from over 100 projects. He recounts a customer whose document processing workload had 22% accuracy with no evals; after building an evaluation framework, accuracy reached 92% and the system became the largest such workload on AWS in North America. The habits include fast 30-second eval cycles using AI-as-judge, quantifiable scores averaged across numerous test cases, explainable reasoning for both generation and scoring, segmented evaluation through prompt decomposition, diverse test sets covering all use cases, and traditional techniques for numeric outputs or cost/latency. He emphasizes that evaluations should primarily discover errors, not just measure quality, and that prompt decomposition into chained steps often boosts accuracy by removing dead space.

Anthropic in the Enterprise — Alexander Bricken & Joe Bayley

Apr 13, 2025 · 20:55

Alexander Bricken and Joe Bayley from Anthropic's Applied AI team argue that enterprise AI implementation often fails due to overengineering, poor data infrastructure, or lack of testing—but industry leaders achieve transformative results with Claude. They detail Anthropic's deployment models (API, cloud partnerships, enterprise solutions) and real-world case studies like Intercom's Fin agent, which solved 86% of support volume using Claude. Best practices include building evals early as intellectual property, identifying intelligence/cost/latency trade-offs based on use-case stakes, and avoiding premature fine-tuning by trying prompt caching, contextual retrieval, and agentic architectures first. They also highlight interpretability research and the Model Context Protocol for reliable AI deployments.

Claude plays Minecraft!

Feb 15, 2025 · 18:16

Derek from AWS demonstrates building Rocky, an autonomous Minecraft agent using Amazon Bedrock, Claude 3 Haiku, and serverless AWS services, proving agentic workflows enable AI to reason and act beyond chatbots. He explains the architecture: Minecraft server on ECS with Mineflare bot framework, agents for Bedrock with return of control, and prompt engineering for tasks like digging and building. In a live demo, Rocky jumps, finds players, digs a hole, and builds a double-decker couch from a chat command, inferring parameters from natural language. The code is open-source and built with CDK/CloudFormation, emphasizing that agentic AI can operate in complex 3D environments through tool orchestration and managed prompt design.

[Full Workshop] How to add secure code interpreting in your AI app: Vasek Mlejnsky

Feb 6, 2025 · 1:48:16

Vasek Mlejnsky, CEO of E2B, demonstrates how to add secure AI code execution to any app using E2B’s Code Interpreter SDK alongside Anthropic's Sonnet 3.5 and Vercel’s AI SDK. The workshop builds an open-source version of Claude’s artifacts UI, running Python code in isolated Firecracker VMs that start in ~900ms and support custom environments via Dockerfiles. Vasek explains how E2B’s Jupyter-server-based sandbox returns stdout, stderr, runtime errors, and rich outputs like PNG charts, which are streamed to the front end using Vercel’s stream data. He covers security architecture—full root access in a VM that self-destructs on escape—and plans for AWS/GCP self-hosting and snapshot-based agent branching. The episode also addresses customizing sandboxes with private packages, mounting cloud storage (S3, GCS), and interpreting table previews as HTML or images.

Moondream: how does a tiny vision model slap so hard? — Vikhyat Korrapati

Nov 14, 2024 · 19:26

Vikhyat Korrapati built Moondream, a tiny open-source vision language model under 2 billion parameters that matches LLaVA 1.5, a model four times its size, on VQA v2 and GQA benchmarks. He attributes this to focusing on image understanding over world knowledge and investing heavily in synthetic training data—a pipeline that generated 35 million images and used two orders of magnitude more compute on data than training. Key lessons: community engagement was critical, open-source builds trust, and safety guardrails should be application-layer, not baked in. He argues tiny models will dominate production due to cost and privacy advantages, and that prompting will replace custom model training for most vision tasks. Moondream raised a seed round from FullySysAscent and the GitHub Fund.

From Software Developer to AI Engineer: Antje Barth

Jul 24, 2024 · 19:48

Antje Barth, a Principal Developer Advocate for generative AI at AWS, outlines five practical steps from software developer to AI engineer: understanding foundation models, getting hands-on with AI developer tools like Amazon Q Developer, prototyping with models via Amazon Bedrock, integrating agents, and staying up to date with community events. She demonstrates Amazon Q Developer's ability to generate, explain, and transform code, reducing unvaluable tasks from 70% to focus on creative work. Barth introduces Amazon Bedrock's unified Converse API for standardized model invocation across providers like Anthropic's Claude 3.5 Sonnet, and shows how agents can be built to control a Minecraft bot through reasoning and tool use. The episode concludes with the announcement of the AI Engineering Hub at the AWS loft in San Francisco for community workshops and events.

Supabase Vector: The Postgres Vector database: Paul Copplestone

Nov 3, 2023 · 16:05

Paul Copplestone, CEO of Supabase, makes the case for pgvector as an embedded vector database within Postgres, arguing it offers production-grade performance and unique advantages for AI applications. He recounts how pgvector was contributed by a single developer, Andrew Kane, and how Supabase integrated it, leading to 12,000 new databases launched weekly with 10-15% using pgvector. Addressing benchmarks that claimed pgvector was 20x slower, Copplestone shows that adding HNSW indexing brought accuracy to 0.99, matching specialized vector databases, and a cost comparison vs. Pinecone ($410 vs. $480 for comparable queries/sec) demonstrates Postgres competitiveness. He demonstrates Postgres's extensibility with a cat image filtering example using partitions and triggers to separate good cats (similarity >0.8) from bad, all in 14 lines of SQL. Finally, he outlines future work on sharding with Citus to handle billions of vectors, and invites design partners for enterprise use cases.