11 Labs

Introducing WebMCP: Agents in the Browser — RL Nabors

May 23, 2026 · 23:08

Rachel Nabors, former web standards contributor and now Principal Developer Experience Engineer at Arise, argues that chat-only agent interfaces (the 'starfish' design) are the CLI of the future and demonstrates how to replace them with rich interactive surfaces using MCP Apps and WebMCP. She explains STDIO vs HTTP transports, shows how MCP Apps bundle HTML, CSS, and JavaScript into single files rendered in agent iframes, and introduces WebMCP, which adds tool attributes to existing page elements so browser agents can call functions without screenshot parsing or DOM traversal. Using her own web comic site as a case study, she builds a comic reader with full panel navigation, transcript mode, and speech synthesis via the Web Speech API, and advocates for better client support of MCP resources to avoid inefficient tool calls for context loading.

Why TTS Models Now Look Like LLMs — Samuel Humeau, Mistral

May 9, 2026 · 22:26

Mistral AI scientist Samuel Humeau explains why text-to-speech models now resemble LLMs, using an autoregressive transformer with neural audio codecs to compress 200 kbps audio into ~500 tokens per second. He demos Mistral's open-weight TTS model, cloning a voice from a few seconds of reference audio and generating speech with 17 ms first-token latency via frame-by-frame streaming. The talk covers the codec-backbone-decoder pipeline, the model's 4B-parameter backbone generating 37 tokens per 80 ms frame via a diffusion decoder, and a live voice agent answering conference schedule questions. Humeau notes the next latency win is handling streaming text input from an LLM, with no settled architecture yet, while keeping the encoder for voice cloning proprietary.

Voice AI: when is the "Her" moment? — Neil Zeghidour, CEO, Gradium AI

May 9, 2026 · 19:27

Neil Zeghidour, CEO of Gradium AI, argues that voice AI remains far from the 'Her' ideal because cascaded systems (speech-to-text, LLM, text-to-speech) suffer from high latency—tool calls alone add 500ms to 4 seconds—while human response time is ~200ms. Speech-to-speech models reduce latency but are half-duplex, meaning they cannot handle overlapping speech or backchanneling, unlike Moshi, Gradium's full-duplex model. However, Moshi lacked intelligence, tool calls, and paralinguistic understanding—the ability to infer tone, hesitation, or discomfort from voice, which is stripped away in text. Cost is another barrier: TTS bills burn through fundraising before user bases grow. Gradium's solution is Phonon, an on-device TTS model running on smartphone CPUs, offering privacy and eliminating API fees. The path forward requires combining full-duplex natural conversation with the reliability and smarts of cascaded systems.

Mastering AI Pricing — Mayank Pant, Stripe

May 1, 2026 · 24:19

Mayank Pant from Stripe explains that AI companies, growing 3x faster than traditional SaaS, face margin risk from power users and unpredictable compute costs, making hybrid pricing (base fee + usage fee) essential—56% of AI leaders now use it. He presents a five-step framework: define customer-perceived value (e.g., automation, augmentation, enhanced service, improved results), choose a charge metric (consumption, workflow, or outcome-based), adopt hybrid pricing with guardrails like usage caps and automated notifications, and iterate pricing frequently—84% agree fast adaptation is a competitive advantage. Pant illustrates with examples: Gamma charges per deck (not API calls), Intercom prices per resolved ticket. To keep customer-facing prices stable while changing features, he advises abstracting value with credits (e.g., 100 credits/month) that can be internally revalued. Stripe's billing infrastructure supports this iteration, with 78% of AI companies building on Stripe using its subscription, usage, and hybrid billing, plus Metronome for enterprise contracts.

Full Workshop: Realtime Voice AI — Mark Backman, Daily

Aug 3, 2025 · 1:09:41

In this hands-on workshop, Mark Backman and Aleś from Daily demonstrate how to build real-time voice AI agents using Pipecat, their open-source Python framework, and Google's Gemini Multimodal Live API. They explain that Pipecat's modular pipeline orchestrates audio transport, speech-to-text, LLM, and text-to-speech services, allowing developers to plug and play vendors like Deepgram, OpenAI, or Cartesia. The session highlights how speech-to-speech models like Gemini Live simplify architecture by handling transcription, LLM logic, and voice generation in one step, reducing latency to under 800 milliseconds. Key topics include the critical role of voice activity detection (VAD) for natural turn-taking, strategies for managing context windows to maintain accuracy, and the trade-offs between speed and reliability when using tool calls in real-time. Despite conference Wi-Fi issues, the team live-codes a functional bot and demonstrates interruption handling and the Word Wrangler game, showcasing Pipecat's production readiness with hundreds of thousands of daily calls.

AI Music Generation, From Prompt to Production: Phlo Young

Feb 11, 2025 · 54:34

Phlo Young, an AI engineer and musician, demystifies AI music generation by distinguishing text-to-music, audio-to-music, and style transfer, and demonstrates hands-on use of Suno and Udio to turn prompts into songs. He showcases examples like a Kanye West voice conversion, a Randy Travis AI comeback, and a BBL Drizzy meme song generated entirely from text. Phlo explains prompt engineering techniques, provides live access to paid accounts for attendees to generate unlimited music, and covers tools like Wave Tool for extracting stems. He also touches on ethical considerations and the RIAA lawsuits against Suno and Udio. The workshop equips beginners with practical skills to create and refine AI-generated music.

Giving a Voice to AI Agents: Scott Stephenson, CEO, Deepgram

Feb 10, 2025 · 13:08

Scott Stephenson, CEO of Deepgram, outlines the evolution of voice AI from slow, domain-specific systems to fast, open-ended conversational agents powered by LLMs, arguing that speed and accuracy are now solved and the key differentiator is contextual understanding. He explains that state-of-the-art speech-to-text and text-to-speech can achieve round-trip latencies under 500 milliseconds—matching human turn-taking—but notes that current systems fail to pass context between components, causing 10-20% of interactions to feel unnatural. Stephenson introduces Deepgram's 'contextual AI' vision, where models are prompted with the full conversation history, including tone, pace, and background audio, enabling the LLM to generate responses and instruct the TTS on delivery. He cautions against monolithic speech-to-speech models for enterprise use due to controllability and cost concerns, advocating for a modular stack that lets businesses optimize each component (e.g., small LLMs for simple tasks like password resets). Deepgram’s upcoming voice AI agent API integrates all components for low latency and offers $250 in free credits for experimentation.

Building and Scaling an AI Agent Swarm of low latency real time voice bots: Damien Murphy

Oct 8, 2024 · 1:07:23

Damien Murphy, Senior Applied Engineer at Deepgram, demonstrates building and scaling low-latency real-time voice bots using Deepgram's new voice agent API, which wraps speech-to-text, LLM, and text-to-speech into a single streaming endpoint. He shows a drive-thru ordering demo with function calling (add/remove items) using GPT-4o, achieving sub-second latency by co-locating components. Murphy explains scaling to millions of concurrent calls through regional Kubernetes clusters and multi-agent swarms (routing, booking, support agents) to reduce complexity and cost. He addresses endpointing challenges, VAD-based barge-in, and cost/quality trade-offs with hosted vs. self-hosted models, noting Deepgram offers 20x cheaper TTS than ElevenLabs and 50ms STT latency self-hosted. The talk emphasizes keeping agents simple, using smallest capable LLMs, and composability for reuse.