Waymo

Reachy Mini: the $300 open source robot you can actually hack — Andres Marafioti, Hugging Face

May 29, 2026 · 21:16

Andres Marafioti from Hugging Face presents Reachy Mini, a $300 open source robot shipped to 7,500 people unassembled, and explains that its most popular app is voice conversation. To make that app responsive, his team optimized Qwen3-TTS from a below-real-time 0.8x factor to 5.8x real time by adding streaming, switching to a static KV cache, and enabling CUDA graph captures, cutting time to first audio to under 200 milliseconds. The full voice pipeline runs Parakeet transcription every 150 milliseconds with partial results, feeds Qwen 3.5 27B for the LLM, and uses the optimized TTS, while infrastructure round trips are handled by a load balancer that separates LLM endpoints from conversation nodes. Marafioti argues that expensive humanoid robots limit creativity and access, whereas Reachy Mini's hackable, repairable design invites hackers, students, and dreamers to build new interactions. He demonstrates the robot's ability to take photos, show emotions, and even be pet, and emphasizes that all models and software are open source.

Shipping AI That Works: An Evaluation Framework for PMs – Aman Khan, Arize

Dec 26, 2025 · 1:26:16

Aman Khan, AI PM at Arize, presents a framework for product managers to evaluate LLM-powered products beyond gut-feel 'vibe checks.' He demonstrates building an AI trip planner with multi-agent LangGraph, then using Arize's tracing and prompt playground to iterate on prompts. Khan shows how to create datasets from production traces, run A/B experiments on prompts, and use LLM-as-a-judge evals for friendliness and discount offers, comparing against human labels to refine evaluators. He argues evals are the new requirements docs, enabling PMs to own the product experience by writing acceptance criteria as eval datasets. The talk covers building eval teams, handling variance with temperature settings, and continuously improving golden datasets with hard examples, citing real-world analogies from self-driving cars at Cruise.

Agents are Robots Too: What Self-Driving Taught Me About Building Agents — Jesse Hu, Abundant

Nov 24, 2025 · 17:37

Jesse Hu, former Waymo engineer and now founder of Abundant, argues that building reliable AI agents mirrors the challenges of self-driving cars, with the same 1% vs 99% problem where the model accounts for only 1% of the work. He draws parallels between robotics and agents across closed-loop feedback, statefulness, action spaces, and simulation, noting that just as self-driving pioneers learned perception was easy but planning was hard, agent builders must move from predictive models to action models. Hu explains how open-loop designs (e.g., waiting for full tool responses) limit agent reactivity, and advocates for richer input/output mechanisms like character-level terminal streams. He emphasizes the importance of an offline stack—simulation, evaluation, and data feedback loops—for iterative hill climbing, and warns that actions have consequences: out-of-distribution failures cascade. Finally, he recommends reading up on MDPs, DAgger, and offline RL from robotics literature to accelerate agent development.

From Self-driving to Autonomous Voice Agents — Brooke Hopkins, Coval

Jul 31, 2025 · 19:32

Brooke Hopkins, founder of Coval and former Waymo eval lead, argues that reliability lessons from self-driving cars directly apply to building trustworthy voice agents. She explains how Waymo moved from brittle scenario-specific tests to large-scale probabilistic simulations that measure success rates across thousands of runs—a framework now used at Coval to evaluate conversational AI at scale. Key practices include denoising: resimulating a failed scenario 100 times to assess true failure probability, and calibrating LLM-as-judge metrics with human feedback until automated evals align. Hopkins outlines a hierarchical eval strategy: start with text-based task evals for speed, then graduate to voice-to-voice tests for latency and interruptions, and finally hyper-realistic simulations for production edge cases. She predicts that within three years every enterprise will need a voice experience comparable to a mobile app, making integrated evaluation infrastructure a competitive necessity.

Waymo's EMMA: Teaching Cars to Think - Jyh Jing Hwang, Waymo

Jul 26, 2025 · 17:28

Jyh-Jing Hwang of Waymo presents EMMA, an end-to-end multimodal model built on Gemini that directly processes camera inputs and route text into driving waypoints, achieving state-of-the-art open-loop planning on the NuScenes benchmark without LiDAR or HD maps. Adding chain-of-thought reasoning—identifying critical objects and meta-decisions—further surpasses specialized models like MotionLM on Waymo's 100K dataset. Co-training on tasks such as 3D detection and road graph estimation maintains competitive performance across domains. For evaluation, Waymo leverages generative video models (e.g., Veo2) to simulate diverse conditions like rain and night, testing EMMA's robustness. The work demonstrates how Gemini's generalization can help scale Waymo's autonomous driving to new cities by handling long-tail scenarios.

Lessons From A Year Building With LLMs

Jul 19, 2024 · 35:21

The six authors of the O'Reilly article "Lessons From A Year Building With LLMs" — Bryan Bischof, Jason Liu, Hamel Husain, Eugene Yan, Shreya Shankar, and Swix — argue that the model itself is not a moat and that success comes from continuous improvement centered on evals and data. They stress that AI engineers should treat models as SaaS, quickly swapping for better ones, and focus on product and user interactions. The talk warns against toxic practices like prematurely hiring ML engineers without data or blindly adopting tools, advocating instead for deliberate eval practice and data literacy. On tactics, they compare LLM-as-judge (quick to prototype) vs fine-tuned evaluators (more precise and faster), and emphasize looking at real user data regularly with automated guardrails. Ultimately, they conclude that going from demo to production requires sustained investment in infrastructure and evaluation, echoing MLOps lessons from a decade ago.