Waymo's EMMA: Teaching Cars to Think - Jyh Jing Hwang, Waymo
Jul 26, 2025 · 17:28
Jyh-Jing Hwang of Waymo presents EMMA, an end-to-end multimodal model built on Gemini that directly processes camera inputs and route text into driving waypoints, achieving state-of-the-art open-loop planning on the NuScenes benchmark without LiDAR or HD maps. Adding chain-of-thought reasoning—identifying critical objects and meta-decisions—further surpasses specialized models like MotionLM on Waymo's 100K dataset. Co-training on tasks such as 3D detection and road graph estimation maintains competitive performance across domains. For evaluation, Waymo leverages generative video models (e.g., Veo2) to simulate diverse conditions like rain and night, testing EMMA's robustness. The work demonstrates how Gemini's generalization can help scale Waymo's autonomous driving to new cities by handling long-tail scenarios.
