Gemini API

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

Jul 14, 2026 · 21:46

Philipp Schmid, Staff Engineer at Google DeepMind, argues that shipping agent skills without evals is like merging code without tests, leading to unreliable performance. He explains that most of the 50,000+ skills on Skill Bench lack evals and are often AI-generated, which can negatively impact performance. The talk distinguishes between capability skills (teaching models new tasks) and preference skills (encoding team-specific workflows), with evals enabling retirement of skills as models improve. Schmid shares practical advice: keep skill descriptions concise (under 500 lines), include negative tests, and write directives over passive information. He details a case study creating 117 test cases for the Gemini interactions API, achieving 90% valid code generation using regex-based asserts and LLM-as-judge. The key takeaway: always run ablation tests with and without skills to measure their true impact, and start small with as few as 10-20 eval samples.

From Transcription to Live Music: Gemini's Audio Stack — Thor Schaeff, Google DeepMind

Jun 9, 2026 · 19:34

Thor Schaeff from Google DeepMind presents the Gemini audio stack—Gemini 3 Flash Preview for deep audio understanding, Gemini 3.1 Flash Live for real-time sound-to-sound multimodal interaction, and Lyria 3 for music generation. He shows how a single API call extracts speaker labels, timestamps, emotions, language detection, and translation, and how speech generation uses a 'director's note' to modify a base voice's accent and tone. The talk culminates in a live demo where the Gemini Live model uses Lyria via tool calls to generate a German techno schlager about the UK startup scene.

Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind

May 22, 2026 · 19:38

Florina Muntenescu and Oli Gaymond from Google DeepMind explain Gemini Nano, an on-device model that ships at 3–4 GB and is shared across apps via the AI Core system service, which handles scheduling, queuing background batch jobs overnight, and prioritizing foreground apps. The MLKit GenAI APIs (prompt API for text/image input, text output) give access to Gemini Nano, but require flagship devices from the last two years for optimal performance; classic MLKit (vision, OCR) runs on over a billion devices. Hybrid inference, launched weeks before this talk, automatically falls back to Gemini Flash in the cloud when the on-device model isn't available, extending reach. An embedding API for RAG-style solutions is coming soon. For fully custom models, LiteRT offers an alternative path but requires more developer effort for testing and optimization.

Any-to-Any: Building Native Multimodal Agents - Patrick Löber, Google DeepMind

May 20, 2026 · 16:21

Patrick Löber, a member of the technical staff at Google DeepMind, explains how to build native multimodal agents using the Gemini API ecosystem, covering multimodal understanding, native image and speech generation, and real-time interaction via the Live API. He demonstrates constructing a NotebookLM clone as an agentic system where a reasoning Gemini model decides whether to generate an infographic or podcast-style audio using function calls to specialized models like Nano Banana for images and a text-to-speech model for speech. The episode details practical implementation: uploading PDFs, video, and audio files; using context caching to reduce costs by 90%; and generating infographics or multi-speaker audio directly from prompts. Löber highlights that native generation models understand world context—like drawing arrows on a map to produce the Golden Gate Bridge—and that the Live API enables audio-to-audio interactions with a single architecture, supporting multiple languages and accents.

Building Conversational Agents — Thor Schaeff and Philipp Schmid, Google DeepMind

Apr 30, 2026 · 1:47:34

Thor Schaeff and Philipp Schmid of Google DeepMind demonstrate building conversational agents with the Gemini Interactions API and Live API, arguing that the new Interactions API simplifies agent development by introducing server-side state management, unified content blocks, and remote MCP support. They walk through creating a coding agent with read/write file and bash tools using Gemini 3 Flash, highlighting how the model uses tool calling to execute commands and return results. The Live API, powering real-time voice and video interactions, supports native multilingual audio, tool calling, and Google Search grounding, though the presenters acknowledge its current limitations in production, including lack of session transcript retrieval and shorter context windows (15 minutes audio-only). They showcase a Live Jukebox demo integrating Lyria 3 music generation and discuss business use cases like Shopify Sidekick, Waymo, and elderly companion apps, emphasizing that while the technology is ready for experimentation, production readiness depends on partner integrations for observability and compliance.

A year of Gemini progress + what comes next — Logan Kilpatrick, Google DeepMind

Jul 10, 2025 · 11:58

Logan Kilpatrick, head of product for Google AI Studio at DeepMind, announces the final update to Gemini 2.5 Pro, which achieves state-of-the-art results on Aider and HLE benchmarks. He details Google's 50x increase in AI inference over the past year, driven by merging research and product teams into DeepMind. The episode outlines Gemini's evolution toward a universal assistant that unifies Google products, with upcoming features including proactivity, native audio and video capabilities (Veo), and smaller models. Kilpatrick also previews developer-focused updates: a SOTA embeddings model, a deep research API, and Veo 3 and Imagine 4 in the API, alongside repositioning AI Studio as a dedicated developer platform.