Vertex AI

Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind

May 22, 2026 · 19:38

Florina Muntenescu and Oli Gaymond from Google DeepMind explain Gemini Nano, an on-device model that ships at 3–4 GB and is shared across apps via the AI Core system service, which handles scheduling, queuing background batch jobs overnight, and prioritizing foreground apps. The MLKit GenAI APIs (prompt API for text/image input, text output) give access to Gemini Nano, but require flagship devices from the last two years for optimal performance; classic MLKit (vision, OCR) runs on over a billion devices. Hybrid inference, launched weeks before this talk, automatically falls back to Gemini Flash in the cloud when the on-device model isn't available, extending reach. An embedding API for RAG-style solutions is coming soon. For fully custom models, LiteRT offers an alternative path but requires more developer effort for testing and optimization.

Let's go Bananas with GenMedia — Guillaume Vernade, Google DeepMind

May 18, 2026 · 1:17:14

Guillaume Vernade from Google DeepMind demonstrates the full GenMedia stack—Nano Banana for images, Veo for video, Lyria for music, and TTS—by illustrating an open-source book live. The core insight is that Gemini acts as the prompt engineer for every other model, and this works partly because the gen media models were trained on prompts written by Gemini. He shows how to generate character portraits, chapter scenes, animated video clips using images as first frames, and distinct music per chapter, with Lyria Realtime allowing continuous music that responds to new prompts like a DJ. A new interactions API caches context server-side to make chained multi-turn calls cheaper. For TTS, he uses a trick: assigning different speaking styles (e.g., long poetic pauses vs. breathless stutter) to the same two voices to create four distinct-sounding characters.

AI Engineering with the Google Gemini 2.5 Model Family - Philipp Schmid, Google DeepMind

Jul 11, 2025 · 1:44:51

In this workshop, Philipp Schmid from Google DeepMind demonstrates AI Engineering with the Gemini 2.5 model family, focusing on using Gemini 2.5 Flash via a free API tier for hands-on coding tasks including text generation, multimodal processing of images, audio, and PDFs, function calling with structured outputs, and integration with MCP servers. The session covers setting up API keys in AI Studio, uploading files via the Files API (free for 1 day), and controlling thinking budgets (0–24,000 tokens) to manage cost and reasoning depth. Schmid shows how Gemini natively processes videos at 1 frame per second for accurate timestamp extraction and how PDFs are handled by combining OCR text with image understanding. He introduces native tools like Google Search with grounding metadata, code execution, and URL context, and explains how MCP servers can be used seamlessly with the Gemini SDK for tool calling. The workshop also covers parallel vs sequential function calling, the Agent Development Kit (ADK), and the upcoming asynchronous function calling for the Live API, providing a practical path from simple generation to agentic workflows.

Veo 3 for Developers — Paige Bailey, Google DeepMind

Jun 21, 2025 · 20:37

Paige Bailey, engineering lead for Google DeepMind's DevRel team, presents Veo 3, the latest video generation model that natively produces synchronized audio—dialogue, sound effects, and music—from text and image prompts, marking a leap in coherence, physics simulation, and cinematic control. She demonstrates Veo 2's creative controls like reference-powered videos, outpainting, and character consistency, and shows how video generation has evolved from 2023's choppy clips to Veo 3's polished raccoon dancing in front of pyramids. A key demo replicates a Chick-fil-A commercial: Veo 2 required stitching multiple tools, while Veo 3 produced the full ad from a single prompt. The models are accessible via Vertex AI private preview and Gemini Ultra plans.

Agentic Workflows on Vertex AI: Rukma Sen

Feb 8, 2025 · 18:06

Google Cloud's Rukma Sen argues that AI agents are the essential bridge between generative models and users, and Vertex AI provides a platform to build and deploy them with enterprise-grade safety and flexibility. She defines agents as systems with a model (brain), tools (hands), and orchestration (nervous system), covering deterministic, generative, and hybrid types. For production reliability, she advocates multi-agent architectures, giving the example of a customer service system with dispatcher, expert, and supervisor agents—and a personal anecdote where a supervisor agent kept rejecting outputs. Use cases span customer support, employee HR, knowledge agents, and voice agents for drive-throughs. Vertex AI offers 150+ models (Google, Anthropic, LLaMA) and Agent Builder from no-code to full-code, with enterprise security and data privacy.

What's new from Anthropic and what's next: Alex Albert

Aug 5, 2024 · 13:38

Alex Albert, Head of Developer Relations at Anthropic, argues the industry is in a 'magic star icon phase'—tacking AI onto existing products—and urges rebuilding from the ground up. He announces Claude 3.5 Sonnet, which scores 64% on pull request evals (vs 38% for 3 Opus), with 200k context, vision improvements, and pricing at $3/M input and $15/M output tokens. New features include Artifacts for collaborative creation of code and React components from screenshots, Projects for team knowledge grounding, and an enhanced ToolUse API supporting hundreds of tools and structured JSON output. Future models (3.5 Haiku/Opus) will be smarter, cheaper, and faster; a steering API for interpretability allows clamping feature values to control outputs.