Flash

GPU Cloud Deployment Without Leaving Your IDE — Audry Hsu, RunPod

Jun 9, 2026 · 20:19

Audry Hsu of RunPod introduces Flash, a Python SDK that deploys GPU cloud functions from a developer's IDE with a single decorator, eliminating the slow iteration cycle of commits, Docker builds, and server allocation. She demonstrates hot reload, swapping Stable Diffusion XL Turbo for DreamShaper instantly, and a pipeline that chains Qwen 3 for prompt generation, DreamShaper for image rendering, and Nano Banana 2 for photo composition. RunPod's serverless H100 pricing is $0.00116 per second, charged only during active inference. Hsu recommends starting with pods for experimentation and switching to serverless when scaling to hundreds of workers across data centers.

How Google DeepMind Runs Agents at Scale — KP Sawhney & Ian Ballantyne, Google DeepMind

May 24, 2026 · 25:13

Ian Ballantyne and KP Sawhney from Google DeepMind explain how the company scales its agentic platform anti-gravity, featuring a Darwinian skills library and quota management prioritizing paying customers. KP explains deep research currently passes huge context blobs, but his focus is replacing them with a shared file system, enabling artifact generation like infographics. Token-hungry agents require brute-force quota limits; 24/7 monitoring stops spikes, and internal teams face worse quotas than customers. Observability uses a custom web app tracing agent trajectories to raw predict requests. The skills library relies on contributions from domain experts, with survival through evaluation in sandboxed environments. For code review, per-language auto-review models fine-tuned on style guides automatically comment on PRs.

Teaching Gemini to Speak YouTube: Adapting LLMs for Video Recommendations to 2B+DAU - Devansh Tandon

Jul 16, 2025 · 22:51

Devansh Tandon, a Product Manager at Google leading YouTube's discovery system, details how YouTube adapted Gemini LLMs to power its recommendation engine for billions of daily active users. The team built SemanticID, a tokenization system that compresses video features into semantically meaningful tokens, creating a new language for YouTube content. They then continued pre-training Gemini on sequences of user watches to make the model bilingual in English and this video language. For generative retrieval, they prompt the adapted model with user demographics and watch history to output video recommendations as SemanticIDs, achieving 95%+ cost savings to serve at scale. Challenges include serving billions of users with low latency and handling video freshness—Taylor Swift's new music video must be recommendable within minutes. Tandon argues LLM-led recommendations are a bigger consumer application than search and hints at future interactive, steerable recommendations and even personalized content creation.

Thinking Deeper in Gemini — Jack Rae, Google DeepMind

Jul 10, 2025 · 18:13

Jack Rae, lead of Gemini Thinking at Google DeepMind, presents thinking as a solution to the fixed test-time compute bottleneck in large language models. He explains that Gemini inserts a thinking stage where the model iterates via reinforcement learning, learning to self-correct and explore multiple strategies. This enables a continuous cost-performance tradeoff via thinking budgets, improving reasoning on math and code. Future work includes Deep Think, which raises USA Math Olympiad performance from the 50th to the 65th percentile by scaling inference compute further. Rae envisions models that, like mathematician Ramanujan, achieve deep, data-efficient reasoning from limited knowledge.