AI core

Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind

May 22, 2026 · 19:38

Florina Muntenescu and Oli Gaymond from Google DeepMind explain Gemini Nano, an on-device model that ships at 3–4 GB and is shared across apps via the AI Core system service, which handles scheduling, queuing background batch jobs overnight, and prioritizing foreground apps. The MLKit GenAI APIs (prompt API for text/image input, text output) give access to Gemini Nano, but require flagship devices from the last two years for optimal performance; classic MLKit (vision, OCR) runs on over a billion devices. Hybrid inference, launched weeks before this talk, automatically falls back to Gemini Flash in the cloud when the on-device model isn't available, extending reach. An embedding API for RAG-style solutions is coming soon. For fully custom models, LiteRT offers an alternative path but requires more developer effort for testing and optimization.

From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google

May 20, 2026 · 21:01

Google's Cormac Brick explains how developers can build on-device AI agents using either system-level Gemini Nano via AI Core or app-level custom models via LiteRT-LM. He demonstrates a skill harness built on Gemma 4 that enables agentic tasks like restaurant roulette, running fully on-device with JavaScript UI. For fine-tuned tiny models, Function Gemma at 270M parameters improved from 46% to over 90% accuracy on eight of ten app-intent functions after synthetic data fine-tuning. The talk also covers the Eloquent transcription app, which chains two Gemma 3-based models (ASR and text polishing) under a few hundred million parameters for offline use. Key trade-offs are latency, privacy, and customization versus system integration effort.

TLMs: Tiny LLMs and Agents on Edge Devices with LiteRT-LM — Cormac Brick, Google

May 3, 2026 · 1:20:58

Cormac Brick from Google AI Edge explains how Tiny LLMs (sub-1B parameters) and on-device agent skills are making edge AI practical. He details LiteRT-LM, an open-source runtime that runs Gemma models on Android, iOS, and embedded systems, achieving over 1,000 tokens/s on high-end phones. Agent skills use progressive disclosure—loading skill details on demand—enabling reliable tool calling on 2B-4B models. For app deployment, fine-tuning boosts tiny model reliability by 20-40 points (e.g., Function Gemma 270M hit 86% accuracy on 10 functions). Synthetic data workflows and modular design (ASR + text polishing) power real apps like AI Edge Eloquent, which runs entirely offline. Safety is managed through system-level checkers and narrow functional scope for tiny models.