MediaPipe

From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google

May 20, 2026 · 21:01

Google's Cormac Brick explains how developers can build on-device AI agents using either system-level Gemini Nano via AI Core or app-level custom models via LiteRT-LM. He demonstrates a skill harness built on Gemma 4 that enables agentic tasks like restaurant roulette, running fully on-device with JavaScript UI. For fine-tuned tiny models, Function Gemma at 270M parameters improved from 46% to over 90% accuracy on eight of ten app-intent functions after synthetic data fine-tuning. The talk also covers the Eloquent transcription app, which chains two Gemma 3-based models (ASR and text polishing) under a few hundred million parameters for offline use. Key trade-offs are latency, privacy, and customization versus system integration effort.

TLMs: Tiny LLMs and Agents on Edge Devices with LiteRT-LM — Cormac Brick, Google

May 3, 2026 · 1:20:58

Cormac Brick from Google AI Edge explains how Tiny LLMs (sub-1B parameters) and on-device agent skills are making edge AI practical. He details LiteRT-LM, an open-source runtime that runs Gemma models on Android, iOS, and embedded systems, achieving over 1,000 tokens/s on high-end phones. Agent skills use progressive disclosure—loading skill details on demand—enabling reliable tool calling on 2B-4B models. For app deployment, fine-tuning boosts tiny model reliability by 20-40 points (e.g., Function Gemma 270M hit 86% accuracy on 10 functions). Synthetic data workflows and modular design (ASR + text polishing) power real apps like AI Edge Eloquent, which runs entirely offline. Safety is managed through system-level checkers and narrow functional scope for tiny models.