Foundry Local: Cutting-Edge AI experiences on device with ONNX Runtime/Olive — Emma Ning, Microsoft
Jun 27, 2025 · 22:52
Emma Ning, Principal PM at Microsoft, presents Foundry Local, a platform for building cross-platform on-device AI applications using ONNX Runtime and Olive. She argues local AI is essential for low-bandwidth, privacy-sensitive, cost-efficient, and real-time latency scenarios, noting that modern hardware and optimized models like Phi-4-mini and DeepSeek make it viable. Ning demonstrates Foundry Local's CLI, model benchmarking (Qwen 1.5B at 90 tok/s vs. Phi-4-mini with richer output), and a cross-platform document summarization app built with JavaScript SDK, running identically on Windows and macOS. She also previews a local agent with MCP servers (file system and OCR) that extracts receipt totals using Phi-4-mini. Customer testimonial from Sava (CEO of Pieces) highlights improved memory management and tokens-per-second, while another partner underscores ease of install and hybrid cloud-local solutions.
