Compilers in the Age of LLMs — Yusuf Olokoba, Muna
Nov 24, 2025 · 17:36
Yusuf Olokoba, founder of Muna, explains how they built a Python compiler that converts plain Python inference functions—like Google's 270M-parameter Gemma embedding model—into self-contained C++ and Rust binaries using LLMs within a verifiable pipeline. The process involves symbolic tracing to generate an intermediate representation, type propagation to infer types for native compilation, and LLM-driven code generation to mass-produce native implementations of Python operations. After compiling into a shared library, the model can be loaded via FFI from any language (e.g., Node.js) and exposed through an OpenAI-compatible client, enabling developers to run open-source models anywhere—locally, cloud, mobile—with minimal code changes. Olokoba details why they abandoned PyTorch FX due to its PyTorch-only focus and reliance on fake inputs, and how LLMs help scale the coverage of elementary operations. The talk argues that this compiler approach solves hybrid inference—small edge models working with large cloud models—by moving beyond Python and Docker to more portable, low-latency native binaries.
