cuDNN

Luminal - Search-Based Deep Learning Compilers - Joe Fioti

Jun 3, 2025 · 24:35

Joe Fioti presents Luminal, a search-based deep learning compiler that simplifies ML libraries to 12 primitive operations and uses search to automatically discover optimized kernels like flash attention. By representing models as directed acyclic graphs of these simple ops, Luminal keeps its codebase under 5,000 lines yet can run all major models. Its compiler applies 20-25 rewrite rules to search through equivalent GPU kernels, profiling to find the fastest—automatically rediscovering flash attention, an algorithm that took five years for the industry to develop. Data movement accounts for 99% of runtime, so kernel fusion merges many ops into one, dramatically speeding execution. An external auto-grad crate adds training support without altering the core. Future plans include supporting AMD, TPUs, and a serverless cloud that exports optimized graphs for inference.

Unlocking Developer Productivity across CPU and GPU with MAX: Chris Lattner

Jul 25, 2024 · 18:33

Chris Lattner, CEO of Modular, presents MAX, a unified AI framework that accelerates Gen AI inference by combining CPU and GPU programming into a single Pythonic model, and Mojo, a new programming language that extends Python to systems programming with 100–1000x speedups. Lattner argues that current fragmentation across PyTorch, ONNX, TensorRT, and hardware-specific libraries slows innovation, and MAX replaces the entire stack—including cuDNN and Intel MKL—with a consistent, compiler-driven approach. He demonstrates that MAX's Int4/Int6 quantization achieves 5x faster performance than llama.cpp on cloud CPUs, and that its GPU matrix multiplication beats NVIDIA's cuBLAS by up to 30%. Mojo enables developers to write for loops and tokenizers (e.g., for LLaMA 3) in Python-like syntax without dropping to C++ or Rust. MAX is free and available now for CPU inference; GPU support launches in September with early access via Discord.