Building AI For All: Amjad Masad & Michele Catasta
Oct 23, 2023 · 25:13
Amjad Masad and Michele Catasta announce Replit's AI for all, giving free AI-powered coding to millions of users. They detail the training of Replit Code v1.5, a 3.3B parameter model trained on 1 trillion tokens of code, which outperforms StarCoder 3B and approaches Code LLaMA 7B in Human Eval despite being half the size. The model is optimized for low-latency inference, generating over 200 tokens per second per GPU. Catasta explains the data pipeline using permissively licensed code from The Stack and Replit public repos, with five epochs of repeated high-quality data. The model is released open-source with a commercially permissive license. Partnerships with Glaive AI for instruction fine-tuning, Morph Labs for a novel fill-in-syntax-tree format, and Perplexity for fast model serving are also unveiled.
