Ghostwriter

Building AI For All: Amjad Masad & Michele Catasta

Oct 23, 2023 · 25:13

Amjad Masad and Michele Catasta announce Replit's AI for all, giving free AI-powered coding to millions of users. They detail the training of Replit Code v1.5, a 3.3B parameter model trained on 1 trillion tokens of code, which outperforms StarCoder 3B and approaches Code LLaMA 7B in Human Eval despite being half the size. The model is optimized for low-latency inference, generating over 200 tokens per second per GPU. Catasta explains the data pipeline using permissively licensed code from The Stack and Replit public repos, with five epochs of repeated high-quality data. The model is released open-source with a commercially permissive license. Partnerships with Glaive AI for instruction fine-tuning, Morph Labs for a novel fill-in-syntax-tree format, and Perplexity for fast model serving are also unveiled.

The 1,000x AI Engineer: Swyx

Oct 23, 2023 · 9:27

Swyx argues that AI engineers are just in time for a 1,000x opportunity, drawing on historical tech cycles and compute scaling laws. He cites Carlota Perez's tech revolution cycles, placing the AI revolution's start at AlexNet in 2012. With compute growing 600x by decade's end, GPT-3 took one person-year of compute, GPT-4 took 100, and GPT-10 will exceed all human compute ever. He defines three AI engineer types: AI Enhanced (Copilot-like), AI Product (Midjourney), and AI Agent (Auto-GPT). To achieve 1,000x, he advises scaling knowledge from O(n) (attending talks) to O(n²) (teaching others) to O(2^n) (building networks).