How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS - Ishan Anand
Jul 13, 2025 · 1:41:34
Ishan Anand shows that GPT-2 small implemented in 600 lines of vanilla JavaScript makes LLMs understandable for web developers without ML backgrounds. He explains tokenization via byte-pair encoding, 768-dimensional embeddings representing semantic meaning via co-occurrence, and the Transformer's attention mechanism that lets tokens share context. The multi-layer perceptron learns next-token prediction through backpropagation, while the language head converts embeddings to token probabilities using softmax. Anand demonstrates each step—tokenization, embedding lookup, positional encoding, attention, MLP, and output—in a browser debugger, and notes that GPT-2's architecture underpins ChatGPT, with innovations like scale, supervised fine-tuning, and RLHF. The workshop provides an intuitive mental model of Transformers, turning perceived AI magic into understandable machinery.
