LLM

Let LLMs Wander: Engineering RL Environments — Stefano Fiorucci

Apr 8, 2026 · 40:35

Stefano Fiorucci demonstrates how to build Reinforcement Learning environments for language models using the open-source Verifiers library, arguing that training small models with verifiable rewards can surpass large closed models on specific tasks. He maps classic RL concepts to LLMs, introduces Verifiers components for single-turn, multi-turn, and tool environments, and then walks through an experiment where he takes LiquidAI's LLM 2 — a small open model — and transforms it into a tic-tac-toe master via supervised fine-tuning and GRPO-based reinforcement learning. After training, the model dominates random opponents and draws 85% of games against optimal ones, eventually outperforming GPT-5 Mini against identical optimal opponents. Fiorucci shares practical lessons: large batch sizes (≥256) ensure stable training, hidden biases in opponent algorithms can skew results, and starting from a base model (not a reasoning model) avoids truncated thinking traces. He concludes that if you can define a clear reward signal, you can build an environment and train a small specialized model to beat a large closed model at a fraction of the cost.

2025 in LLMs so far, illustrated by Pelicans on Bicycles — Simon Willison

Jul 9, 2025 · 18:30

Simon Willison reviews the past six months of LLM releases — including AWS Nova, Llama 3.3 70B, DeepSeek R1, Mistral Small 3, Claude 3.7 Sonnet, GPT 4.5, Gemini 2.5 Pro, GPT-4o, Llama 4, GPT 4.1, O3/O4 Mini, and Claude 4 — using his 'pelican on bicycle' SVG benchmark to argue that local models have become good enough to run GPT-4 class models on a laptop and that combining tools with reasoning is the most powerful technique in AI engineering, while noting risks like prompt injection and the 'lethal trifecta'. He tracks 30 significant model releases, highlighting that Mistral Small 3 (24B) matches Llama 3 70B's performance, which itself matched the 405B model, enabling local inference. DeepSeek's R1 caused a $500B+ Nvidia stock drop on January 27. GPT 4.1 Nano is the cheapest model yet at a fraction of a cent per pelican. He also examines bugs: ChatGPT's sycophantic 'shit-on-a-stick' incident and Claude 4's tendency to snitch to authorities when given ethical instructions and email tools. Willison concludes that while the pace is accelerating, control over context and security remain critical.

Open Questions for AI Engineering: Simon Willison

Nov 25, 2023 · 24:33

Simon Willison recaps the AI industry's past year—from ChatGPT's breakthrough to open-source local models—and poses key open questions for AI engineering. He argues that ChatGPT's chat interface, while popular, is a poor fit for advanced use, urging better UIs like his command-line tool LLM. He celebrates Meta's Llama release as a 'stable diffusion moment' for language models and highlights the rise of small, locally-run models such as Replit's 3B model, asking how small models can remain useful. On security, he warns that prompt injection remains unsolved after 13 months, limiting what can safely be built. He champions ChatGPT's Code Interpreter (which he dubs 'Coding Intern') as the most exciting tool, able to write and compile C code on a phone, and argues that LLMs flatten the learning curve, making programming accessible to more people. He concludes by urging the community to build tools that enable anyone to automate tedious tasks.