Everything you need to know about Fine-tuning and Merging LLMs: Maxime Labonne
Sep 25, 2024 · 17:52
Maxime Labonne from Liquid AI explains the LLM training lifecycle—pre-training, supervised fine-tuning (SFT), and preference alignment—and when to use fine-tuning over prompt engineering. He details SFT dataset creation (accuracy, diversity, complexity) and techniques like full fine-tuning, LoRA, and QLoRA, with key hyperparameters. The core of the talk is model merging: combining weights of fine-tuned models without GPU, using methods like SLERP (spherical linear interpolation for two models), TIES (pruning redundant parameters to merge many models), pass-through (concatenating layers, e.g., Meta LLaMA 3 120B Instruct by repeating layers gives strong creative writing), and Franken-MoE (extracting FFN layers from domain-specific models with a router). Labonne demonstrates these with his NeuralBeagle and Beyonder models, noting merged models dominate the OpenLLM leaderboard and that TIES merging often outperforms more experimental Mixture of Experts approaches.
