Fine tune 20 Llama Models in 5 Minutes: Santosh Radha
Feb 9, 2025 · 6:26
Santosh Radha, Head of Product/Research at Agnostiq, demonstrates Covalent, an open-source platform that lets users fine-tune and deploy hundreds of Llama models directly from Python without Kubernetes or Docker. By adding a single decorator to Python functions, users specify GPU requirements (e.g., H100 with 48 GB, 18-hour limit) and run them on remote compute, paying only for actual usage (e.g., 87 cents for 6 minutes on an L14, 11 cents on a V100). Covalent supports job submission, inference endpoints with custom autoscaling (e.g., scale to 10 GPUs at 9 AM daily), and automated workflows for training, evaluation, and deployment. Radha shows a workflow that iterates over 20 models, fine-tunes each, evaluates accuracy, sorts, and deploys the best—all from a Jupyter notebook with a single dispatch call. The talk, recorded at the AI Engineer World's Fair, emphasizes eliminating infrastructure overhead for accelerated compute.
