What Lies Beneath the API — Benjamin Cowen, Modal
Jun 2, 2026 · 12:40
In this episode, Ben Cowen from Modal argues that as AI products mature, fine-tuning becomes essential, citing cases like Intercom beating their frontier API at 1/10th the cost. He identifies three signals it's time to fine-tune: paying more for the API than customers pay you, evaluative plateaus, and latency requirements that shared endpoints can't meet. Cowen explains that supervised fine-tuning now fits in 300 lines of Python and that reinforcement learning rollouts can scale to 50,000 sandboxes using serverless platforms like Modal. He contends that frontier labs aim to win at everything, while businesses need to win at their specific logic, making fine-tuning a natural destination. The episode provides practical guidance on when and how to make the leap, emphasizing that building an agent harness and collecting eval data already sets the stage for training.
