Under 5 minutes to a deployed LLM endpoint — Audry Hsu, RunPod
Jun 7, 2026 · 13:26
Audry Hsu of RunPod presents a cloud AI infrastructure platform that lets developers deploy LLM endpoints in under five minutes. RunPod originated from two failed crypto mining rigs in a basement in 2022; the founders offered the GPUs for free on Reddit in exchange for feedback, and the company now has 500,000 developers and $120 million in annual recurring revenue. The demo shows selecting a model from the Hub, configuring the context window, and deploying a serverless endpoint on H100s. The first request queues for 41 seconds due to cold start (container initialization and model download), while subsequent requests execute in about 1.5 seconds. Users pay only while a worker handles a request, making serverless ideal for bursty or batch workloads with autoscaling up to 15 workers.
