Effective AI Agents Need Data Flywheels, Not The Next Biggest LLM – Sylendran Arunagiri, NVIDIA
Jun 3, 2025 · 16:41
Sylendran Arunagiri of NVIDIA argues that effective AI agents rely on data flywheels, not the largest LLMs, enabling smaller models to achieve top accuracy at a fraction of cost. He details how NVIDIA's NeMo microservices power a continuous cycle of data curation, fine-tuning, evaluation, and guardrailing. Using an internal employee support agent (NVINFO), they achieved 96% accuracy with a 70B model but found that fine-tuning a smaller 8B model matched that accuracy, while a 1B model reached 94% with 98% lower inference cost and 70x latency reduction. The process involved curating 685 ground truth data points from user feedback and error analysis. He provides a framework: monitor user feedback, attribute errors, plan model experimentation, and execute regular retraining cycles.
