Understanding AI Stakes to Break Production Code: Philip Rathle
Dec 31, 2024 · 23:24
Philip Rathle, CTO of Neo4j, argues that the level of 'stakes' in an AI application determines the production barriers and appropriate solutions, from low-stakes summarization to high-stakes uses requiring knowledge graphs and human-in-the-loop systems. He distinguishes pilot (full autonomy) from co-pilot (human oversight), showing how vector RAG solves moderate stakes but fails for high-stakes needs like tightening bolts on a 737 MAX 9. He promotes Graph RAG for deterministic reasoning and fact retrieval. Attendees share learnings: using LLMs to write code rather than reason directly, adapting to user keyword habits, focusing on few projects with achievable accuracy, building eval pipelines early, and for regulated loans offering three options with explanations instead of a single recommendation.
