LLM Scientific Reasoning: How to Make AI Capable of Nobel Prize Discoveries: Hubert Misztela
Sep 23, 2024 · 20:00
Hubert Misztela, an AI researcher at Novartis, argues that LLMs require more than naive RAG to achieve scientific reasoning capable of Nobel-level discoveries. He uses the 1990s petunia flower experiment — where three separate biological phenomena (including RNA interference) went unexplained for eight years until their common cause was found — as a benchmark. Misztela classifies question complexity from one-to-one to multi-needle problems and demonstrates that reasoning before retrieval (e.g., routing, GraphRAG) and after retrieval (e.g., relevance classifiers) improves hypothesis generation. His experiments show that strict prompting and a relevance classifier that evaluates each paper's contribution to advancing a hypothesis can recover the correct RNA/DNA link without post-discovery knowledge. He concludes that harder problems demand reasoning steps and that brute-force checking with LLMs may outperform embedding distances.
