Yegor Denisov-Blanch

Can you prove AI ROI in Software Eng? (Stanford 120k Devs Study) – Yegor Denisov-Blanch, Stanford

Dec 11, 2025 · 16:40

Stanford researcher Yegor Denisov-Blanch presents research based on 120,000+ developers across 600+ companies, arguing that AI ROI in software engineering is often negative despite apparent productivity gains. The study finds a median 10% productivity lift from AI, but a widening gap between top and bottom performers. Key drivers include codebase hygiene (environment cleanliness index with R² 0.40) and AI usage quality over volume. A company case study shows pull requests increased 14% but code quality dropped 9% and rework rose 2.5x, yielding no net effective output gain. Denisov-Blanch proposes a measurement framework using a primary metric (engineering output via ML model) paired with guardrail metrics, and emphasizes that companies can retroactively measure impact via git history without waiting for experiments.

Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford

Jul 23, 2025 · 18:12

Yegor Denisov-Blanch from Stanford presents findings from a study of nearly 100,000 developers across 600+ companies, showing AI boosts developer productivity by an average of 15-20%, but the effect varies widely. The study measures functionality delivered, not commits, and reveals that AI introduces significant rework—bug fixes from AI-generated code. Productivity gains depend on task complexity, codebase maturity (Greenfield vs. Brownfield), language popularity, and codebase size: low-complexity Greenfield tasks see 30-40% gains, while high-complexity Brownfield tasks see only 0-10%. AI can even decrease productivity for low-popularity languages like COBOL or Haskell, and context window limitations reduce performance on larger codebases. The study provides data-driven guidance for when AI helps or hinders, with a matrix showing 20% gains for low-complexity common languages and minimal gains for complex niche ones.