Etsy

What if the harness mattered more than the model? - Aditya Bhargava, Etsy

Jul 7, 2026 · 32:04

Aditya Bhargava, a Staff Engineer at Etsy, argues that the harness—the tooling around a model—matters more than the model itself, especially for weaker models, and introduces Agency, a language he built for constructing agents. He cites a benchmark where only the harness changed, producing a 52.4% to 76.2% score range (over 20 points), showing that a better harness can make local open-source models perform near proprietary ones. Through seven evolving examples of a coding agent, he demonstrates key harness improvements: adding tools, ensuring safety via handlers and partial function application (PFA) to lock directory access, implementing a reason-act-feedback loop, using subagents to add capabilities without context bloat, and self-optimization via a built-in optimizer to systematically measure and improve prompts. He advocates for building expertise in harnesses to reduce reliance on proprietary models, enabling use of locally runnable ones. Agency provides simple tool syntax, safety features like interrupts and true pause/resume, and built-in optimizers for measured improvement.

Recsys Keynote: Improving Recommendation Systems & Search in the Age of LLMs - Eugene Yan, Amazon

Jul 16, 2025 · 20:54

Eugene Yan's keynote presents three innovations for recommendation systems: Semantic IDs, LLM-augmented data, and unified models. Kuaishou’s trainable multimodal Semantic IDs increased cold-start coverage by 3.6% and velocity by 3.5% by clustering content embeddings. Indeed used GPT-4 fine-tuning and distillation to filter bad job recommendations, reducing bad recs by 20% while boosting application rate 4% and cutting unsubscribes 5%. Spotify’s LLM-generated exploratory search queries drove a 9% increase in exploratory queries for new categories like podcasts. Netflix’s Unicorn unified ranker matched or exceeded specialized models across search and recommendations, while Etsy’s unified embeddings with a quality vector achieved a 2.6% sitewide conversion lift and 5% more search purchases.