What We Learned from Using LLMs in Pinterest — Mukuntha Narayanan, Han Wang, Pinterest
Jul 16, 2025 · 18:13
Pinterest search engineers Han Wang and Mukuntha Narayanan present four key learnings from integrating LLMs into the platform's search relevance system. Fine-tuned LLM cross-encoders improve relevance prediction by 12% over multilingual BERT and 20% over SearchSage using an 8B model. VLM-generated image captions and user action features enrich pin text representations and further boost performance. Knowledge distillation into a bi-encoder student model, trained on 100x more data via semi-supervised learning, enables production serving with real-time query embeddings and 85% cache hit rate. Relevance-tuned embeddings serve as general-purpose semantic representations across multiple surfaces, and the system yields relevance gains in multiple languages and countries despite a predominantly US training set.
