Embedding Models

RAG is dead, right?? — Kuba Rogut, Turbopuffer

Jun 9, 2026 · 11:13

Kuba Rogut (Turbopuffer) argues that RAG isn't dead—it's evolving into agentic retrieval, contrasting Cursor's upfront indexing (which boosted answer accuracy 24% in their composer model) with Claude Code's per-session grep approach. He frames embeddings as cached compute, citing Jeff Dean: 'You don't need a trillion at once, you need the right million.'

How to look at your data — Jeff Huber (Chroma) + Jason Liu (567)

Aug 6, 2025 · 19:23

Jeff Huber (Chroma CEO) and Jason Liu argue that AI practitioners should look at their data—both inputs and outputs—with fast evals and conversation analysis to systematically improve retrieval and product decisions. Huber advocates for fast evals using golden datasets of query-document pairs over public benchmarks or expensive LLM judges, showing how synthetic queries aligned to real user behavior can empirically compare embedding models; in a Weights & Biases chatbot case study, Voyage 3 large outperformed text-embedding-3-small and others. Jason Liu focuses on outputs: extract structured metadata from conversations, cluster them to find segments, then compare KPIs like performance across clusters to prioritize what to fix, build, or ignore—for example, if 40% of conversations involve data visualization and the agent performs poorly, invest in better plotting tools. This population-level analysis enables impact-weighted decisions, turning vague metrics like 0.5 factuality into actionable insights by segment. The episode emphasizes that retrieval improvements are foundational, and once you have users, looking at conversation structure drives a data-driven product roadmap.

Layering every technique in RAG, one query at a time - David Karam, Pi Labs (fmr. Google Search)

Jul 29, 2025 · 20:22

David Karam of Pi Labs (formerly Google Search) walks through layering every technique in RAG, from in-memory retrieval to planet-scale search with 70+ corpus mix of token, embeddings, and knowledge graphs, jointly retrieved and re-ranked at 160,000 queries per second in under 200msec. He advocates a Quality Engineering Loop: baseline with simplest methods, analyze losses, then apply incremental techniques based on complexity-adjusted impact. Karam explains why queries like "falafel" are notoriously hard due to ambiguous intent, highlights failures of chunking documents, and shows when BM25 suffices, when relevance embeddings are needed, and when custom embeddings or domain-specific signals (price, popularity, user clicks) become essential. He covers query orchestration via fan-out, supplementary retrieval across back ends, and distillation for cost optimization, concluding that at sufficient complexity, problems must be punted to LLM or UX for graceful degradation.

POC to PROD: Hard Lessons from 200+ Enterprise GenAI Deployments - Randall Hunt, Caylent

Jul 23, 2025 · 19:16

Randall Hunt from Caylent shares hard lessons from over 200 enterprise GenAI deployments, arguing that evals, embeddings, and prompt engineering matter far more than fine-tuning. He emphasizes that speed and UX are critical; a slow inference kills adoption, while techniques like generative UI and caching can mitigate latency. Hunt details real-world examples: using audio amplitude spectrographs for sports highlight reels, pooling multimodal embeddings for nature footage search, and noting that nurses prefer chat over voice bots in noisy hospitals. He reports zero regressions moving from Claude 3.7 to 4, and advises optimizing context and economics, such as leveraging Amazon Bedrock batch for 50% cost reduction. The talk underscores that knowing your end customer and minimizing irrelevant context are key to production success.

What We Learned from Using LLMs in Pinterest — Mukuntha Narayanan, Han Wang, Pinterest

Jul 16, 2025 · 18:13

Pinterest search engineers Han Wang and Mukuntha Narayanan present four key learnings from integrating LLMs into the platform's search relevance system. Fine-tuned LLM cross-encoders improve relevance prediction by 12% over multilingual BERT and 20% over SearchSage using an 8B model. VLM-generated image captions and user action features enrich pin text representations and further boost performance. Knowledge distillation into a bi-encoder student model, trained on 100x more data via semi-supervised learning, enables production serving with real-time query embeddings and 85% cache hit rate. Relevance-tuned embeddings serve as general-purpose semantic representations across multiple surfaces, and the system yields relevance gains in multiple languages and countries despite a predominantly US training set.

A year of Gemini progress + what comes next — Logan Kilpatrick, Google DeepMind

Jul 10, 2025 · 11:58

Logan Kilpatrick, head of product for Google AI Studio at DeepMind, announces the final update to Gemini 2.5 Pro, which achieves state-of-the-art results on Aider and HLE benchmarks. He details Google's 50x increase in AI inference over the past year, driven by merging research and product teams into DeepMind. The episode outlines Gemini's evolution toward a universal assistant that unifies Google products, with upcoming features including proactivity, native audio and video capabilities (Veo), and smaller models. Kilpatrick also previews developer-focused updates: a SOTA embeddings model, a deep research API, and Veo 3 and Imagine 4 in the API, alongside repositioning AI Studio as a dedicated developer platform.

We accidentally made an AI platform: Jamie Turner

Oct 8, 2024 · 5:36

Jamie Turner, co-founder of Convex, explains how his company's backend platform accidentally became an AI platform because its reactive data flow paradigm—extending React's state reactivity to the server with subscribable queries and mutations—perfectly suits generative AI workflows. He describes Convex's architecture as seamlessly syncing state between backend steps and the application, enabling concurrent chains of tasks like automatic speech recognition, summarization, embedding generation, and finding related notes. Turner notes that post-ChatGPT, over 90% of Convex projects are generative AI. To support this shift, Convex added native vector indexes for schema fields and launched a startups program with discounts and exclusive forums. Upcoming high-level components will encapsulate state machines for sophisticated workflows. Turner presents Convex as a way for generative AI engineers to ship quickly and confidently.

Git push get an AI API: Ryan Fox-Tyler

Aug 23, 2024 · 45:01

Ryan Fox-Tyler and Matt Johnson Pint from Hypermode present a hands-on workshop demonstrating how to build and iteratively improve AI features using their platform, focusing on a GitHub issue triage app. They first illustrate the process with a multiplayer game (Hypercategories) that uses AI for classification and scoring. Then they build a trend summary function using OpenAI GPT-4 to summarize repository issues, and a classify issue function with a Hugging Face DistilBERT model for labeling issues as bug, feature, or question. Finally, they add natural language search for similar issues by creating an embeddings model (MiniLM) and a Hypermode collection, enabling vector search without external databases. The workshop emphasizes incremental iteration, mixing AI models with traditional code, and using Hypermode's automatic GraphQL generation and observability to speed development.

The Hidden Life of Embeddings: Linus Lee

Nov 7, 2023 · 18:15

Linus Lee, a Research Engineer at Notion, presents a tour of embedding visualization and manipulation at the AI Engineer Summit 2023. He demonstrates an encoder-decoder model fine-tuned from T5 that can reconstruct text from embeddings, allowing direct manipulation of features like length and sentiment by moving in latent space. Lee shows that mixing embeddings by splicing dimensions from two texts produces a semantic blend, and a linear adapter can decode text from OpenAI's text-embedding-ada-002 embedding space. Using CLIP, he interpolates between photographic and cartoon images and performs vector arithmetic to modify facial expressions. Lee releases these custom text embedding models on Hugging Face, enabling others to explore and interact with latent spaces. He argues that making model internals visible and manipulable fosters deeper understanding and more humane interfaces to generative AI.