LanceDB

Scaling Enterprise-Grade RAG: Lessons from Legal Frontier - Calvin Qi (Harvey), Chang She (Lance)

Jul 29, 2025 · 16:40

Calvin Qi of Harvey and Chang She of LanceDB detail how Harvey builds enterprise-grade RAG for legal use cases, tackling massive scale (tens of millions of documents per corpus), spikey workloads, and non-negotiable privacy. They explain why domain-specific reranking is essential for ambiguous, expert queries like 'What is the applicable regime to covered bonds issued before 9 July 2022 under Directive EU 2019/2062?' and stress that evaluation-driven development—from expert reviews to automated metrics—is key to accuracy. Chang shows how LanceDB's AI-native multimodal lakehouse, built on the open-source Lance format, enables low-latency, high-throughput retrieval across billions of vectors (indexing 3–4 billion in under 3 hours) while supporting offline analytics, GPU indexing, and compute-storage separation. The talk emphasizes that creative data understanding, iteration speed via good evals, and flexible infrastructure are critical for scaling RAG in domain-specific, privacy-sensitive environments.

The Hierarchy of Needs for Training Dataset Development: Chang She and Noah Shpak

Oct 15, 2024 · 16:32

Chang She (CEO of LanceDB) and Noah Shpak (AI data platform lead at Character AI) argue that data infrastructure is the critical bottleneck for LLM training, and LanceDB's columnar format solves the 'new cap theorem' for AI: needing fast scans, random access, and handling large multimodal blobs simultaneously. Noah explains how Character AI structures pre-training around wide domain coverage and post-training around granular analytics like token counts and difficulty scores, using synthetic data, quality scoring, and dataset selection to improve models. Chang details how Lance format provides zero-copy schema evolution, time travel, and indexing extensions for vector, scalar, and full-text search, enabling a single table to serve SQL analytics, PyTorch training, and production vector search. The episode emphasizes that speed and iterative dataset management are key to accelerating AI research, with LanceDB facilitating cheap random access and low-infra billion-scale vector search.

The AI Pivot: With Chris White of Prefect & Bryan Bischof of Hex

Nov 7, 2023 · 35:16

Chris White (CTO of Prefect) and Bryan Bischof (Head of AI at Hex) detail how their non-AI startups successfully pivoted to integrate AI, arguing that ruthless prioritization and deep product integration are key. White explains Prefect built the open-source Marvin project to learn from LLM experimenters, while adding AI features like error summaries to its core orchestration product, but had to restrain over-enthusiasm from engineers. Bischof describes Hex's Magic feature as an augmentation, not a separate product, and explains they built their own evaluation system but chose not to build a vector database, and killed a promising feature called Crystal Ball to avoid splitting the product experience. They both emphasize machine-to-machine interfaces and typed outputs, and caution that the work is often tedious data engineering. Their hot takes: White says stop building chat interfaces, AI is a tool; Bischof warns that the journey is boring but worth it.