Chroma

Building Intelligent Research Agents with Manus - Ivan Leo, Manus AI (now Meta Superintelligence)

Dec 30, 2025 · 1:21:30

Ivan Leo of Manus AI introduces the Manus API and Manus 1.5, positioning the platform as a general action engine for building intelligent research agents. He demonstrates how the API enables asynchronous task dispatching, file uploads with automatic 48-hour deletion, and webhooks for scalable workflows. The workshop covers creating a Slack bot that integrates Manus's browser operator, file attachments, and connectors to private data sources like Notion. Leo shows real-time demos including a custom French learning app, a conference event scraper with calendar integration, and an invoice processing bot that references company policies. He also discusses the roadmap for memory persistence, document exporting (PPTX/PDF), and enhanced permission systems for browser automation.

Jack Morris: Stuffing Context is not Memory, Updating Weights is

Dec 29, 2025 · 1:02:44

Jack Morris argues that large language models fail at niche, long-tail knowledge tasks, such as optimizing AMD GPU kernels or answering private company queries, because they rely on context windows and RAG, which suffer from quadratic self-attention costs and context rot. He advocates for a third paradigm—training knowledge directly into model weights—using synthetic data generation (e.g., synthetic continued pretraining from Stanford) to expand small datasets and parameter-efficient methods like LoRA or memory layers to avoid catastrophic forgetting. Morris demonstrates that full fine-tuning on a 3M 10-K report causes the model to only regurgitate exact sentences, whereas generating diverse synthetic question-answer pairs enables better generalization. He notes that RL-based fine-tuning (e.g., GRPO) can achieve improvements with as few as 14 parameters, while memory layers offer the best trade-off between learning and forgetting. The episode also explores temporal information handling, federated learning resurgence, and the practical decision boundary between RAG and weight-based injection based on data freshness and volume.

Wisdom-Driven Knowledge Augmented Generation at Scale - Chin Keong Lam, Patho AI

Aug 22, 2025 · 18:43

Chin Keong Lam, founder and CEO of Patho.ai, argues that Wisdom-Driven Knowledge Graphs (KAG) significantly outperform traditional RAG systems for complex quantitative analysis and expert-level advice. He presents a wisdom-decision-situation feedback loop where knowledge, experience, and insight feed into wisdom, mapped to a practical competitive analysis chatbot for a marketing client. The system uses N8n and a multi-agent architecture with a supervisory Wisdom Agent overseeing insight, knowledge, and strategy agents, all updating a centralized knowledge graph. Lam cites five reasons knowledge graphs beat RAG: capturing complex relationships, improved accuracy, scalability, rich multi-hop query capability, and seamless data integration. He demonstrates how vector RAG fails on numerical reasoning (e.g., Apple revenue query) while KAG returns precise evidence-based answers. Lam recommends a hybrid graph extraction combining LLM extraction with expert pruning, and reports benchmark results of 91% accuracy, 85% flexibility, and strong traceability and scalability.

Prompt Engineering is Dead — Nir Gazit, Traceloop

Jun 27, 2025 · 14:19

Nir Gazit, CEO of Traceloop, argues that manual prompt engineering is ineffective and demonstrates an automated, test-driven alternative. He improves a RAG-based documentation chatbot by 5x without hand-tuning prompts, using an LLM-as-a-judge evaluator with 20 example questions (each with 3 expected facts) and a CrewAI agent that iteratively optimizes the prompt based on evaluation scores. The initial score of 0.4 jumps to 0.9 after just two iterations. Gazit warns of overfitting to the small dataset and suggests splitting into train/test sets. The approach replaces manual tweaking with a 'gradient ascent' on evaluators, effectively vibe-engineering prompts.

OpenLLMetry is all you need

Feb 22, 2025 · 9:12

Nir, CEO of Trace Loop, introduces OpenLLMetry, an open-source project extending OpenTelemetry for tracing and monitoring GenAI applications. OpenTelemetry, maintained by CNCF, standardizes logging, metrics, and traces across cloud environments, supported by platforms like Datadog, New Relic, and Grafana. OpenLLMetry provides over 40 automatic instrumentations for foundation models (OpenAI, Anthropic, Cohere), vector databases (Pinecone, Chroma), and frameworks (LangChain, LlamaIndex, CrewAI). These instrumentations emit logs, metrics, and traces in OpenTelemetry format, allowing users to send data to any supported observability backend with a configuration change, avoiding vendor lock-in.

Building Production-Ready RAG Applications: Jerry Liu

Nov 15, 2023 · 18:35

Jerry Liu, CEO of LlamaIndex, explains how to productionize Retrieval Augmented Generation (RAG) systems by moving beyond naive implementations. He identifies key challenges: low retrieval precision causing hallucination, low recall from insufficient top-K, and lost-in-the-middle problems. Liu advocates starting with 'table stakes' improvements like tuning chunk sizes (showing optimal values per dataset), adding metadata filters (e.g., year=2021 for SEC 10Q queries), and hybrid search. More advanced techniques include 'small-to-big retrieval', embedding smaller chunks for precision then expanding windows for synthesis, and using reranking to improve recall. Finally, he explores agent architectures where each document becomes a tool for summarization or QA, and fine-tuning—generating synthetic query datasets from raw text to fine-tune embeddings, or distilling GPT-4's chain-of-thought into GPT-3.5 Turbo for better reasoning.

Retrieval Augmented Generation in the Wild: Anton Troynikov

Nov 15, 2023 · 12:20

Anton Troynikov, co-founder of Chroma, explains that retrieval augmented generation (RAG) requires more than simple vector search—it needs human feedback, self-updating memory, and agent interaction to handle dynamic data. He covers challenges like choosing the right embedding model, chunking strategies (including using language model perplexity), and determining result relevance without distractors. Chroma is building a horizontally scalable cluster, a cloud technical preview by December, and support for multimodal data. The episode argues that a capable memory system is key to making AI agents truly functional, citing the Voyager paper where Chroma stored learned skills for Minecraft agents.

AI Engineering 201: The Rest of the Owl

Nov 8, 2023 · 56:57

Charles Frye, instructor of the Full Stack LLM Bootcamp, presents essential patterns for building language user interfaces (LUIs), arguing that while RAG chatbots are the 'to-do list app' of AI engineering, structured outputs via function calling (e.g., OpenAI's JSON schema, Instructor library) improve robustness, and agents with memory (like generative agents or Voyager in Minecraft) represent the true AI frontier. He emphasizes the need for hybrid search combining vector and keyword retrieval (citing Vespa, Postgres, Redis), and warns that monitoring and evaluation are the hardest engineering challenges: monitoring user behavior, latency quantiles (especially 99th percentile), and costs must be paired with observability tools like Honeycomb or Gantry, while evaluation often requires iterated decomposition or using LLMs as evaluators (GPT-4 as 90th percentile crowd worker). The episode concludes that shipping to learn — starting with production data to generate tests — is the dominant engineering mindset, and that the field is still filling in the gaps between inference and full product value.

The AI Pivot: With Chris White of Prefect & Bryan Bischof of Hex

Nov 7, 2023 · 35:16

Chris White (CTO of Prefect) and Bryan Bischof (Head of AI at Hex) detail how their non-AI startups successfully pivoted to integrate AI, arguing that ruthless prioritization and deep product integration are key. White explains Prefect built the open-source Marvin project to learn from LLM experimenters, while adding AI features like error summaries to its core orchestration product, but had to restrain over-enthusiasm from engineers. Bischof describes Hex's Magic feature as an augmentation, not a separate product, and explains they built their own evaluation system but chose not to build a vector database, and killed a promising feature called Crystal Ball to avoid splitting the product experience. They both emphasize machine-to-machine interfaces and typed outputs, and caution that the work is often tedious data engineering. Their hot takes: White says stop building chat interfaces, AI is a tool; Bischof warns that the journey is boring but worth it.