ChatGPT

Agent Output Is Not UX: Rendering Layer Your LLM Pipeline Is Missing - Bala Ramdoss, Amazon Lens

Jul 20, 2026 · 14:13

Bala Ramdoss (Amazon) argues that the UI delivery layer—generative UI—determines whether agentic AI features ship successfully, not the model itself. He presents three patterns from building Amazon Lens at scale: (1) a typed, versioned rendering contract where the model selects from a fixed catalog of components (e.g., flight carousel vs. list) and the client falls back safely for unknown types; (2) streaming UI chunks to show skeletons then fill progressively, shifting focus from total latency to time-to-first-chunk; and (3) a Backend-for-Frontend (BFF) that absorbs model output, hydrates actions and conversation context, and reuses existing native components so the AI feels like a natural part of the app. The episode demonstrates that mobile apps, which cannot be patched instantly, require this architecture to avoid crashes and deliver a snappy, trustworthy experience.

The UX of AI: Making AI-Powered Apps Your Users Don't Hate - Kathryn Grayson Nanz, Progress Software

Jul 18, 2026 · 35:59

Kathryn Grayson Nanz, Senior Design and Developer Advocate at Progress Software, argues that the success of AI-powered applications now depends on user experience rather than model performance. She identifies five pillars—trust, clarity, control, transparency, and meaningful benefit—and provides concrete patterns: citing sources to build trust, streaming text for clarity, allowing undo and version history for control, requesting granular permissions for transparency, and offering templates and next-step actions to ensure meaningful benefit. She emphasizes that developers must design these patterns themselves because AI can only remix existing interfaces, and that users need gradual introduction to AI features to avoid disengagement. The talk stresses that without addressing these UX challenges, users will abandon AI tools after a few failed attempts.

On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft

Jul 17, 2026 · 17:35

Pablo Castro, Distinguished Engineer and CVP for AI Knowledge at Microsoft, argues that AI agents require three knowledge types—intrinsic (model training), extrinsic (RAG and grounding), and learned (optimization loops)—and demonstrates how Microsoft's Foundry IQ, Azure AI Search, and Agent Optimizer deliver them. He shows that combining retrieval methods outperforms individual ones, citing Azure AI Search evaluations where combined methods improve evidence recall and answer completeness. He introduces Foundry IQ as a layered system with agentic retrieval that reflects on data to satisfy information needs before returning results. Castro demonstrates creating a knowledge base that connects unstructured data, structured Parquet tables, and the web, then linking it to an agent via MCP server. He also showcases the Agent Optimizer, which uses hill climbing on baseline evaluations to automatically refine agent instructions, tool definitions, and skills based on runtime traces, enabling continuous learning loops that capture organizational differentiation.

Imagination Engineering — Eve Bouffard, Head of Design, Y Combinator

Jul 16, 2026 · 16:04

Eve Bouffard, Head of Design at Y Combinator, introduces 'Imagination Engineering' as the art of stretching the mind to invent what seems impossible, arguing that idea generation is the new bottleneck as AI models become incredibly capable. She shares her experiment in 'thinking in public' with a Slack channel called 'Eve Thoughts,' where she dumps her stream of consciousness, and then used Opus 4.8 to build a personalized website (EveBouffard.com) that aggregates and visualizes those thoughts. The site dynamically surfaces her projects, quotes, tools, and books, and even applies shaders and translations. She also built 'Shape of Minds'—a tool that analyzes commonalities across history's greatest minds, revealing patterns like taking naps and barely eating. Bouffard demonstrates how to spin up agents on demand for learning and productivity, emphasizing that anyone can now create software on the fly from their stream of consciousness.

Stop Evaluating Models Like It's the 50s - Alejandro Vidal, Mindmakers

Jul 13, 2026 · 23:35

Alejandro Vidal of Mindmakers argues that counting correct answers in LLM benchmarks—classical test theory—should be replaced by item response theory (IRT) from psychometrics for richer evaluations. Using real data from Epoch.ai, he shows that while Claude Opus 4.1 scored 245 right and Gemini 3 Pro 247, IRT reveals Gemini is nearly one standard deviation more intelligent because it answers harder questions. IRT assigns each item difficulty (B) and discrimination (A), enabling benchmark auditing—Vidal flagged mislabeled items like a question about passengers whose gold answer was actually total people killed. He demonstrates reducing a 484-item benchmark to 97 items with 99% ranking correlation by selecting high-discrimination items, saving tokens and money. IRT also detects contamination via unexpected residuals, protects benchmarks through adaptive testing with unique fingerprint sets, and identifies model families (e.g., DeepSeek distillations correlate 0.38). Vidal previews future extensions like multidimensional models and alignment measurement.

From fork() to Fleet: Designing an Agent Sandbox Cloud — Abhishek Bhardwaj, OpenAI

Jul 13, 2026 · 44:34

Abhishek Bhardwaj from OpenAI explains the design of a secure, scalable agent sandbox cloud, arguing that microVMs (like Firecracker and Cloud Hypervisor) provide the strongest isolation for running untrusted AI-generated code despite performance trade-offs, and that persistent disk storage is the next frontier for unlocking long-running, stateful agent tasks. He compares runtime isolation technologies from simple fork-exec to containers (with namespaces and cgroups) to gVisor, and advocates for hardware-backed virtualization via microVMs to prevent guest exploits from reaching the host kernel. For persistence, he details incremental block-level snapshotting using copy-on-write filesystems and always-on distributed filesystems, enabling reliable checkpointing and Monte Carlo-style exploration. Orchestration challenges include low-latency sandbox creation through pre-warming or memory snapshots, and intelligent routing based on cached snapshot layers to minimize restore time. The talk positions storage and fast snapshot restore as key enablers for advanced agent capabilities, such as long-horizon tasks and failure recovery.

A Song of Types and Agents - Roberto Stagi, Ratel

Jul 12, 2026 · 14:16

Roberto Stagi (Ratel) argues that TypeScript is overtaking Python as the dominant language for building AI agents and applications, driven by the rise of coding agents like Lovable, Claude Code, and Cursor that default to TypeScript. He notes that Python still rules training, research, and GPU serving, but the application layer—where AI is shipped inside products—now belongs to TypeScript. Stagi highlights that TypeScript surpassed Python as the most used language on GitHub in August 2025, with GitHub attributing the shift to AI agents. He cites the Vercel AI SDK growing from 1.6M to 15.1M weekly downloads in one year and npm's rich ecosystem of authentication, payments, and UI packages as reasons to build agents in TypeScript. Consistent end-to-end typing via Zod and a unified codebase for agent loops, tools, backends, and UIs further strengthen the case. Stagi concludes by invoking Atwood's Law: any application that can be written in JavaScript will eventually be written in TypeScript, urging developers to build agents in TypeScript to avoid falling behind.

Claws Out: Securing and Building with OpenClaw - Nick Taylor, Pomerium

Jul 11, 2026 · 17:12

Nick Taylor, a dev advocate at Pomerium, explains how he contributed trusted-proxy auth mode to OpenClaw, allowing a trusted identity-aware proxy to handle WebSocket authentication and removing the need for tokens and device pairing. He demonstrates building tools like Clawspace, a browser-based file explorer and editor for his OpenClaw workspace, and an MCP server for ChatGPT, all secured with Pomerium's open-core Identity-Aware Proxy. In a live demo, he creates an MCP server by editing files in his OpenClaw workspace via Discord, with changes reflected instantly through Vite hot reload. Taylor argues that hardening OpenClaw access with a trusted proxy improves both security and user experience, showcasing his workflow of building software entirely from his phone.

Develop at Idea Velocity - Jeffrey Lee-Chan, Snapchat

Jul 11, 2026 · 15:28

Jeffrey Lee-Chan (Snapchat) and Austin (CMUX) explain how to achieve 'idea velocity' by building parallel multi-agent harnesses that let one engineer direct 10–20 coding agents instead of becoming the bottleneck. Lee-Chan argues the key is separating 'Agent Orchestrator Managers' from specialized workers to prevent low-level implementation bias, using OpenClaw for frictionless Slack-based communication and CMUX terminals for real-time parallelization and unbiased manager oversight. He demonstrates two apps—an AI RPG with dice-roll mechanics and a multi-AI analysis tool—built with this stack, and discusses token-burn tradeoffs between models like Codex 5.3, GPT-5.4, and Minimax. The episode also covers staging environments to avoid doubling token usage, the benefit of persistent memory and contextual guardrails, and the importance of pushing human interaction to the start or end of the workflow for improved parallelization.

Design Patterns for AI Trust: Juries, Libraries, and Agent Tiers — Alex Bauer, Upside.tech

Jul 11, 2026 · 17:09

Alex Bauer, co-founder of Upside, argues that go-to-market teams can solve AI's trust problem by managing agents like humans, using patterns such as a librarian for just-in-time knowledge and a jury-and-judge model for subjective decisions. He demonstrates three solutions: first, scaffolding an AI website rebuild with anchor assets like a product capabilities reference that ladders directly into homepage copy; second, a librarian that consults documentation and prior failed queries before answering questions like "How much pipeline in Q1?" to catch fiscal-year or stage definitions; third, a jury-and-judge workflow for multi-touch attribution where independent analysts research deals and a judge weighs reasoning quality before producing a consensus verdict. Bauer warns against low-intelligence models for important work, citing Slackbot's MCP integration as "horrifically stupid," and advocates using tier-two platforms with sub-agents, plan mode, and full MCP support.

MCP Apps: Primitives, discovery, and the Future of Software - Pietro Zullo, Manufact, Inc

Jul 5, 2026 · 28:54

Pietro Zullo, co-founder of Manufact, explains that MCP Apps are MCP servers that return interactive UI elements in sandboxed iframes, enabling bidirectional communication between the UI and the host. He details primitives like set-model-context and send-message, and shows how apps can stream tool inputs into the UI in real time. The ChatGPT, Claude, and Cursor stores now accept self-serve submissions, and Claude's dynamic discovery matches connectors to user intent. Manufact’s open-source SDK, mcp-use, has 8M+ downloads and provides a cloud platform for building, testing, and submitting MCP Apps.

The Prompt Is Still a Punch Card - Ted Johnson, JoinIn AI

Jul 2, 2026 · 20:13

Ted Johnson argues that AI interfaces still use the batch protocol of punch cards, forcing humans to adapt to machines rather than the reverse. He introduces three concepts—channel, expression, protocol—to show how expression exploded with LLMs but the protocol remained static. Examples include voice mode misinterpreting a side conversation, PersonaPlex's turn-taking, and a meeting AI that follows group dynamics without prompts. Johnson calls for designing interfaces where AI understands timing, ambiguity, and shared context, moving from prompting to genuine communication.

The Future Is Domain-Specific Agents - Justin Schroeder, StandardAgents

Jun 29, 2026 · 30:38

Justin Schroeder of StandardAgents argues that domain-specific agents—small, focused AI agents each limited to a single domain—will outperform general-purpose agents that rely on accumulating tools and context via inheritance, and that composition of such agents is the key to building practical, cost-effective AI systems. He defines an agent as deterministic software harnessing non-deterministic model outputs, then critiques the current approach of piling tools and skills into a single agent (inheritance), proposing instead a coordinator agent that delegates to many specialized sub-agents (composition), each with its own minimal context, system prompt, and tools. He claims these domain-specific agents achieve over 80% token efficiency, enable use of cheaper small models (e.g., DeepSeek V4 Flash is 137x cheaper than a frontier model), improve safety by limiting capabilities, and scale easily. Schroeder predicts rapid adoption through 2026-2027, with multi-agent orchestration becoming mainstream and tokens no longer getting cheaper—up 29% when adjusted for IQ. He describes an ideal agent architecture with hooks, rules, and recursive sub-agents, and invites listeners to try…

Voice In, Visuals Out: The Agony and the Ecstasy - Allen Pike, Forestwalk Labs

Jun 28, 2026 · 13:05

Allen Pike, co-founder of Forestwalk Labs, argues that voice is the preferred input for AI but visuals are the preferred output, as Andrej Karpathy recently claimed. He explains that while voice-in, voice-out requires sub-200ms latency for seamless conversation, voice-in, visuals-out can work within a more forgiving one-second response envelope. Pike shares three techniques his team uses to achieve this: using a fast model like Haiku with low p95 latency, sending inference every one to two seconds as the user speaks rather than waiting for silence, and leveraging stable prefix caching to keep the first 90% of the context identical, reducing cost and latency. He illustrates with a real example where a voice agent filed a Linear issue in under a second during a team call, demonstrating how the approach creates a natural, non-interruptive experience.

HTML is All You Need (for Agents to Make Graphics) - Amol Kapoor, Nori

Jun 28, 2026 · 7:00

Amol Kapoor, CEO of Nori Agentic, argues that coding agents can create high-quality visual artifacts like slides, docs, and videos by using HTML instead of traditional design tools. He explains that agents think in language and structure, not pixels, so tools like PowerPoint or Figma force them into a human-centric approach that fails. By giving agents HTML—a language they understand intuitively—they can produce properly aligned, themed graphics without manually placing coordinates. Kapoor demonstrates this with the 'pelican riding a bicycle' test, where models fail at raw SVG but succeed with HTML. He applies this method at Nori to build board decks, sales decks, and even animated videos purely with HTML and CSS. The episode advocates for 'thinking like the model' and concludes that for graphics, HTML is all you need.

Turn 10,994 Notes Into Memory - Paul Iusztin, Decoding AI & Louis-François Bouchard, Towards AI

Jun 26, 2026 · 39:32

Paul Iusztin and Louis-François Bouchard introduce an AI Research OS that transforms a personal Second Brain (e.g., an Obsidian vault of 5,000+ notes growing by 250 files/month) from a passive archive into a live, queryable memory for AI agents. They argue that bigger context windows aren’t enough; proper memory and context management are needed. The system evolved through three iterations: first a simple deep research loop generating static research Markdown files (used for 35 course lessons), then targeting both the public web and the user’s Second Brain, and finally adding a persistent wiki layer. The architecture uses plain files (Markdown, YAML index) instead of vector databases, with a hierarchy of raw sources, source summaries, and wiki derivatives (concepts, comparisons, entities). The wiki evolves with every question, not just during ingestion. Demos show ingesting Obsidian notes, GitHub repos, and arbitrary links, then querying the wiki to update it dynamically.

Your Agent's Biggest Lie: "I Searched the Web" — Rafael Levi, Bright Data

Jun 17, 2026 · 15:49

Rafael Levi from Bright Data argues that LLMs often hallucinate and provide fake citations because they fail to actually access live web data, getting blocked by anti-bot systems like CAPTCHAs and Cloudflare's AI labyrinth. He demonstrates this with a comparison: without Bright Data's MCP, GPT-5 failed all five tasks accessing sites like LinkedIn and Amazon; with the MCP's 66 tools—including a CAPTCHA-solving browser that mimics human behavior—four succeeded. Levi explains that agents enter an invisible failure loop where they get blocked or served fake data but still answer confidently, making up numbers or non-existent URLs. He emphasizes that 20% of the web is blocked by Cloudflare from AI crawling, and that fake data fed to bots increases hallucinations. The episode covers how Bright Data's MCP provides real web access with search, scraping, and remote browsers, offering a free tier of 5,000 requests per month for experimentation.

Why MCP and ChatGPT Apps Use Double Iframes — Frédéric Barthelet, Alpic

Jun 15, 2026 · 20:11

Frédéric Barthelet, CTO of Alpic, explains why ChatGPT and other MCP hosts render third-party app UI inside a double iframe. He traces how simpler approaches fail: `srcdoc` shares the parent origin, letting CSP block scripts and risking data access; sandboxing removes origin storage; and `allow-same-origin` recreates the escape. The resulting double iframe—an outer iframe from a controlled subdomain loading app HTML via `srcdoc` into an inner frame—ensures isolation and prevents cross-app storage collisions. Barthelet warns developers must declare every external domain their view uses in MCP app metadata or face submission rejection, and demos Skybridge's CSP inspector that diffs declared domains against actual network calls.

Your Attention Is the Bottleneck, Not Your Agents — Zack Proser, WorkOS

Jun 11, 2026 · 25:17

Zack Proser from WorkOS argues that human attention, not agent speed, is the real bottleneck in AI-assisted coding. He proposes a sustainable stack: signal layers to filter Slack and Linear, voice-first flows at 184 wpm, remote control of agents from a phone to leverage diffuse thinking, and weekly self-improvement passes over JSONL conversation history. He also integrates an Oura ring via MCP so Claude can nudge him about sleep, emphasizing balance over burnout.

Why Eval++ Is the Next Great Compute Primitive — Sunil Pai & Matt Carey, Cloudflare

Jun 8, 2026 · 24:51

Matt Carey and Sunil Pai from Cloudflare's agents team argue that Durable Objects — stateful serverless with 15ms London latency — and Dynamic Workers — a safe, sandboxed eval for LLM-generated code — form the next compute primitive for AI agents. They explain how Durable Objects enable resumable streaming, multi-tab sync, and background scheduling out of the box without distributed systems engineering. Dynamic Workers allow running generated JavaScript strings in isolated sandboxes, reclaiming 30 years of avoided eval. The pair tease upcoming talks: one on collapsing Cloudflare's 2,600 API endpoints into a 1,000-token MCP tool, and another on a coding agent harness built entirely on Workers that they are already shipping.

Dark Factory: OpenClaw Ships Faster Than You Can Read the Diff — Vincent Koc, OpenClaw

Jun 5, 2026 · 16:44

Vincent Koc, a core maintainer at OpenClaw, explains how the open-source project ships code faster than humans can read diffs by running 60–70 autonomous coding agents across swim lanes. In a single night, he and Peter Steinberger executed 2,700 commits touching 82% of the core codebase, launching a plugin architecture and changing close to a million lines of code. Koc describes managing agents as managing people: the key skill is reading reasoning tokens to detect when an agent is bullshitting, gained through sheer volume of token maxing. He argues 2025 was about token maxing; 2026 is about not wasting them, with agent-in-the-loop processes and opinionated swim lanes replacing blind Ralph looping. The episode details his agent development environment using .skills files, Git work trees, and a semantic graph for triaging 60K PRs, emphasizing that the bottleneck shifts from engineering to taste and process.

Beyond Components: Designing Generative UI for MCP Apps — Ruben Casas, Postman

Jun 3, 2026 · 16:58

Ruben Casas from Postman argues that AI models can now write better frontend code than he can—his prompt to rewrite his blog produced a search box with blur animation and accessibility out of the box—yet most agent UIs still invoke static prebuilt components. He presents three levels of UI generation: static components (AG UI, Goose) passing props to predefined React elements; declarative UI where the model generates JSON or YAML for a rendering engine (e.g., Vercel's JSON Render), which he deems the current ideal balance; and fully generative UI where the model writes HTML, CSS, and JavaScript on demand, as in his weather agent that does so in one tool call. The key obstacle is trust, necessitating sandboxing, and MCP apps with their double iframe default are the best delivery mechanism. He likens today to early TV—radio shows with cameras—and predicts the future lies beyond components in collaborative human-agent interfaces on shared canvases, as seen with the Skeletro MCP app.

Reverse engineering a Viking VOIP phone protocol with Claude Code — Boris Starkov, Eleven Labs

May 29, 2026 · 20:11

Boris Starkov from ElevenLabs used Claude Code to reverse engineer the undocumented protocol of a Viking VOIP phone that three senior engineers and ChatGPT failed to configure in a year. Claude brute-forced all 676 possible two-letter command combinations, found 80 valid ones, then set up a TCP proxy between a Windows XP virtual machine and the phone to intercept traffic from the proprietary software. The critical missing piece—a one-byte checksum in the persistence command—was deduced by Claude by running known input-output pairs through the derived formula, confirming a simple subtraction. Starkov acted as Claude's hands, physically rebooting the phone and reporting beeps. The protocol is now open-sourced as a Claude Code skill, enabling direct programming without Windows software. The project culminated in a red phone booth at AI Engineer Europe where picking up the receiver connects to a Michael Caine voice agent quizzing attendees on British AI history.

Why Rust is the Ideal Language for Vibe-Coding — Daniel Szoke, Sentry

May 27, 2026 · 16:25

Daniel Szoke, Sentry's Rust SDK maintainer, argues Rust is ideal for vibe-coding: its compiler enforces correctness, catching bugs that Python and TypeScript allow to compile and run. He challenges the preference for easy-to-write languages, warning that dynamic typing and null values invite subtle bugs. Szoke highlights Rust's type safety, null safety, and fearless concurrency, demonstrating with a 100-thread counter where TypeScript's data race compiles and fails intermittently while Rust rejects it with an error naming the non-Send type. He argues every compile error an agent fixes is a production bug avoided, and the Rust compiler is faster and more reliable than a review agent.

Run Frontier AI at Home — Alex Cheema, EXO Labs

May 26, 2026 · 1:45:02

Alex Cheema of EXO Labs argues that running frontier AI locally has 100x improvement potential in cost and performance, demonstrated with GLM 5.1—a trillion-parameter model—running across four Mac Studios at roughly 20 tokens per second for $40,000. He details kernel fusion that recovered 30% performance on Qwen 3.5 by eliminating unnecessary kernel launches, and RDMA integration that cut node-to-node latency from 300 microseconds to single digits, enabling tensor parallelism to actually scale. Cheema advocates splitting inference: prefill on compute-dense hardware (e.g., an RTX Spark) and decode on high-bandwidth hardware (e.g., Mac), cutting large-prompt inference roughly in half. He warns against misleading benchmarks like one-bit quantized models, and outlines how multi-agent setups, test-time scaling, and continual learning could further improve local inference efficiency. The talk includes a live demo of GLM 5.1 across four Mac Studios connected via Thunderbolt 5, and a preview of EXO's upcoming benchmarking site to track intelligence per Joule.

Scaling Agents on Kubernetes with acpx and ACP — Onur Solmaz, OpenClaw

May 21, 2026 · 19:00

Onur Solmaz, a founding engineer at textcortex and OpenClaw maintainer, built acpx, a headless CLI for the Agent Client Protocol (ACP), to automate processing of 300–500 daily AI-generated pull requests on OpenClaw. He replaces manual PTY scraping with structured agent-to-client communication, driving Codex sessions through a node-based workflow: reproduce the bug, judge the implementation, check for conflicts, run a review loop, and emit structured JSON. Solmaz runs parallel Codex sessions from five Discord channels while traveling, one channel per task. He also presents an open-source Kubernetes operator called Spritz (from textcortex) that provisions disposable full-compute pods per task, wires them into Slack, and tears them down when work is done, advocating for on-demand disposable agents over persistent chat apps.

How Building with AI Can Double the Throughput of Your Engineering Team — Brian Scanlan, Intercom

May 15, 2026 · 21:49

Brian Scanlan of Intercom explains how treating Claude Code like a new hire—onboarding it to a 15-year-old Rails monolith, writing skills for every recurring task, and connecting it to production systems—doubled engineering PR throughput in under a year. By going all in on one platform instead of letting everyone choose their own tool, Intercom achieved a 17.6% automatic PR approval rate with SOC 2 sign-off, and its CI infrastructure collapsed under the volume. The key principle: give agents problems, not tasks. Scanlan recounts a security incident where Claude automatically pulled files, ran analysis, and handed back next steps in two minutes using a skill he didn't know existed. The talk emphasizes that all engineering work—debugging, testing, planning—should be agent-first, and that companies must invest in platform-level adoption and continuous skill improvement.

Lessons from Trillion Token Deployments at Fortune 500s — Alessandro Cappelli, Adaptive ML

May 12, 2026 · 18:35

Alessandro Cappelli, co-founder of Adaptive ML, argues that 95% of GenAI pilots fail to reach production because they rely on proprietary models or instruction fine-tuning, which lack systematic feedback integration. Reinforcement learning (RL) is the only post-training technique that mathematically incorporates defects, business metrics, and production signals to continuously improve models. RL enables smaller, cheaper, faster models that enterprises can own—critical for scaling use cases like AT&T’s transcript summarization or Manulife’s agents. For agents, RL naturally fits because it was designed for environments; synthetic data is generated as a byproduct of environment training, not a prerequisite. Reward signals come from business KPIs (e.g., containment rate) or LLM judges defined by human rubrics in hours, not weeks. Adaptive ML’s Adaptive Engine abstracts away RL complexity (orchestrating four models for PPO) and provides pre-built recipes to industrialize model deployment.

MCP UI: Extending the frontier — Liad Yosef and Ido Salomon, MCP Apps

May 6, 2026 · 22:21

In this episode, Ido Salomon and Liad Yosef explain MCP Apps, an MCP extension that lets tools send interactive, branded UI instead of plain text responses inside chat hosts like ChatGPT, Claude, VS Code, Cursor, and Copilot. They argue that text-based chat reduces companies to a wall of text, erasing identity, while MCP Apps preserves branding by returning HTML resources that hosts render as secure, interactive components. The architecture ensures every user click sends a message back to the host (not directly to the backend), keeping interactions in context—as demonstrated with a PostHog funnel analysis in Claude. Adoption includes Shopify, Hugging Face, GitHub, ChatGPT, and Claude, with ChatGPT recommending MCP Apps for building ChatGPT Apps. They note 800 million weekly ChatGPT users (10% of world population), calling this a once-in-20-years opportunity to rethink app distribution. Future work covers reusable views, model-UI interaction, and interoperability with generative UI protocols like A2UI and WebMCP, aiming to standardize UI in chat as the new web.

Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs

May 4, 2026 · 1:21:26

Angelos Perivolaropoulos from ElevenLabs walks through building a small GPT-2-like LLM from scratch on a local machine, demonstrating that the core techniques used by major labs are accessible in a few hundred lines of PyTorch code. The workshop uses character-level tokenization (65 tokens) on a Shakespeare dataset to enable fast training with limited compute. The model architecture includes multi-head self-attention, MLP layers, residual connections, and layer normalization, totaling 10 million parameters across six transformer blocks with a 256-token context window. The training loop employs next-token prediction with a warm-up cosine decay learning rate schedule and validation loss to detect overfitting. Inference uses temperature sampling (default 0.7) and top-k sampling to improve creativity. Perivolaropoulos explains that audio and multimodal models share the same transformer foundation but differ in tokenization (e.g., mel-spectrograms for audio) and use specialized losses like L2 or KL divergence, while reasoning models result from post-training base models with high-quality chain-of-thought data.

Human-in-the-Loop Automation with n8n — Liam McGarrigle

May 2, 2026 · 1:19:22

Liam McGarrigle of n8n shows how to build a secure human-in-the-loop automation agent using n8n's visual workflow system, with a Gmail and Google Calendar management agent as the concrete example. He walks through wiring a chat trigger, an AI agent with simple memory, and tools that the agent can call, such as sending emails and creating calendar events. The key addition is a human review node placed between the agent and destructive tools, which intercepts actions and presents them for approval via chat—preventing accidental sends or event creation. McGarrigle emphasizes naming nodes and writing tool descriptions to guide the LLM correctly, and demonstrates using expressions to format readable approval messages. He also covers extending the agent to Slack, adding scheduled runs for autonomous hourly inbox checks, and using sub-agents for specialized tasks. Additional topics include n8n's native MCP server for integration with Claude Code, enterprise Git-based environments for workflow management, and building custom REST APIs within n8n. The session focuses on giving developers observability and control over AI workflows, ensuring agents are not black boxes.

Software Engineering Is Becoming Plan and Review — Louis Knight-Webb, Vibe Kanban

May 2, 2026 · 20:23

Louis Knight-Webb, founder of Vibe Kanban, argues that software engineering is becoming all about planning and reviewing AI-generated code, not writing it. He presents two approaches: plan-heavy (spending time on specs to minimize review) and review-heavy (letting agents run and correcting output), noting five minutes of planning saves thirty minutes of review. As agents run longer—crossing the five-minute threshold—engineers must parallelize multiple agents and manage workflows, which Vibe Kanban aimed to facilitate. Knight-Webb announces shutting down Vibe Kanban despite 30,000 monthly active users because it resold tokens to enterprises and struggled to make money, advising that enterprise sales are crucial. He reflects on the journey positively, emphasizing learning to work with great people and the importance of hard work.

OpenAI Codex Masterclass — Vaibhav Srivastav & Katia Gil Guzman

Apr 29, 2026 · 1:01:59

Katia Gil Guzman and Vaibhav Srivastav showcase OpenAI's Codex as a full software engineering agent, not just a coding assistant, powered by models like GPT-5.4 and GPT-5.3 Codex Spark. They demonstrate plugins bundling skills and MCP servers, automations that run scheduled tasks (e.g., Slack or Gmail summaries), and code review that 100% of OpenAI pull requests use by default. Subagents parallelize work across up to 20 agents for tasks like reviewing 45 persona files, with each subagent configurable by model, sandbox mode, and MCP access. New features include Guardian approvals for privileged operations, hooks for custom triggers, and a Cloud Code plugin. The episode also highlights the milestone of 3 million weekly active users, tripled since January, and covers security scanning for vulnerabilities.

Full Walkthrough: Workflow for AI Coding — Matt Pocock

Apr 24, 2026 · 1:36:30

Matt Pocock presents a hands-on workshop on building a full AI-assisted coding workflow, arguing that software engineering fundamentals—not hype—make agents effective. He introduces the 'smart zone' and 'dumb zone' of LLMs (performance drops after ~100k tokens) and the 'Memento problem' (agents forget between sessions). His process starts with a 'Grill Me' skill that relentlessly questions the user until shared understanding is reached, then produces a PRD without reading it, slices work into vertical 'tracer bullet' issues, and runs agents AFK using TDD. He advocates designing codebases with deep, testable modules and shows Sandcastle, a TypeScript library for parallel agent execution with separate implementer (Sonnet) and reviewer (Opus). The workshop transforms ambiguous briefs into shippable features while keeping humans in the loop for QA and taste.

The End of Apps — Kitze, Sizzy.co

Apr 23, 2026 · 20:18

Kitze, creator of Benji and founder of Tinker Club, argues that current productivity apps and AI agents are unreliable and too complex for most users, predicting a future where AI prompts humans instead of the reverse. He traces his 24-year obsession with productivity tools—from a childhood to-do list to building Benji with 60 features—and his move to self-hosting after the ChatGPT moment. Kitze critiques custom agents like OpenClaw as too tinkerer-focused, cloud agents like ChatGPT as too nerfed, and believes local models on devices (Apple, Google Pixel) will win for normies, enabling AI to automatically manage notifications, emails, and tasks. He shares his own experiment, Wolfer, designed for predictable multi-agent orchestration without memory systems, using nested topics for context. The episode delivers a strong claim: the role of AI will invert, with machines prompting humans for decisions while handling all background work.

Full Workshop: Build Your Own Deep Research Agents - Louis-François Bouchard, Paul Iusztin, Samridhi

Apr 20, 2026 · 1:57:03

Louis-François Bouchard, Samridhi Vaid, and Paul Iusztin built a combined system of a deep research agent and a deterministic writing workflow to automate high-quality technical content creation, trading flexibility for control. The research agent uses an MCP server (FastMCP) with tools for Google search, YouTube analysis, and report compilation, powered by Gemini and orchestrated by Claude Code. The writing workflow is a static pipeline that loads a user guideline, writing profiles (structure, terminology, character), and few-shot examples into a system prompt, then applies an evaluator-optimizer loop: a reviewer outputs structured pydantic objects with profile/location/comment to flag violations, and the editor applies them in priority order (guideline > research > profile). They stress that research demands autonomy while writing needs constraint, and show how to avoid AI slop by banning specific words and using profiles. For observability, they use Opik to capture all traces, and for evals they built a 20-sample dataset from real LinkedIn posts, split into train/dev/test, to calibrate an LLM judge (binary pass/fail) that they run on dev and test splits to measure F1 and detect…

The Future of MCP — David Soria Parra, Anthropic

Apr 19, 2026 · 18:46

David Soria Parra from Anthropic argues that MCP (Model Context Protocol) is the key to connecting agents to tools and data in production, with 110 million monthly downloads—outpacing React's growth at the same stage. He lays out a 2026 connectivity stack combining Skills, MCP, and CLI/Computer Use, each suited for different needs, and emphasizes that best agents will use all three seamlessly. To improve client harnesses, he introduces Progressive Discovery—deferring tool loading via Tool Search to reduce context usage—and Programmatic Tool Calling, where models write scripts to compose tool outputs efficiently. Upcoming MCP protocol improvements include stateless transport (with Google) for easier scaling, async agent-to-agent tasks, enterprise features like Cross App Access and Server Discovery via well-known URLs, and a Skills-over-MCP extension for shipping usage instructions with servers. He calls for community feedback on these directions.

Your MCP Server is Bad (and you should feel bad) - Jeremiah Lowin, Prefect

Jan 12, 2026 · 54:33

Jeremiah Lowin, CEO of Prefect and creator of FastMCP, argues that most MCP servers are poorly designed because they are simply REST API wrappers, leading to confused LLMs and wasted tokens. He presents a framework for agent-native product design based on three dimensions—discovery, iteration, and context—and offers five actionable principles: outcomes over operations, flatten arguments, treat instructions as context, respect the token budget, and curate ruthlessly. Lowin emphasizes that agents fail when given too many tools or complex arguments, citing a hard limit of 50 tools per agent for reliable performance. He shares real-world examples, including a company with 800 endpoints that could not be effectively exposed, and critiques client implementations like Claude Desktop for not respecting spec-compliant features. The central takeaway: MCP servers should be designed as user interfaces for agents, not as developer APIs.

Jack Morris: Stuffing Context is not Memory, Updating Weights is

Dec 29, 2025 · 1:02:44

Jack Morris argues that large language models fail at niche, long-tail knowledge tasks, such as optimizing AMD GPU kernels or answering private company queries, because they rely on context windows and RAG, which suffer from quadratic self-attention costs and context rot. He advocates for a third paradigm—training knowledge directly into model weights—using synthetic data generation (e.g., synthetic continued pretraining from Stanford) to expand small datasets and parameter-efficient methods like LoRA or memory layers to avoid catastrophic forgetting. Morris demonstrates that full fine-tuning on a 3M 10-K report causes the model to only regurgitate exact sentences, whereas generating diverse synthetic question-answer pairs enables better generalization. He notes that RL-based fine-tuning (e.g., GRPO) can achieve improvements with as few as 14 parameters, while memory layers offer the best trade-off between learning and forgetting. The episode also explores temporal information handling, federated learning resurgence, and the practical decision boundary between RAG and weight-based injection based on data freshness and volume.

AGI: The Path Forward – Jason Warner & Eiso Kant, Poolside

Dec 27, 2025 · 15:56

Jason Warner and Eiso Kant, co-founders of Poolside, present their vision and roadmap towards AGI-level capabilities for knowledge work, demonstrating their second-generation model Malibu Agent converting ADA code to Rust live on stage. They argue that next-token prediction paired with reinforcement learning is the key breakthrough, a contrarian bet they made two and a half years ago. The episode centers on their work in high-consequence code environments for defense and government, where agents must operate with tight permissions. They announce a large compute cluster of over 40,000 GB300s coming online and a public API release early next year via AWS Bedrock. Warner recounts meeting Kant through a failed GitHub acquisition, and Kant invites the audience to build with their models, emphasizing that future agents will handle tasks over days as intelligence scales.

Shipping AI That Works: An Evaluation Framework for PMs – Aman Khan, Arize

Dec 26, 2025 · 1:26:16

Aman Khan, AI PM at Arize, presents a framework for product managers to evaluate LLM-powered products beyond gut-feel 'vibe checks.' He demonstrates building an AI trip planner with multi-agent LangGraph, then using Arize's tracing and prompt playground to iterate on prompts. Khan shows how to create datasets from production traces, run A/B experiments on prompts, and use LLM-as-a-judge evals for friendliness and discount offers, comparing against human labels to refine evaluators. He argues evals are the new requirements docs, enabling PMs to own the product experience by writing acceptance criteria as eval datasets. The talk covers building eval teams, handling variance with temperature settings, and continuously improving golden datasets with hard examples, citing real-world analogies from self-driving cars at Cruise.

Small Bets, Big Impact Building GenBI at a Fortune 100 – Asaf Bord, Northwestern Mutual

Dec 23, 2025 · 22:50

Asaf Bord, AI Product Lead at Northwestern Mutual, shares how his team built GenBI, an LLM-powered analytics copilot, by flipping the logic from a single big bet to an incremental roadmap of small, fundable projects. Using real, messy data from the 160-year-old company, they deployed a modular architecture with metadata, RAG, SQL, and BI agents, each productizable independently. The RAG agent alone automated 80% of the 20% of BI team capacity spent on finding and sharing reports, saving roughly two full-time employees. Bord explains how a crawl-walk-run release strategy built trust with both users and leadership, starting with BI experts before expanding to business managers, and how each six-week sprint delivered tangible business value—like proving the ROI of enriched metadata through A/B tests against a semantic layer initiative. He also explores the future of SaaS pricing in the GenAI era, questioning whether per-seat models still make sense when individuals become 10x more effective.

From Vibe Coding To Vibe Engineering – Kitze, Sizzy

Dec 14, 2025 · 25:28

Kitze, founder of Sizzy, argues that "vibe engineering"—actively steering AI agents with technical knowledge—trumps passive "vibe coding" for building real software. He shares how Cursor's Composer 1 let him port Benji.so and Glink to Next.js 16 with Monorepo in under a week, reviving near-dead projects. Kitze insists LLMs excel at React because humans are bad at it, and that repetitive code is fine since agents don't care. He warns against giving AI tools to juniors without oversight, and predicts the bottom of the job market will thin as agents replace interns. The talk ends with a pitch for "vibe code fixers" as a new role, maintaining legacy systems like cowboy coders.

Enterprise Deep Research: The Next Killer App for Enterprise AI — Ofer Mendelevitch, Vectara

Nov 24, 2025 · 5:19

Ofer Mendelevitch from Vectara introduces Enterprise Deep Research as the next killer app for enterprise AI, applying autonomous, multi-step reasoning to internal knowledge bases. Vectara's trustworthy agent OS enables this with multimodal ingest, hybrid retrieval, and hallucination mitigation (HHEM model at 5.5M downloads). Deep Research queries private data to generate comprehensive reports with citations, replacing manual workflows. Key use cases include responding to RFPs by scanning enterprise datasets, generating on-demand employee onboarding guides from Jira/Notion/SharePoint, and creating investment memos in financial services. The system uses multi-agent parallel execution and corpus understanding for accurate planning, addressing 73% of LLM users' top challenge: factual accuracy.

AI changes *Nothing* — Dax Raad, OpenCode

Nov 23, 2025 · 16:39

Dax Raad, core contributor to SST and creator of OpenCode, argues that AI changes nothing when building a winning product. In this contrarian talk, he breaks down the three unchanging fundamentals: marketing requires creating things so cool people share them, which AI cannot do because it is too corny; the aha moment demands ruthlessly eliminating friction to get users to that singular moment of understanding, requiring deep taste not algorithmic optimization; retention hinges on building deep primitives first to support power users without sacrificing simplicity for newcomers, a design challenge AI cannot hallucinate. Raad uses examples from ChatGPT's instant aha moment—any input box that responds like a human—and Facebook's poke as a deniable flirting aha moment. He concludes that AI amplifies capabilities but does not save anyone from the hard, creative work of product success.

#define AI Engineer - Greg Brockman, OpenAI (ft. Jensen Huang)

Aug 10, 2025 · 41:05

Greg Brockman, co-founder and president of OpenAI, discusses his journey from Stripe's first CTO (growing from 40 to 250 people) to leading AI development, sharing insights on independent study (completing three years of high school math in one year), the engineering-research partnership at OpenAI (emphasizing technical humility), and scaling challenges like ChatGPT hitting 1 million users in five days and ImageGen reaching 100 million in five days. He explains how Codex transforms coding (low double-digit percent of internal PRs written by Codex, 24,000 PRs merged in public GitHub in one day) and introduces "vibe coding" as an empowerment mechanism. In response to Jensen Huang's questions, Brockman predicts a future with diverse AI infrastructure (homogeneous accelerators as default, purpose-built for specific workloads) and a menagerie of domain-specific agents leveraging distillation, driving 10x more economic activity. He identifies basic research as a renewed scaling bottleneck alongside compute, data, algorithms, power, and money.

Designing AI-Intensive Applications - swyx

Aug 9, 2025 · 13:02

In this conference talk, swyx (host) proposes the search for a 'Standard Model' in AI engineering, analogous to physics' Standard Model, to guide practitioners. He examines candidate models: LMOS (LLM Operating System), LMSDLC (AI Software Development Lifecycle), and Anthropic's 'building effective agents' framework. Swyx introduces his own SPADE model (Sync, Plan, Analyze, Deliver, Evaluate), derived from building AI News, a daily tool that scrapes Discord, Reddit, and Twitter with thousands of LLM calls. He argues that the ratio of human input to AI output—ranging from 1:1 in ChatGPT to 0:n in ambient agents—is more useful than debating 'workflow versus agent.' The talk emphasizes that early SDLC stages (LLMs, monitoring, RAG) are becoming commodity, while real value comes from evals, security orchestration, and hard engineering. Swyx encourages attendees to identify and refine their own Standard Models to build products people want.

Evals Are Not Unit Tests — Ido Pesok, Vercel v0

Aug 6, 2025 · 15:22

Ido Pesok, an engineer at Vercel working on v0, argues that evals—not unit tests—are the key to making LLM applications reliable, using the basketball court analogy of understanding your domain (the court), collecting user queries (data points), and scoring passes/fails. He illustrates with the Fruit Letter Counter app, which failed in production despite passing manual tests, because LLMs are inherently non-deterministic. Pesok advises collecting data from thumbs-up/down, logs, and forums, and structuring evals with constants in data and variables in task (e.g., system prompt). He emphasizes deterministic pass/fail scoring, adding extra prompt tags for easy parsing, and integrating evals into CI/CD to detect regressions when changing models or prompts. The episode stresses that improvement without measurement is limited, and evals enable systematic reliability gains.

2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI

Aug 6, 2025 · 19:14

John Dickerson, CEO of Mozilla AI and former co-founder of Arthur AI, argues that 2025 is finally the year for AI evaluation (evals) due to three converging forces: ChatGPT making AI tangible to the C-suite, enterprise budget freezes that funnelled funding into GenAI pet projects, and the rise of agentic systems acting autonomously. He traces how ML monitoring existed pre-2022 but lacked C-suite attention, with Jamie Dimon’s JPMC spending only $100 million on AI from 2017 to 2021. The ChatGPT launch on November 30, 2022, unlocked CEO discretionary budgets for GenAI, leading to 2023 science projects, 2024 production deployments, and 2025 scaling. Now, as agents reason and act, connecting evals to downstream business KPIs—risk mitigation, revenue gains—is a first-class discussion for CEOs, CFOs, CISOs, and CTOs. Dickerson notes that evaluation companies have shifted to multi-agent system monitoring, and while LLM-as-judge is popular, biases require human validation, with companies like Mercor hiring experts at $50–200/hour to lockstep multi-agent outputs.

Vibe Coding with Confidence — Itamar Friedman, Qodo

Aug 6, 2025 · 20:55

Itamar Friedman, CEO of Qodo, argues that the command-line interface (CLI) is the future for achieving 'vibe coding with confidence' in enterprise software development. He contrasts vibe coding's greenfield success with the challenges of large codebases, where context, workflows, and multi-agent collaboration are essential. Friedman explains Qodo's multi-agent architecture, including tools for code review (Qodo Merge) and coverage (Qodo Cover), and announces the soft launch of a new CLI tool that lets developers create and pipe agents, supporting MCP and A2A protocols. He envisions a future 'swarm of agents' each specialized with credentials and best practices, enabling end-to-end workflows across the SDLC. The episode emphasizes moving from chat-based agents to command-driven multi-agent systems for reliable, high-quality code generation.

Vision AI in 2025 — Peter Robicheaux, Roboflow

Aug 3, 2025 · 17:24

Peter Robicheaux, ML lead at Roboflow, argues vision models are not smart because they fail at basic visual tasks like telling time on a watch or identifying a school bus direction. He attributes this to saturated evals like ImageNet and COCO, and to vision models not leveraging large pre-training like language models. Roboflow introduces RFDetter, a real-time detector using a Dynav2 backbone, achieving second SOTA on COCO and large gains on their new RF100VL dataset of 100 diverse domains. The benchmark shows specialist models fine-tuned on 10-shot examples outperform large VLMs like Qwen2.5 VL 72B, indicating VLMs are strong linguistically but weak visually. Roboflow's platform is free for researchers contributing data back; dataset at rf100vl.org.

The 2025 AI Engineering Report — Barr Yaron, Amplify

Aug 1, 2025 · 12:33

Barr Yaron presents early findings from the 2025 State of AI Engineering survey at the AI Engineer World's Fair. The survey reveals that while titles vary widely, the community is broad and technical, with nearly half of seasoned software engineers having worked with AI for three years or less. Over half of respondents use LLMs for both internal and external use cases, with 3 of the top 5 models for customer-facing products from OpenAI. RAG is the most popular customization method (70%), and fine-tuning is surprisingly common, with 40% using LoRA or QLoRA. More than 50% update models monthly, and 70% update prompts at least monthly. A 'multimodal production gap' exists, with image, video, and audio usage lagging text. 80% say LLMs work well, but less than 20% say the same about agents, though most plan to use them. Evaluation is the top pain point.

Why ChatGPT Keeps Interrupting You — Dr. Tom Shapland, LiveKit

Jul 31, 2025 · 27:03

Dr. Tom Shapland of LiveKit explains why voice AI agents like ChatGPT's Advanced Voice Mode keep interrupting users: they rely on a simple VAD that triggers after a silence threshold, unlike humans who predict turn endings using semantics, syntax, and prosody. To solve this, LiveKit developed a semantic end-of-utterance model that considers the last four conversation turns, extending the VAD's silence window when the user is not done. A side-by-side demo shows dramatic reduction in interruptions. Shapland contrasts this with full-duplex models like Moshi and Meta's SyncLLM, which process input and generate speech simultaneously but lack instruction-following. He predicts commercial systems will improve via smarter VAD augmentations and faster cascade pipelines, not full-duplex. The talk also covers handling backchannels (e.g., 'mm-hmm'), the challenge of benchmarking turn-taking, and why OpenAI doesn't yet use LiveKit's model.

How to defend your sites from AI bots — David Mytton, Arcjet

Jul 30, 2025 · 20:12

David Mytton, founder of Arcjet, explains that nearly 50% of web traffic is now automated, with AI crawlers significantly worsening the problem for site owners. He cites specific cases: Diaspora saw 24% of its traffic from OpenAI's GPTbot, and ReadTheDocs cut bandwidth from 800 GB to 200 GB per day by blocking AI crawlers. Mytton describes the evolving landscape of AI bots—OpenAI alone has at least four types (Search, ChatGPT User, GPTbot, Operator)—and argues that detection requires layered defenses. He walks through eight defenses: robots.txt (voluntary), user agent string verification with reverse DNS, IP reputation (e.g., 12% of bot traffic on Cloudflare comes from AWS), CAPTCHAs (increasingly easy for AI to solve), proof-of-work puzzles (e.g., Anubis proxy), experimental HTTP message signatures (Cloudflare's proposal), fingerprinting via JA4 hash (TLS-level), and rate limiting keyed to fingerprints or session IDs. The practical takeaway is that most sites can manage with user agent verification plus IP checks, but high-value or high-traffic sites need full stacking of these techniques.

The Unofficial Guide to Apple’s Private Cloud Compute - Jmo, CONFSEC

Jul 30, 2025 · 20:36

Jonathan Mortensen explains Apple's Private Cloud Compute (PCC), a paradigm shift in confidential computing that delivers cryptographically provable privacy for AI inference on remote servers. He details Apple's five requirements—stateless computation, enforceable guarantees, non-targetability, no privileged runtime access, and verifiable transparency—and six technical components, including oblivious HTTP, blind signatures, secure enclave, remote attestation, transparency logs, and secure boot. The talk highlights how remote attestation allows an iPhone to verify server software and tie encryption to that verified state, while the transparency log enables public auditing of all deployed binaries. Mortensen notes trade-offs: complete trust in Apple's supply chain, higher latency and compute costs, and no support for custom models or third-party developers. He points to alternative tools like TPMs, SigStore, and confidential VMs for non-Apple environments, and concludes that Apple's approach is being adopted by Azure AI and Meta.

OpenAI on Securing Code-Executing AI Agents — Fouad Matin (Codex, Agent Robustness)

Jul 30, 2025 · 14:00

Fouad Matin, an OpenAI engineer working on agent robustness, argues that code-executing agents like Codex CLI require layered security safeguards as they become capable of autonomously writing and running code. He emphasizes three main defenses: sandboxing agents via containerization (e.g., ChatGPT spins up fully isolated containers) or OS-level sandboxing using SeatBelt on macOS or Seccomp and Landlock on Linux; disabling or limiting internet access to prevent prompt injection and data exfiltration, with configurable allowlists for approved endpoints; and requiring human review through PR approvals and monitoring tasks to catch unintended actions like installing malicious packages. Matin also details new API tools—local shell and apply patch—that enable agents to execute code and apply diffs securely, and recommends using remote containers (available in the Agents SDK and Responses API) as the safest deployment option. He concludes by acknowledging that while LLM-based monitors are valuable, deterministic system-level controls remain essential, and invites applications to OpenAI's new Agent Robustness team.

Layering every technique in RAG, one query at a time - David Karam, Pi Labs (fmr. Google Search)

Jul 29, 2025 · 20:22

David Karam of Pi Labs (formerly Google Search) walks through layering every technique in RAG, from in-memory retrieval to planet-scale search with 70+ corpus mix of token, embeddings, and knowledge graphs, jointly retrieved and re-ranked at 160,000 queries per second in under 200msec. He advocates a Quality Engineering Loop: baseline with simplest methods, analyze losses, then apply incremental techniques based on complexity-adjusted impact. Karam explains why queries like "falafel" are notoriously hard due to ambiguous intent, highlights failures of chunking documents, and shows when BM25 suffices, when relevance embeddings are needed, and when custom embeddings or domain-specific signals (price, popularity, user clicks) become essential. He covers query orchestration via fan-out, supplementary retrieval across back ends, and distillation for cost optimization, concluding that at sufficient complexity, problems must be punted to LLM or UX for graceful degradation.

Building a Smarter AI Agent with Neural RAG - Will Bryk, Exa.ai

Jul 29, 2025 · 18:42

Will Bryk, CEO of Exa.ai, argues that traditional keyword search engines like Google are ill-suited for AI agents, which need semantic understanding, complex multi-paragraph queries, and comprehensive results. He explains Exa's neural search approach using embeddings to capture ideas and context, unlike keyword-based systems that miss nuances like negation. Bryk demonstrates a live-coded agent combining neural searches (e.g., finding personal sites of SF engineers who like information retrieval) with keyword searches (e.g., retrieving specific GitHub profiles), showing how Exa's API exposes toggles for date ranges, domains, and result counts. He emphasizes that AI agents require search engines that return exactly what they ask for, not what humans click on, and can handle thousands of results. The episode also previews Exa's new research endpoint for automated deep research.

Shipping something to someone always wins — Kenneth Auchenberg (ex. Stripe, VSCode)

Jul 28, 2025 · 16:17

Kenneth Auchenberg, former VSCode and Stripe developer platform lead, argues that great AI products come from rapid iterative loops — shipping continuously viable 'skateboards' to real users rather than aiming for a perfect final product. He advocates for writing launch blog posts before building, working intimately with a few real customers (even texting them), and ignoring constraints like legal initially to design the best product. APIs are harder to change than UI, so early user feedback is critical. In AI, the fundamentals haven't changed: customer knowledge and iteration velocity matter more than ever, even as AI tools accelerate building. The goal should be to run a product feedback loop in under a day.

Why your product needs an AI product manager, and why it should be you — James Lowe, i.AI

Jul 28, 2025 · 18:37

James Lowe, Head of AI Engineering at the UK Government's Incubator for AI, argues that technical expertise makes AI engineers ideal candidates for the critical role of AI Product Manager, which balances business viability, technological feasibility, user desirability, and the core question of possibility. He shares three hard-won lessons from building products like CONSULT, Minute, and Redbox: resolve AI uncertainties early with evaluations and real-user tests (achieving 1,000x faster and 400x cheaper consultation analysis), experiment widely with features then cut back (streamlining a transcription tool from overwhelming options to a focused Justice Transcribe used for Prime Ministerial meetings), and pivot harder than ever as the landscape shifts (Redbox evolved from digitizing ministerial red boxes to a secure LLM chat client, then to an MCP-based tool provider when Microsoft Copilot Chat went free). The episode emphasizes that AI product management is a mindset, not just a title, and that engineers should step into this leadership gap to build impactful AI products.

Everything is ugly, so go build something that isn't — Raiza Martin, Huxe (ex NotebookLM)

Jul 28, 2025 · 25:15

Raiza Martin, former lead of Google's NotebookLM and founder of Huxe, argues that the current chaotic phase of AI product design is a once-in-a-career opportunity to rebuild from first principles, calling everything we use 'the ugliest that it will ever be.' Drawing from her experience forcing NotebookLM into existence against skepticism, she emphasizes that personal clarity of vision fuels product building, and purpose must be relentlessly focused on a single outcome—for NotebookLM, enabling users to upload 50 files and interact with them. She stresses earning trust by nailing deterministic behavior first, noting that 90% of first queries were summarization and failures drove users away forever, then layering on delightful probabilistic features like podcast generation. Finally, she warns against the 'kitchen sink' approach, citing her own Huxe app that did everything but users only used one feature, advocating restraint as an innovation multiplier and focus on one excellent outcome to avoid building ugly products.

Introduction to LLM serving with SGLang - Philip Kiely and Yineng Zhang, Baseten

Jul 26, 2025 · 43:42

Philip Kiely and Yineng Zhang introduce SGLang, an open-source fast serving framework for LLMs and VLMs, arguing it offers production-ready performance with day-zero support for new models like DeepSeek and Qwen, and a customizable codebase for contributions. Yineng, a core maintainer, traces SGLang's rapid growth from a December 2023 paper to nearly 15,000 GitHub stars and adoption by xAI, AMD, and Meituan. The workshop demonstrates deploying a first model via Baseten's trunk packaging, then tuning the CUDA Graph max batch size flag on an L4 GPU to maintain CUDA Graph-enabled decoding during higher concurrency, boosting generation throughput. They also cover Eagle 3 speculative decoding, where a draft model derived from the target model speculates tokens; users can benchmark different step and top-k configurations on representative prompts to find optimal settings for production. Finally, they invite contributions via GitHub's 'good first issue' tags and highlight Baseten's job openings.

Building AI Products That Actually Work — Ben Hylak (Raindrop), Sid Bendre (Oleve)

Jul 24, 2025 · 18:42

Ben Hylak (Raindrop) and Sid Bendre (Oleve) argue that building reliable AI products requires iterative real-world signals over traditional evals. Ben debunks eval myths—evals don't measure product quality, LLM-as-judge fails, and production evals are costly—and stresses tracking explicit signals (thumbs up/down, copy rate) and implicit signals (refusals, frustration) to identify issues. Sid introduces Trellis, a framework for scaling viral AI apps that uses discretization—breaking infinite output into intent buckets—prioritization by volume times negative sentiment times achievable delta, and recursive refinement. Starting with an MVP, teams classify user intents, convert them into semi-deterministic workflows, then repeatedly drill into sub-intents to engineer repeatable, attributable magic. Oleve's approach, powered by Raindrop, has scaled six viral products to $6M ARR profitably with four people.

Structuring a modern AI team — Denys Linkov, Wisedocs

Jul 24, 2025 · 17:40

Denys Linkov, who leads ML at Wisedocs, argues that building a modern AI team hinges on identifying your company's bottleneck—shipping features, acquiring users, or scalability—rather than reflexively hiring AI researchers. He introduces Ampere's Wager: trading your entire domain-savvy team for five top-lab researchers is usually a losing bet. For early-stage AI strategy, generalists who blend model training, serving, and business acumen outperform specialists; Linkov lived this in 2021 building a custom MLOps platform for a conversational AI startup and again in 2024 using advanced open-source tools for medical record processing. He stresses reskilling existing teams through weekly learning cadences and moving domain experts from giving feedback to writing evaluations. Hiring should hold context and act on it, verifying trends like 'don't hire juniors' against YC's AI school drawing 2,000 young people.

How agents will unlock the $500B promise of AI - Donald Hruska, Retool

Jul 23, 2025 · 16:22

Donald Hruska, engineering lead for Retool's Agents product, argues that AI agents will unlock the $500B promise of AI by moving enterprises beyond toy chatbots into production-grade systems. He explains that building a basic agent is easy (e.g., 100 lines of code using React) but getting it into production requires solving security, cost overruns, observability, and compliance. Hruska outlines four options—build from scratch, use a framework like LangGraph, a managed platform like Retool Agents, or verticalized agents—and advises building for core differentiators and buying for commodity workflows. He cites Retool customers like ClickUp saving over $200,000 in vendor costs and Descript saving hundreds of hours weekly, while noting inference costs dropped 99.7% from 2022-2024. Retool charges $3 per hour for its cheapest agent and supports on-prem deployment.

Machines of Buying and Selling Grace - Adam Behrens, New Generation

Jul 23, 2025 · 19:37

Adam Behrens, CEO of New Gen, argues that AI will transform commerce from static websites to agentic interactions where buyer and seller agents negotiate via intent infrastructure. He traces the evolution from clerk-assisted stores to e-commerce, now to AI-natives where ChatGPT and Claude act as shopping interfaces. Behrens details three challenges: payment delegation (solved via Viza's delegated authentication), product discovery (a unified API akin to Plaid for merchants), and preference representation (two-sided, dynamic, with market-design solutions). He cites Samsung's adoption of an MCP server for chat clients and notes that AI-sourced users convert at higher rates. Behrens predicts revenue sharing via affiliate models and that agents may bypass credit cards for stablecoins, while merchants retain control by embedding transportable data into model providers' surfaces.

Building Agents (the hard parts!) - Rita Kozlov, Cloudflare

Jul 23, 2025 · 21:12

Rita Kozlov, VP of Product for Cloudflare's developer platform, presents the building blocks of AI agents—client, AI reasoning, workflows, and tools—arguing that effective agents require all four components. She highlights the Model Context Protocol (MCP) as a standard for exposing APIs to LLMs, and demonstrates Cloudflare's Agents SDK, which simplifies hosting remote MCP servers with built-in OAuth, state management via durable objects, and real-time WebSocket communication. Kozlov cites real-world impact: companies using agents for sales automation see 20% revenue increases, 90% faster support response times, and 50–75% time savings. She walks through a human-in-the-loop credit card approval workflow built with Nok, showing how durable objects maintain long-running state, prevent duplicate actions, and route approvals across Slack, email, or in-app notifications. The talk emphasizes that once an MCP server is deployed, it can be used directly from Cursor, Claude, ChatGPT, or a custom client, including voice interfaces via WebRTC-to-WebSocket translation.

Books reimagined: AI to create new experiences for things you know — Lukasz Gandecki, TheBrain.pro

Jul 22, 2025 · 9:44

Łukasz Gandecki of TheBrain.pro presents Books Reimagined, an AI-powered reading companion that adds music, graphics, and conversational AI to books like '1984' and 'The Snow Queen', arguing that AI can create magical experiences by hiding itself behind a polished user interface. He started by vibe-coding a companion for a Trump reelection book to track characters, then iterated into a full multimedia reader. He demonstrates features including 100-millisecond voice response, natural language search via embeddings, and deep research that scans the entire book. Gandecki emphasizes that human curation of AI-generated music and graphics is essential for quality, and he is open-sourcing the player at bookgenius.net.

Stop Using RAG as Memory — Daniel Chalef, Zep

Jul 22, 2025 · 7:02

Daniel Chalef, founder of Zep, explains why using RAG as memory for AI agents fails: vector databases rely on semantic similarity, not business relevance, so irrelevant facts pollute retrieval and cause hallucinations. He advocates for domain-aware memory using temporal knowledge graphs, demonstrated through Zep's open-source Graphiti framework. Chalef shows how developers define custom entity types (e.g., financial goals, debts) with Pydantic or Zod schemas, register them as ontology objects, and build tools that concurrently search Zep filtered by node type. A live demo of a finance coach agent illustrates Zep capturing structured facts like rent payments into a knowledge graph, enabling precise, context-rich memory for accurate agent reasoning.

The Bitter Layout or: How I Learned to Love the Model Picker — Maximillian Piras, Yutori

Jul 21, 2025 · 14:24

In this talk, Maximillian Piras argues that conversational interfaces remain dominant in AI apps not because they are natural, but because they are 'conformable' — able to absorb the next model's capabilities without redesign. He calls this pattern 'The Bitter Layout': an input field, turn-by-turn flow, and a model picker, which prioritizes flexibility over usability. Applying Clayton Christensen's theory of commoditization, Piras claims that as long as scaling laws keep models from commoditizing, the interface itself is the commodity, and designers must conform to the next model. He traces the debate over chat UX from Linus Lee (2022) to Julian Lear (2024), and criticizes the model picker as a 'mode selector' that creates usability issues. Looking ahead, Piras suggests designers shift from procedural thinking to setting goals and constraints, and speculates that future AI UX will be 'grown' like a garden rather than built.

[Full Workshop] Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han

Jul 19, 2025 · 2:42:28

Daniel Han of Unsloth presents a technical workshop covering reinforcement learning (RL), kernels, reasoning, quantization, and agents, arguing that RL with verifiable rewards (RLVR) is the key to unlocking LLM capabilities beyond supervised fine-tuning. He explains why open-source models plateaued after September 2024 until DeepSeek-R1 showed that RL can elicit reasoning, and breaks down PPO, GRPO, and the REINFORCE algorithm, emphasizing that GRPO removes the value model for efficiency. Han details how reward functions—not algorithms—are the hardest part, with examples like distance-based scoring for math. He demonstrates a free Colab notebook training a base model to reason, and shows that dynamic quantization can shrink models like DeepSeek-R1 from 730 GB to 140 GB with only ~1% accuracy loss, arguing that GPUs may stop getting faster after FP4 precision.

Google Photos Magic Editor: GenAI Under the Hood of a Billion-User App - Kelvin Ma, Google Photos

Jul 19, 2025 · 20:28

Kelvin Ma, an engineer on Google Photos' editing team, explains how the billion-user app built the Magic Editor by integrating generative AI with on-device computational photography. He traces the evolution from earlier ML features like post-capture segmentation (UNet model, 10 MB) and Magic Eraser (a system of models) to the new server-side generative AI experience, which handles tasks like relocating objects and reimagining backgrounds. Key challenges include managing model size (now hundreds of MB), client-server latency, ambiguous problem scoping (e.g., moving from 5% to 80% reliability), and trust and safety. Ma advocates using evals, reducing ambiguity through product-research collaboration, and iterating from large general models to smaller, faster ones for production. He also highlights that Google Photos serves 1.5 billion monthly active users and processes hundreds of millions of edits per month, and the editor is being rebuilt as AI-first.

Dream Machine: Scaling to 1m users in 4 days — Keegan McCallum, Luma AI

Jul 19, 2025 · 19:03

Keegan McCallum, Head of ML Infrastructure at Luma AI, details how the company's Dream Machine model scaled from 500 to 9,000 H100 GPUs within hours to handle 1 million users in four days, outpacing ChatGPT's initial growth. He explains that their initial Triton inference server setup was brittle and ill-suited for multi-GPU, multi-node video models, prompting a re-architecture to a custom serving stack on vanilla PyTorch. To solve work starvation across user tiers, they implemented an SLO-based aging system that ranks jobs by the percentage of their worst-case wait time elapsed. For managing dozens of model versions, they store immutable full Python environments and checkpoints in object storage, with a YAML file controlling active deployments and enabling zero-downtime rollouts across thousands of GPUs. McCallum also discusses partnerships with Nvidia, AMD, and Grok, and how Luma's broader mission is to build general multimodal intelligence that generates, understands, and operates in the physical world.

Design like Karpathy is watching — Zeke Sikelianos, Replicate

Jul 19, 2025 · 19:26

Zeke Sikelianos of Replicate analyzes Andrej Karpathy's experience building and deploying MenuGen, a vibe-coded web app that turns menu photos into images, to argue that LLMs are now the primary audience for developer tools. He details Replicate's specific failures—rate limiting and outdated docs that blocked Karpathy—and the fixes they implemented: adding LLMs.txt files for Markdown-friendly documentation, promoting curl commands as the LLM-friendly interface, and launching an MCP server built on OpenAPI schemas. Sikelianos advocates for 'boring technology' like SQL and 'good API hygiene' to keep payloads small and information-dense for LLM context windows. He also calls for better payment acceptance and documentation discipline, stressing that unblocking power users like Karpathy (whose CEO intervened) should not require a viral blog post.

On Curiosity — Sharif Shameem, Lexica

Jul 19, 2025 · 18:35

Sharif Shameem, founder of Lexica, argues that curiosity is the main force for pulling ideas from the future into the present, and that building and sharing demos is the best way to explore AI models' hidden capabilities. He recounts early GPT-3 demos from 2020-2021, when the model had a 2,000-token context window and cost $75 per million output tokens, showing how he built a JSX compiler in the browser, a shopping agent that parsed web pages, and a multi-step reasoning tool called MultiVAC. Shameem emphasizes that AI engineering is more like excavating than traditional engineering, and that researchers often don't know the full capabilities of their own models. He closes by invoking computing pioneer J.C.R. Licklider, arguing that today's AI engineers have a moral obligation to follow their curiosity and share what they discover.

Teaching Gemini to Speak YouTube: Adapting LLMs for Video Recommendations to 2B+DAU - Devansh Tandon

Jul 16, 2025 · 22:51

Devansh Tandon, a Product Manager at Google leading YouTube's discovery system, details how YouTube adapted Gemini LLMs to power its recommendation engine for billions of daily active users. The team built SemanticID, a tokenization system that compresses video features into semantically meaningful tokens, creating a new language for YouTube content. They then continued pre-training Gemini on sequences of user watches to make the model bilingual in English and this video language. For generative retrieval, they prompt the adapted model with user demographics and watch history to output video recommendations as SemanticIDs, achieving 95%+ cost savings to serve at scale. Challenges include serving billions of users with low latency and handling video freshness—Taylor Swift's new music video must be recommendable within minutes. Tandon argues LLM-led recommendations are a bigger consumer application than search and hints at future interactive, steerable recommendations and even personalized content creation.

RL for Autonomous Coding — Aakanksha Chowdhery, Reflection.ai

Jul 16, 2025 · 19:27

Aakanksha Chowdhery, CEO of Reflection AI and former lead researcher on PaLM and Gemini at Google, argues that reinforcement learning (RL) scaling is the next frontier for autonomous coding agents. She explains that inference-time techniques like majority voting and self-revision improve accuracy but require many samples (e.g., 10,000 for rare correct generations). RL training can learn to generate correct outputs directly, especially in verifiable domains like code with unit tests. Reflection AI aims to build superintelligence starting with autonomous coding, leveraging automated verification to design better reward functions. She notes challenges in scaling RL, including system complexity and reward hacking, but sees coding as ideal due to execution feedback.

Benchmarks Are Memes: How What We Measure Shapes AI—and Us - Alex Duffy, Every.to

Jul 15, 2025 · 15:44

Alex Duffy argues that AI benchmarks function as cultural memes—ideas that spread and shape what models learn—giving those who design them immense power over AI's trajectory. He traces the lifecycle from a single person's idea to saturation, using examples like 'How many Rs in strawberry' and Pokémon. Duffy introduces AI Diplomacy, a benchmark where language models negotiate and betray each other, revealing that models like DeepSeek R1 and Gemini 2.5 Flash excel at social manipulation while Claude models are naively optimistic. He warns against benchmarks that reward sycophancy (like ChatGPT's thumbs-up training) and advocates for multifaceted, experiential, and generative benchmarks that empower people. Duffy urges the audience to ask non-AI people what they care about, turning benchmarks into tools that build trust and define humanity's role in an AI world.

Prompt Engineering and AI Red Teaming — Sander Schulhoff, HackAPrompt/LearnPrompting

Jul 14, 2025 · 2:01:05

Sander Schulhoff, creator of Learn Prompting and HackAPrompt, argues prompt engineering remains vital despite claims of its demise, drawing on his systematic review of 1,500+ papers for 'The Prompt Report.' He covers advanced techniques including chain-of-thought, decomposition, ensembling, and few-shot prompting, noting that role prompting is ineffective for accuracy-based tasks and that example ordering can swing performance by 50%. He then explains AI red teaming, distinguishing jailbreaking from prompt injection, and warns that system prompts and guardrails cannot prevent attacks—even simple obfuscation like base64 or typos still works. He highlights the critical unsolved problem of agentic security, where agents with real-world actions are easily tricked, and introduces a live competition at the conference to gather more attack data. His key takeaway: AI security is fundamentally harder than classical cybersecurity because 'you cannot patch a brain.'

Survive the AI Knife Fight: Building Products That Win — Brian Balfour, Reforge

Jul 14, 2025 · 14:10

Brian Balfour, CEO of Reforge, argues that winning in today's AI knife-fight requires answering 'What do I build and why will it win?' by focusing on proprietary data, unique functionality, and unmet customer needs rather than building custom AI. He illustrates with Granola, which entered a crowded AI note-taker market by understanding users wanted help taking better notes, not full automation, and assembled off-the-shelf AI (DeepGram, Anthropic, OpenAI) with unique data (user notes plus transcription) and functionality (Mac app, calendar integration) to create a competitive edge. Balfour warns competitive advantages now last only 2-3 weeks, so teams must sequence smaller moats continuously, each buying time to execute faster. The talk emphasizes treating AI as Lego blocks—assembling pre-trained models, data, and product superpowers into a system that spins a data flywheel.

How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS - Ishan Anand

Jul 13, 2025 · 1:41:34

Ishan Anand shows that GPT-2 small implemented in 600 lines of vanilla JavaScript makes LLMs understandable for web developers without ML backgrounds. He explains tokenization via byte-pair encoding, 768-dimensional embeddings representing semantic meaning via co-occurrence, and the Transformer's attention mechanism that lets tokens share context. The multi-layer perceptron learns next-token prediction through backpropagation, while the language head converts embeddings to token probabilities using softmax. Anand demonstrates each step—tokenization, embedding lookup, positional encoding, attention, MLP, and output—in a browser debugger, and notes that GPT-2's architecture underpins ChatGPT, with innovations like scale, supervised fine-tuning, and RLHF. The workshop provides an intuitive mental model of Transformers, turning perceived AI magic into understandable machinery.

AI Engineering with the Google Gemini 2.5 Model Family - Philipp Schmid, Google DeepMind

Jul 11, 2025 · 1:44:51

In this workshop, Philipp Schmid from Google DeepMind demonstrates AI Engineering with the Gemini 2.5 model family, focusing on using Gemini 2.5 Flash via a free API tier for hands-on coding tasks including text generation, multimodal processing of images, audio, and PDFs, function calling with structured outputs, and integration with MCP servers. The session covers setting up API keys in AI Studio, uploading files via the Files API (free for 1 day), and controlling thinking budgets (0–24,000 tokens) to manage cost and reasoning depth. Schmid shows how Gemini natively processes videos at 1 frame per second for accurate timestamp extraction and how PDFs are handled by combining OCR text with image understanding. He introduces native tools like Google Search with grounding metadata, code execution, and URL context, and explains how MCP servers can be used seamlessly with the Gemini SDK for tool calling. The workshop also covers parallel vs sequential function calling, the Agent Development Kit (ADK), and the upcoming asynchronous function calling for the Live API, providing a practical path from simple generation to agentic workflows.

Production software keeps breaking and it will only get worse — Anish Agarwal, Traversal.ai

Jul 10, 2025 · 18:13

Anish Agarwal and Matthew Schoenbauer of Traversal.ai argue that as AI writes more code, production troubleshooting will become vastly harder, requiring a new approach combining causal machine learning, reasoning models, and agentic swarms to autonomously resolve incidents in minutes. They explain that traditional AI ops generates too many false positives, LLMs can't handle petabyte-scale data, and simple agents depend on deprecated runbooks. Their Traversal AI orchestrates thousands of parallel agentic tool calls to sift through trillions of logs and metrics, identifying root causes and citing observability data. A case study with DigitalOcean shows a 40% reduction in mean time to resolution (MTTR), with the system delivering findings in about five minutes. The episode details how this approach turns frantic incident Slack channels into autonomous, cited root-cause analysis, freeing engineers to focus on system design.

Thinking Deeper in Gemini — Jack Rae, Google DeepMind

Jul 10, 2025 · 18:13

Jack Rae, lead of Gemini Thinking at Google DeepMind, presents thinking as a solution to the fixed test-time compute bottleneck in large language models. He explains that Gemini inserts a thinking stage where the model iterates via reinforcement learning, learning to self-correct and explore multiple strategies. This enables a continuous cost-performance tradeoff via thinking budgets, improving reasoning on math and code. Future work includes Deep Think, which raises USA Math Olympiad performance from the 50th to the 65th percentile by scaling inference compute further. Rae envisions models that, like mathematician Ramanujan, achieve deep, data-efficient reasoning from limited knowledge.

2025 in LLMs so far, illustrated by Pelicans on Bicycles — Simon Willison

Jul 9, 2025 · 18:30

Simon Willison reviews the past six months of LLM releases — including AWS Nova, Llama 3.3 70B, DeepSeek R1, Mistral Small 3, Claude 3.7 Sonnet, GPT 4.5, Gemini 2.5 Pro, GPT-4o, Llama 4, GPT 4.1, O3/O4 Mini, and Claude 4 — using his 'pelican on bicycle' SVG benchmark to argue that local models have become good enough to run GPT-4 class models on a laptop and that combining tools with reasoning is the most powerful technique in AI engineering, while noting risks like prompt injection and the 'lethal trifecta'. He tracks 30 significant model releases, highlighting that Mistral Small 3 (24B) matches Llama 3 70B's performance, which itself matched the 405B model, enabling local inference. DeepSeek's R1 caused a $500B+ Nvidia stock drop on January 27. GPT 4.1 Nano is the cheapest model yet at a fraction of a cent per pelican. He also examines bugs: ChatGPT's sycophantic 'shit-on-a-stick' incident and Claude 4's tendency to snitch to authorities when given ethical instructions and email tools. Willison concludes that while the pace is accelerating, control over context and security remain critical.

Trends Across the AI Frontier — George Cameron, ArtificialAnalysis.ai

Jul 8, 2025 · 17:52

George Cameron of Artificial Analysis presents multiple frontiers—reasoning, open-weights, cost, speed—across the AI stack, arguing that trade-offs between intelligence, latency, and expense are critical for building applications. Reasoning models like O4 mini high use an order of magnitude more output tokens (72M vs GPT-4.1's 7M) and take over 40 seconds per response versus 4.7 seconds, impacting agentic workflows where 30 sequential calls multiply latency. The open-weights gap has nearly closed, with China-based labs like DeepSeek R1 and Alibaba's Qwen 3 leading. O3 cost roughly $2,000 to run the intelligence index, while GPT-4.1 nano is over 500 times cheaper. Output speeds have jumped from GPT-4's 40 tokens/s in 2023 to over 1,000 on a B200 accelerator. Despite efficiency gains, demand for compute will keep rising due to larger models, reasoning’s extra tokens, and multi-step agents.

New York Times' Connections: A Case Study on NLP in Word Games — Shafik Quoraishee, NYT Games

Jul 5, 2025 · 18:31

Shafik Quoraishee, a game developer at NYT Games, presents his independent research into applying AI to solve the New York Times Connections word game. He explains that Connections, launched in June 2023 and second only to Wordle with hundreds of millions of plays, challenges AI's abstract reasoning through intentional decoys and overlapping categories. Quoraishee models the game as a graph coloring problem and uses semantic similarity, relational alignment scores across multiple dimensions (orthography, morphology, encyclopedic, etc.), and graph neural networks combined with reinforcement learning to build a solver. His preliminary results show increased solvability for hard puzzles, contrasting with LLMs that may simply recall internet solutions. The work aims to provide a transparent, explainable AI approach for puzzle-solving and potential game development applications.

12-Factor Agents: Patterns of reliable LLM applications — Dex Horthy, HumanLayer

Jul 3, 2025 · 17:06

Dex Horthy, founder of HumanLayer, presents the 12-Factor Agents framework for building reliable LLM-powered applications, arguing that production-grade agents are primarily deterministic software with targeted LLM steps rather than fully autonomous loops. He distills patterns: own prompts and context windows, treat tools as JSON and code, use small focused agents with three to ten steps, contact humans via tool calls. Horthy emphasizes context engineering—LLMs are pure functions—and shows how to compact errors, unify state, and add pause/resume via APIs. He shares a DevOps agent that became a bash script, and advocates for outer-loop agents. The framework, which gained 4,000 GitHub stars in two months, treats agents as stateless reducers that meet users on any channel, with engineers controlling the inner loop of token and control flow.

The emerging skillset of wielding coding agents — Beyang Liu, Sourcegraph / Amp

Jun 30, 2025 · 35:06

Beyang Liu, CTO and co-founder of Sourcegraph, argues that coding agents are a real and high-ceiling skill, contrary to skeptics like Jonathan Blow and Eric S. Raymond. He presents design decisions for the agentic era: agents should directly edit files instead of asking permission, UIs should be minimal (like Amp's bare-bones VS Code extension and CLI), and fixed pricing should give way to usage-based models. In a live demo, Liu uses Amp to implement a custom icon for the linear connector in Amp's own codebase, demonstrating agentic search and sub-agents. He shares power-user patterns, including writing long prompts, constructing feedback loops with Playwright and Storybook, and running multiple agents in parallel—as exemplified by Jeff Huntley using Amp to build a compiler while sleeping. Liu emphasizes that agents should enable more thorough code reviews, not replace human understanding.

Turning Fails into Features: Zapier’s Hard-Won Eval Lessons — Rafal Willinski, Vitor Balocco, Zapier

Jun 30, 2025 · 16:15

Zapier AI Tech Lead Rafal Willinski and Staff Engineer Vitor Balocco explain how Zapier's evaluation system turns agent failures into targeted improvements through a data flywheel. They detail collecting explicit feedback at critical moments and mining implicit signals like testing behavior, cursing, and user follow-ups. The pair advocates building unit test evals for specific failure modes, then trajectory evals and LLM-as-judge with rubrics to avoid overfitting and capture multi-turn criteria. They share that over-indexing on unit tests hurt model benchmarking, and reasoning models can compare model runs, revealing differences like Claude as a decisive executor versus Gemini's yapping. Ultimately, they argue that the goal is user satisfaction, so A/B testing on a small traffic fraction is the ultimate verification.

How to build world-class AI products — Sarah Sachs (AI lead @ Notion) & Carlos Esteban (Braintrust)

Jun 27, 2025 · 1:43:46

Sarah Sachs (AI lead at Notion) and Carlos Esteban (Braintrust) explain that building great AI products requires 10% prompting and 90% evals and observability, with Notion AI using Braintrust to iterate on prompts and models. Sachs details their cycle: curate small datasets, tie them to scoring functions (LLM-as-a-judge with per-sample prompts and heuristic checks), run evals before shipping, and use production logs to catch regressions. She notes Notion AI supports 100M+ users, switches models in under a day, and 60% of enterprise users are non-English—requiring multilingual eval rigor. Carlos and Doug then walk through Braintrust's framework: tasks (prompts, tools, agents), datasets, and scores (0-1). They demonstrate offline evals via playground and SDK, online scoring in production, user feedback capture, human review setup, and remote evals to bridge complex code with the playground. The workshop covers moving from pre-prod evals to production monitoring, closing the feedback loop by adding underperforming spans to datasets.

From Mixture of Experts to Mixture of Agents with Super Fast Inference - Daniel Kim & Daria Soboleva

Jun 27, 2025 · 53:15

Daria Soboleva and Daniel Kim of Cerebras explain how Mixture of Experts (MoE) architectures enable scaling large language models efficiently by replacing monolithic feedforward networks with specialized experts, a technique used by GPT-4 and Claude. They then introduce Mixture of Agents (MoA), which combines multiple LLMs with custom prompts to outperform frontier models like GPT-4o on complex tasks, reducing a 293-second reasoning problem to 7.4 seconds using Cerebras' ultra-fast inference. The workshop guides participants to build their own MoA system, configure agents for bug fixing and performance optimization on a Python function, and achieve scores up to 120/120. Daniel details Cerebras' wafer-scale chip with 900,000 cores and distributed memory that eliminates memory bandwidth bottlenecks, enabling linear scaling and 15.5x faster inference on Llama 3.3-70B versus GPUs. Daria discusses ongoing research in diffusion models and sparsity, while Daniel notes plans for multimodal APIs and LoRA fine-tuning support.

Forget RAG Pipelines—Build Production Ready Agents in 15 Mins: Nina Lopatina, Rajiv Shah, Contextual

Jun 27, 2025 · 1:15:43

Rajiv Shah, Nina Lopatina, and Matthew from Contextual AI demonstrate how developers can build production-ready RAG agents in minutes using Contextual AI's managed RAG platform, emphasizing that RAG should be treated as a managed service to avoid reinventing infrastructure. They walk through ingesting documents like NVIDIA financials and spurious correlation reports, then querying the agent with questions requiring quantitative reasoning across tables. The platform handles extraction, layout analysis, image captioning, hybrid retrieval, and a state-of-the-art reranker, capped by a grounded language model that avoids hallucinations and provides attribution. Evaluation is done via LM Unit, a model-as-judge that scores responses on criteria like accuracy and causation. The episode also shows integrating the agent with Claude Desktop via MCP and answers audience questions on pricing (consumption-based with a $25 credit), scalability, entitlements, HIPAA, and domain-specific language.

[Full Workshop] Vibe Coding at Scale: Customizing AI Assistants for Enterprise Environments

Jun 27, 2025 · 1:20:38

Harold from the VS Code team demonstrates "Vibe Coding at Scale" at the AI Engineer World's Fair, presenting three stages—YOLO, structured, and spectrum vibes—for customizing AI assistants in enterprise environments. He live-codes a hydration tracking app using GitHub Copilot's agent mode, auto-approve, and new workspace scaffolding, then shows how to enforce design principles via Copilot instructions and reusable prompts. Harold introduces custom modes (e.g., TDD mode), tool sets, and MCP servers like Playwright and Perplexity for research and browser testing. He emphasizes iterating on instructions, committing often, and using spec-driven development to balance speed with reliability, arguing these techniques enable true flow-state collaboration even on complex codebases.

Building Code First AI Agents with Azure AI Agent Service — Cedric Vidal, Microsoft

Jun 27, 2025 · 1:54:06

Cedric Vidal, Principal AI Advocate at Microsoft, demonstrates building code-first AI agents with Azure AI Agent Service, using function calling, code interpreter, file search, and Bing grounding for a sales analysis use case. He creates a conversational agent that queries SQLite, generates pie charts from Python code, and blends product PDF data with relational data. The workshop explains stateful agents, tool routing (LLM generates JSON to call functions), and the limits of single-step agents versus multi-agent orchestration with AutoGen. Cedric addresses when to use Agent Service (managed persistence and tools) versus raw LLM endpoints, and covers MCP servers as a tool lifecycle manager. Key insights include using instructions to ground agent behavior and the need for eval frameworks for agent quality.

Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB

Jun 27, 2025 · 17:37

Richmond Alake from MongoDB presents memory management as the key pillar for building believable, capable, and reliable AI agents, arguing that agentic systems require structured memory types—persona, toolbox, conversational, workflow, episodic, and entity—to achieve statefulness and reduce reliance on prompt engineering. He introduces MemoRiz, an open-source library implementing these memory design patterns, and positions MongoDB as a flexible memory provider with its document model and hybrid retrieval capabilities (vector, text, graph). Alake details practical patterns: storing tool schemas for scalable tool use, persisting conversation history with timestamps and recall signals, leveraging workflow memory to learn from failures, and using MongoDB's upcoming integration of Voyage AI embedding models to simplify chunking and retrieval. He connects these advances to neuroscience, citing how feline visual cortex research inspired CNNs and noting recent collaborations with neuroscientists to further agent memory research.

Memory Masterclass: Make Your AI Agents Remember What They Do! — Mark Bain, AIUS

Jun 27, 2025 · 51:25

Mark Bain, Vasilia Markovits, Alex Gilmore, and Daniel Chalev demonstrate that AI memory requires causal relationships and graph databases, not just vector similarity, to solve hallucinations and enable agentic workflows. Bain argues that memory is any data affecting change, and that attention, diffusion, and VAEs follow the same geometric principles as gravity and entropy. Alex shows Neo4j's MCP server storing semantic memory as entities and relationships retrieved across conversations. Vasilia demos Cognee building semantic graphs from GitHub data for agentic hiring decisions. Daniel presents Graphiti's domain-aware memory using custom Pydantic schemas to filter irrelevant facts. Bain introduces a GraphRAG chat arena that switches between memory solutions on a single Neo4j graph, testing different implementations for episodic and temporal recall.

Mastering Engineering Flow with Windsurf - Eashan Sinha, Windsurf

Jun 27, 2025 · 15:20

Eashan Sinha, a Deployed Engineer at Windsurf, explains how engineering teams can master agentic IDEs using Windsurf's Cascade agent. He argues that developers should treat AI as a teammate rather than a tool to abuse, emphasizing Windsurf's flow awareness—tracking user actions, edits, and terminal commands to infer intent. Sinha details Windsurf's evolution from copilots to agents, combining collaborative chat with autonomous tool calling, and introduces features like state-of-the-art context engine, workflows for deterministic steps, rules, memories, and simultaneous Cascades. He provides a practical four-step framework—explore, plan, build, test—for maximizing agentic flow, including using planning files, checkboxes, and automated test generation. The episode delivers a concrete guide to integrating AI into development without sacrificing quality, recorded at the AI Engineer World's Fair in San Francisco.

Fun stories from building OpenRouter and where all this is going - Alex Atallah, OpenRouter

Jun 25, 2025 · 18:47

Alex Atallah, founder of OpenRouter, tells the story of how he launched the LLM aggregator in early 2023 after observing the open-source race sparked by Meta's Llama 1 and Stanford's Alpaca distillation under $600. He argues the inference market is not winner-take-all, citing OpenRouter's data showing Google Gemini growing from 2-3% to 34-35% of tokens over 12 months. The platform evolved from a simple model collection into a marketplace with over 400 models and 60 providers, solving architecture challenges like 30-millisecond latency, cancelable streams, and a middleware plugin system for web search and PDF parsing. Atallah explains that OpenRouter grew 10-100% month-over-month for two years, and he predicts future additions like transfusion models that generate images and more powerful geographic routing for enterprise optimization.

Building AI Agents that actually automate Knowledge Work - Jerry Liu, LlamaIndex

Jun 24, 2025 · 17:57

Jerry Liu, CEO of LlamaIndex, argues that building AI agents which actually automate knowledge work requires a combination of really good tools and carefully tailored agent reasoning, moving beyond naive RAG to a 'document toolbox' that includes parsing, extraction, indexing, and manipulation of complex unstructured data like PDFs, Excel, and PowerPoints. He introduces two main agent categories: assistive agents (chat-based, human-in-the-loop, unconstrained reasoning) and automation agents (batch-processing, constrained workflows, less human oversight). A key new capability is an Excel agent that uses reinforcement learning to learn a semantic map of unnormalized spreadsheets, achieving 95% accuracy on data transformation, surpassing the 75% baseline of LLMs with code interpreter and even human baselines of 90%. Real-world examples include financial due diligence combining automation (inhaling financial data) and assistant interfaces (analyst co-pilot), enterprise search with specialized agentic RAG, and technical data sheet injection for a global electronics company that reduced weeks of manual work to an automated extraction pipeline. Liu emphasizes that the right preprocessing…

How to Build Trustworthy AI — Allie Howe

Jun 16, 2025 · 24:22

Allie Howe, VC CISO at GrowthCyber, argues that trustworthy AI equals AI Security (how the world harms your AI) plus AI Safety (how your AI harms the world), and that builders are legally and reputationally responsible. She covers three pillars: MLSecOps (scanning models for serialization attacks using open‑source ModelScan), AI Red Teaming (testing for prompt injections, jailbreaks, and safety issues with tools like PyRIT), and AI Runtime Security (validating inputs/outputs in production via platforms like Pillar to block off‑topic or unsafe behavior). Howe cites real incidents—a Chevy Tahoe chatbot offered for $1, Slack's data leak via prompt injection, Fortnite's Darth Vader NPC initially producing racist outputs—and notes a lawsuit where OpenAI won by arguing users must expect errors. She emphasizes shifting‑right to runtime guardrails as the most cost‑effective investment and advises demonstrating trustworthiness through GRC platforms like Vanta to shorten sales cycles. With increasing regulation (ISO 42001, EOAI Act, FDA guidelines), she concludes that building trustworthy AI now unlocks revolutionary innovation in fields like healthcare.

Beyond Conversation: Why Documents Transform Natural Language into Code - Filip Kozera

Jun 10, 2025 · 10:57

Filip Kozera argues that chat-based interfaces like ChatGPT create ephemeral, polluted context windows and lack forced clarity, making them poor for specifying complex systems, while documents naturally force rigor and structured thinking. He advocates shifting from conversational brainstorming to document-driven workflows, where humans precisely articulate intent for AI agents. Kozera introduces background agents that run asynchronously, triggered by events like emails or meetings, and surface only when needing human approval, turning the human role into one of 'swiping left or right' to approve or edit outputs. He envisions a future where humans manage swarms of such agents, first adopted by prosumers and later by enterprises, with taste and intent becoming critical for imbuing personal brand into agent outputs. The episode presents a clear progression from chat-based brainstorming to structured document-driven agent systems.

MCPs are Boring (or: Why we are losing the Sparkle of LLMs) - Manuel Odendahl

Jun 10, 2025 · 28:32

Manuel Odendahl argues that MCPs (Model Context Protocols) are boring because they constrain LLMs to rigid, predefined tool calls, while LLMs truly shine as code generators that can create dynamic tools on the fly. He critiques the inefficiency of tool calling—repeating context, wasting tokens, and struggling with many tools—and advocates for an 'Eval' approach where the LLM writes and executes code (e.g., SQL, JavaScript) to solve tasks directly. Demonstrating with a JavaScript sandbox, he shows how a single Eval tool can introspect databases, write queries, create REST APIs, and even build a full CRM interface in one call, saving time and tokens. Odendahl urges engineers to think recursively: ask the LLM to write code that writes more code, unlocking infinite loops of creation and restoring the sparkle of LLMs beyond boring function calls.

The Agent Native Company — Rick Blalock, Agentuity

Jun 3, 2025 · 20:58

Rick Blalock of Agentuity argues that an agent-native company, built from the ground up with AI agents at the core of product, operations, and culture, is fundamentally different from an AI-enhanced one that merely uses AI as a tool. He contrasts the two: removing agents from an agent-native company would halt productivity, while an AI-enhanced business would just become less efficient. Blalock describes the agent-native workday, where humans oversee agents that handle routine tasks, and notes the rise of roles like 'Agent Manager' and the importance of AI fluency in hiring. He shares how his 7-person team built an entire agentic cloud infrastructure in weeks using agents like Devin, arguing that this paradigm shift requires founders to rethink org charts, roles, and skills. The episode concludes that businesses must decide whether they are just using AI or ready to be built around it.

Blender MCP and The Future Of Creative Tools - Siddharth Ahuja

Jun 3, 2025 · 16:50

Siddharth Ahuja introduces Blender MCP, an open-source project that lets LLMs like Claude control Blender via the Model Context Protocol, dramatically lowering the barrier to 3D creation. The tool, which has garnered 11,500 GitHub stars and over 160,000 downloads, enables users to generate complex scenes—like a dragon guarding a pot of gold or a full terrain with nodes—in minutes through natural language prompts. Ahuja explains that Blender’s scripting capabilities are key, and notes that keeping MCP tools lean avoids confusion for the LLM. He demonstrates integrations with AI asset generators and even built an Ableton MCP to create soundtracks, arguing that MCPs will become the glue for creative toolchains, allowing LLMs to orchestrate Blender, Unity, and Ableton together. This shifts creators from tool experts to orchestra conductors, unlocking a new wave of accessible content creation.

The Coherence Trap: Why LLMs Feel Smart (But Aren’t Thinking) - Travis Frisinger

Jun 3, 2025 · 20:47

Travis Frisinger argues that large language models are not intelligent but coherent, introducing 'coherence reconstruction' as a mental model for understanding their utility. He explains that hallucinations are a feature, not a bug, as models fill gaps predictably to maintain pattern completion. Prompts act as force vectors navigating latent space, and the key to reliable outputs is a 'frame, generate, judge, iterate' loop. Frisinger presents a three-layer model: latent space, execution layer (tools and RAG), and conversational interface. He advises engineers to design for emergence rather than control, using dense context as anchors to steer generation, and to watch for breakdowns in tone as early signs of lost coherence.

How agents broke app-level infrastructure - Evan Boyle

Jun 3, 2025 · 13:32

Evan Boyle, founder of GenSX, argues that LLM-powered agents have broken assumptions about app-level infrastructure, as p1 latency jumps from milliseconds to seconds and workflows run minutes to hours. He explains that traditional serverless providers timeout after 5 minutes, lack native streaming, and force brittle Rube Goldberg machines on top of queues like SQS or tools like Airflow. Boyle presents an open-source library that separates API and compute layers using Redis streams for resumability, enabling users to refresh pages or navigate away without losing progress. He demonstrates components with built-in retries, caching, and tracing, and a workflow that automates Hacker News analysis. Key architectural lessons include starting simple but planning for long-running agents, keeping compute and API planes separate, and leaning on Redis for resumability.

Invisible Users, Invisible Interfaces: Accelerating Design Iteration with AI Simulation - Alex Liss

Jun 3, 2025 · 12:37

Alex Liss, VP of Data Science and AI at Huge, argues that AI should be used as a tool for design itself rather than as a feature, proposing Intelligent Twins—AI simulations of user personas—to accelerate needfinding and interface testing. Citing a trust gap where only 32% of US adults trust AI, he advocates for repairing it by designing better interfaces, not adding chatbots. He demonstrates a global audit of sports websites (NBA, Olympics, Premier League) where Intelligent Twins simulated casual and super fans across 72 tasks, revealing strong navigation but declining performance in information architecture and engagement. The methodology uses computer use agents for visual evaluation and human-in-the-loop review to generate focused design briefs. Limitations include reproducibility and the need for test-and-control validation, with future potential in integrating MCP protocol for turning Figma components into code. Liss concludes that AI simulation can help teams gather insights faster and create simpler, trustworthy interfaces.

Text-to-Speech Data Preparation and Fine-tuning Workshop - Ronan McGovern

Jun 3, 2025 · 34:00

Ronan McGovern walks through fine-tuning Sesame's CSM-1B text-to-speech model on a specific voice, using a YouTube video as the data source. He explains token-based TTS models, including how audio is represented via codebooks and how CSM-1B uses a main transformer for zeroth tokens and a secondary transformer for 31 hierarchical tokens. The workshop covers data preparation: downloading audio with yt-dlp, transcribing with Whisper Turbo, manually correcting the transcript, and splitting audio into 30-second chunks (41 clips from a 30-minute video). Fine-tuning uses Unsloth with LoRA adapters (rank 32, alpha 16) on linear layers, training for one epoch with a batch size of 2 and virtual size of 8, reducing loss from ~6.34 to ~3.72. Evaluation compares zero-shot inference (random speaker), voice cloning (closer but imperfect), and fine-tuned plus cloning (best result, producing an Irish-accented voice with natural errors). McGovern recommends 50+ 30-second clips for noticeable effect and notes that combining fine-tuning with voice cloning yields good performance even with limited data.

Unlocking Africa's Potential with AI — Thabang Ledwaba

Jun 3, 2025 · 26:17

Thabang Ledwaba argues that Africa, often seen as a latecomer to AI, actually has immense potential to lead in AI-driven innovation by leveraging its unique challenges and creativity. He points to Kenya's third-highest daily ChatGPT usage and fintech successes like M-Pesa as evidence of immersion in technology. Criticizing over-engineered solutions, he contrasts ticket systems for home office queues with his idea to auto-initiate ID applications at age of eligibility. He highlights Africa's 30% of earth minerals, noting the irony of exporting raw materials only to import finished goods, and calls for a mindset shift akin to China's 'serve yourself first' strategy. Ledwaba showcases African innovations like Nigerian pharma wings and Moroccan Project Cumulus, urging Africans to see themselves as producers, not just consumers, and to harness AI for sustainable development.

Analyzing 10,000 Sales Calls With AI In 2 Weeks — Charlie Guo

Jun 3, 2025 · 9:49

Charlie Guo explains how his team at Pully used Claude 3.5 Sonnet to analyze 10,000 sales call transcripts in two weeks, turning a task that would take 625 days manually into a $500 project. They chose Claude over smaller models due to unacceptable hallucination rates, and reduced costs by up to 90% using prompt caching and extended outputs. The system combined RAG enrichment, chain-of-thought prompting, and structured JSON outputs with citations to ensure accuracy. The analysis, originally for the executive team, became a company-wide resource, enabling marketing to pull customer quotes and sales to automate transcript downloads, saving dozens of hours weekly. Guo emphasizes that good engineering—JSON outputs, database schemas, and thoughtful integration—matters as much as AI capability, and challenges listeners to mine their own untapped customer data.

Letting AI Interface with your App with MCP — Kent C Dodds

Jun 3, 2025 · 19:54

Kent C Dodds explains how Model Context Protocol (MCP) enables AI assistants like Claude Desktop to interface with any app or service, moving beyond limited built-in integrations. He traces the evolution from early LLM chat (phase one) through host applications with custom integrations (phase two) to MCP's open standard (phase three), arguing that MCP lets developers build once for any assistant. In his live demo, he runs a Locationator server to derive his location, a GetWeather server for weather, and an Epic Me server with OAuth authentication to create a journal entry. MCP servers handle tools, resources, and prompts; the client standardizes communication while the server controls unique logic. Kent contends this protocol is how we give AI "hands" to perform real tasks, leading toward a universal Jarvis-like assistant that can dynamically connect to any service without manual integration per platform.

ChatGPT is poorly designed. So I fixed it

Jun 3, 2025 · 5:21

Ben Holmes critiques ChatGPT's confusing design, calling it a case of 'shipping the org chart' where voice and text features feel like separate apps. He demonstrates a fix by building FixGPT, an app that allows simultaneous voice and text interaction and smartly routes requests to the right model via tool calls. Using off-the-shelf APIs like 4.0 real-time and simple prompts, Holmes shows how a 'send chat message' tool enables detailed text responses alongside voice, while a reasoning model tool handles complex queries. The source code is available on GitHub, showcasing how to avoid OpenAI's science fair of options.

Designing AI To Scale Human Thought — Jun Yu Tan, Tusk

Jun 3, 2025 · 12:24

Jun Yu Tan, founding engineer at Tusk, argues that AI should augment human thinking rather than automate tasks, introducing three interaction patterns — blind spot detection, cognitive partnership, and proactive guidance — to help users reveal blind spots, spark creativity, and amplify decision-making. At Tusk, this approach caught verified bugs in 43% of pull requests and added nearly 1,000 tests in two months by systematically surfacing edge cases and assumptions. Tan emphasizes progressive, contextual, and bidirectional trust, and advocates for product metrics that track user skill growth, not just engagement. The episode concludes that the most profound technologies make us more thoughtful, not just productive, and that AI should act as a mirror for our minds, showing us blind spots we habitually miss.

Creating Agents that Co-Create — Karina Nguyen, OpenAI

Apr 30, 2025 · 24:22

Karina Nguyen, an AI researcher at OpenAI and former Anthropic researcher, discusses two major AI scaling paradigms—next-token prediction (pretraining) and reinforcement learning on chain of thought—that shift AI from narrow tools to collaborative agents that co-create. Pretraining builds world understanding by predicting the next token, but hard tasks like math and creative writing require chain of thought reasoning, scaled with OpenAI’s O1 model. Post-training via RLHF and synthetic data enables rapid iteration, and the next stage is co-innovators: agents with reasoning, tool use, long context, and creativity. Nguyen shares product lessons from developing ChatGPT and Claude: 100K context via file uploads, ChatGPT Tasks for scheduled reminders that scale with model capabilities, and Canvas as a flexible interface for co-writing, coding, and research that can morph into an IDE, a tutor, or a data scientist. She envisions a future of invisible software creation where AI generates personalized, multimodal outputs on the fly, reducing reliance on clicking links, and the interface becomes a blank canvas that adapts to user intent.

Function Calling is All You Need — Full Workshop, with Ilan Bigio of OpenAI

Apr 23, 2025 · 1:42:54

Ilan Bigio from OpenAI argues function calling is the essential primitive for building AI agents, demonstrating how to implement memory, delegation, asynchronous task execution, and even self-modifying tools—all using the same loop. Starting from a basic weather tool, he builds an agent that stores user facts in a JSON file, delegates complex requests to o1, runs multiple weather lookups in parallel with asyncio, and creates a Bootstrap agent that writes its own Python functions on the fly. He also shares practical tips from his work on Swarm and the Real-Time API, including a stay-silent function to prevent premature responses and the ability to read scripts with XML tags. The workshop emphasizes that sophisticated agent behaviors emerge from simple function calling patterns without needing heavy frameworks.

Frontier Feud: Anthropic, Google DeepMind, Meta FAIR, Thinking Machines — Barr Yaron, Amplify

Apr 19, 2025 · 22:26

Teams from Anthropic, Google DeepMind, Meta FAIR, and Thinking Machines compete in a Frontier Feud game hosted by Barr Yaron at the AI Engineer Summit 2025. Surveyed AI engineers name Ilya Sutskever as the most influential AI researcher, with Andrej Karpathy, Jeff Hinton, and Yann LeCun also on the board. Cost is the top consideration when choosing a model, followed by latency, eval scores, and open vs. closed source. The buzzword everyone is tired of hearing is AGI, with RAG and prompt engineering trailing. In the fast money round, Cursor tops favorite AI tools, 'Attention Is All You Need' wins most influential paper, and hardware failure is the biggest 2 a.m. nightmare. The winning team, Rocco's Basilisk, takes home a massive llama and other prizes.

AI Engineers: The Next Generation — Stefania Druga, Google Gemini

Apr 13, 2025 · 21:47

Stefania Druga, a research scientist on Google Gemini, presents Cognimates and its new AI copilot that teaches children to become AI engineers. Built on Scratch’s visual programming, Cognimates lets kids train custom models, program robots, and build games, shifting their perception from ‘AI is magic’ to understanding data and confidence levels. Druga’s design studies with 18 kids from 11 countries showed the copilot doubled programming time by supporting ideation, debugging, and creative agency—never giving answers unless stuck three times. The tool now integrates screenshot analysis and asset generation, with plans to embed the agent across the OS. Druga ties this to the EU AI Act’s mandate for AI literacy, arguing early, hands-on creation is the path to informed users.

Personal, Local, Private AI Agents: Soumith Chintala

Apr 6, 2025 · 20:32

Soumith Chintala, co-founder of PyTorch, argues that personal AI agents should run locally and privately to maintain trust and control over intimate data. He warns that cloud-based agents, lacking complete context (like access to all messaging or financial accounts), become unreliable and potentially dangerous—catastrophic actions like buying a Tesla instead of Tide Pods are possible. Key technical challenges include slow local inference, immature open-source computer-use models, and poor catastrophic action classification. He is bullish on open models surpassing closed ones through coordinated improvement, citing Linux, Llama, and DeepSeek. Chintala also plugs open reasoning data from gr.ink and PyTorch’s work on enabling local agents, urging AI engineers to tackle these gaps.

Scaling Agents for Gen AI Products - Anju Kambadur, Bloomberg Head of AI Engineering

Apr 1, 2025 · 19:38

Bloomberg's Head of AI Engineering, Anju Kambadur, details the company's shift from building proprietary LLMs to leveraging open-source models for agentic products, emphasizing that scaling agents requires accepting inherent fragility and building resilient guardrails rather than seeking perfect upstream systems. Drawing on Bloomberg's daily scale—400 billion structured data ticks, over a billion unstructured messages, and 40+ years of history—Anju explains how their 400-person AI team organized into 50 teams across London, New York, Princeton, and Toronto. He argues that agents must be semi-autonomous with non-optional guardrails (e.g., preventing financial advice, ensuring factuality) because compounding errors from evolving APIs and LLMs demand self-contained safety checks. The talk uses the example of a research analyst agent that factors query understanding, answer generation, and guardrails into separate components, reflecting an org structure that collapses vertically for fast iteration on single agents then later introduces horizontal teams (like guardrails) for optimization and cost reduction. Anju stresses that readiness to build horizontal teams comes after multiple…

Why Agent Engineering — swyx

Mar 24, 2025 · 11:45

Swyx argues that 2025 is the year of agent engineering, pivoting the AI Engineer Summit to focus exclusively on agents and explaining why agents are now viable due to advancements in reasoning, tool use, model diversity, and a 1000x cost reduction in GPT-4-level intelligence over 18 months. He highlights that OpenAI's ChatGPT grew 33% to 400M users in three months after shipping agentic O1 models, and projects it will reach 1B users by year-end. Swyx cites Simon Willison's crowdsourced definitions and OpenAI's new agent definition, while calling for a halt to demos of flight booking agents and Reddit astroturfing. He frames agent engineering as the evolution of AI engineering, distinct from MLE and software engineering, with practical use cases like coding and support agents having clear product-market fit.

Rethinking how we Scaffold AI Agents - Rahul Sengottuvelu, Ramp

Mar 19, 2025 · 16:32

Rahul Sengottuvelu, Head of Applied AI at Ramp and co-founder of Cohere.io, argues that building AI agents should follow the 'bitter lesson' from AI research: systems that scale with compute outperform handcrafted, deterministic code. He illustrates this with Ramp's switching report agent, which ingests arbitrary CSV files from third-party card providers. Three approaches are compared: manually coding parsers for the 50 most common vendors, using LLMs only for column classification, and a fully LLM-driven method where the model writes and runs pandas code via a code interpreter, repeated 50 times in parallel. The last approach, though using 10,000x more compute, costs under a dollar and generalizes better, saving Ramp far more in failed transactions. Sengottuvelu also demonstrates a prototype email client where the backend is an LLM with access to a code interpreter and the user's Gmail token; the LLM renders the UI as Markdown and handles clicks by re-prompting itself, simulating a full web app without traditional backend code. He contends that as models exponentially improve, shifting more execution into 'fuzzy' LLM compute—rather than rigid code—lets builders ride the trend for…

How Windsurf writes 90% of your code with an Agentic IDE - Kevin Hou, Windsurf

Mar 11, 2025 · 20:50

Kevin Hou, head of product engineering at Codeium, introduces Windsurf as the first AI agent-powered editor, arguing that agents are the future of software development. The editor is built on three principles: trajectories, a unified timeline that lets the agent read the user's mind and execute commands like 'continue my work'; meta-learning, which auto-generates memories of user preferences and codebase context; and scaling with intelligence, removing legacy features like chat and @-mentions as models improve. Windsurf launched three months ago and has already generated 4.5 billion lines of code, with 90% of code now written via its Cascade agent. Hou emphasizes that the agent reduces human-in-the-loop effort by inferring context, running terminal commands in the user's environment, and dynamically retrieving documentation—allowing developers to contribute less input while getting more production-ready output.

OpenAI for VP's of AI + Advice for Building Agents

Mar 5, 2025 · 16:52

OpenAI's Toki Sherbakov and Prashant Mital explain how enterprises adopt AI through a three-phase journey: building an AI-enabled workforce with ChatGPT, automating operations with APIs, and infusing AI into end products. They detail a Morgan Stanley case study where retrieval methods improved an internal knowledge assistant's accuracy from 45% to 98%. The pair define agents as models with instructions, tools, and self-terminating execution loops, then share four field lessons: build with primitives before frameworks, start with a single purpose-built agent, graduate to a network of specialized agents with handoffs for complex tasks, and keep prompt instructions simple while running guardrails in parallel using fast models like GPT-4o mini for safety and reliability.

How to Improve Your Agents: Academic Lit Review

Feb 22, 2025 · 39:02

Joe from Columbia University and founder of Arklex AI explains research on AI agents, focusing on improving their reasoning and planning through self-reflection, test-time compute, and tree search methods, without relying on human supervision. He details the TriPath method, which uses larger models to edit smaller models' feedback for better self-improvement, achieving up to 48% accuracy on math benchmarks. He shows how multi-color tree search (MCTS) with contrastive reflection and multi-agent debate (RMCTS) outperforms other search methods on Visual Web Arena and OS World, achieving top non-trained results. He proposes exploratory learning, where models learn from search trajectories rather than optimal actions, improving performance under compute budgets. Finally, he presents the Arklex open-source agent framework, combining machine learning, systems, and security for practical multi-agent orchestration.

Reverse Conway's law and GenAI: How agents will take over the organisation - Patrick Debois

Feb 22, 2025 · 28:12

Patrick Debois argues that generative AI will reverse Conway's law, reshaping organizations around agents instead of human teams. He traces a progression from AI as a copilot to a team member, then a peer, and eventually a manager, with each stage unbundling human tasks and shrinking team sizes. Debois cites Amazon's AI pricing glitch as a reminder that humans remain needed for failure cleanup, and notes that LLMs mimic human collaboration behaviors, as shown in a multi-agent simulation. He warns of agent toxic behavior and the need for guardrails akin to human codes of conduct, while speculating that companies may replace SaaS with internally built AI services, making performance reviews and ROI calculations for agents inevitable. Debois advises engineers to focus on building the AI that builds their current work, not the work itself.

Your AI Agent Isn't an Engineer: The Art of Thoughtful Anthropomorphism

Feb 22, 2025 · 20:20

Rizel Scarlet, a staff developer advocate at Block, argues that marketing AI agents as "software engineers" creates unrealistic expectations and alienates developers. She explains that such anthropomorphism—attributing human traits to AI—stems from lazy marketing to executives, but it backfires by making developers feel threatened and discrediting products when AI fails to meet human-level performance. Citing a Wired article, she notes 52% of game companies have adopted generative AI, yet 30% of surveyed developers expressed negative sentiment. To build trust, she proposes a framework: understand how AI works (e.g., the agentic loop), use non-human names like "copilot" (pioneered by GitHub), focus on augmentation over replacement, be transparent, give developers control, show real demos (including live troubleshooting), provide clear documentation, and foster open collaboration (e.g., Cursor.Directory). The episode reframes AI as a tool to work in parallel, not a replacement.

The Adversarial Path to the Personal Assistant: Sumit Agarwal

Feb 15, 2025 · 18:46

Sumit Agarwal, founder of Ario AI, discusses building a personal AI assistant that uses adversarial ETL to extract users' data from services like Google, Amazon, and Doordash, enabling immediate personalization without manual input. He announces $16M in funding and demonstrates how the assistant generates data portraits, recommends travel and routines based on personal history, and manages schedules by detecting conflicts. Agarwal shares RAG insights: avoid LLMs for trivial math, use search and pre-processed profiles, and present the right data at the right time. Ario Boost, a browser extension, lets users download their data locally in developer mode without creating an account.

RAG at scale: production ready GenAI apps with Azure AI Search

Feb 13, 2025 · 21:53

Pablo, a Distinguished Engineer at Microsoft, explains how Azure AI Search enables production-grade RAG applications at scale by addressing key scaling challenges: data volume, query load, workflow complexity, and diverse data types. He demonstrates the platform's hybrid retrieval combining vector search (HNSW, exhaustive), keyword search (BM25), and filters, with a two-stage system using cross-encoder re-ranking for improved quality, achieving 100ms re-ranking latency. Azure AI Search supports multi-billion vector indexes through 10–12x storage limit increases and quantization (int8, single-bit) preserving 90–95% precision, with oversampling for full-precision re-ranking. Integrated vectorization automates ingestion from Azure sources (blob, Cosmos DB, OneLake) with change tracking and format handling (PDFs, Office documents, images). OpenAI uses Azure AI Search to back ChatGPT file uploads and the Assistants API, scaling user limits 500x after Microsoft's infrastructure upgrades.

AI Music Generation, From Prompt to Production: Phlo Young

Feb 11, 2025 · 54:34

Phlo Young, an AI engineer and musician, demystifies AI music generation by distinguishing text-to-music, audio-to-music, and style transfer, and demonstrates hands-on use of Suno and Udio to turn prompts into songs. He showcases examples like a Kanye West voice conversion, a Randy Travis AI comeback, and a BBL Drizzy meme song generated entirely from text. Phlo explains prompt engineering techniques, provides live access to paid accounts for attendees to generate unlimited music, and covers tools like Wave Tool for extracting stems. He also touches on ethical considerations and the RIAA lawsuits against Suno and Udio. The workshop equips beginners with practical skills to create and refine AI-generated music.

RAG and the MongoDB Document Model: Ben Flast

Feb 8, 2025 · 13:13

Ben Flast, Director of Product at MongoDB, explains how the MongoDB document model and Atlas Vector Search enable more efficient and scalable Retrieval Augmented Generation (RAG). He contrasts MongoDB's JSON-based documents with relational databases, highlighting that documents store application objects directly without stitching tables. Flast details MongoDB's HNSW-based vector search, which stores embeddings alongside transactional data in the same document, supporting up to 4,096 dimensions. He introduces Search Nodes, which decouple vector search infrastructure from transactional databases for independent scaling. Flast covers AI integrations with LangChain, LlamaIndex, and others, enabling features like semantic caching and chat history memory within a single database. He cites the startup 4149, which uses MongoDB to store user data, meeting notes, and vector embeddings for an AI teammate that assists with tasks by combining transactional and semantic search.

[Full Workshop] Llama 3 at 1,000 tok/s on the SambaNova AI Platform

Feb 7, 2025 · 1:00:58

Michelle Matern and Petro Milan of SambaNova present their full-stack AI platform, built on the SN40L RDU chip with a three-tiered memory architecture capable of storing up to 5 trillion parameters. They demonstrate Samba1 Composition of Experts (CoE), a trillion-parameter model combining 92 expert models behind a single endpoint, and show Llama-3-8B achieving 1,000 tokens per second with a time-to-first-token of 0.09 seconds and total inference time of 0.65 seconds—far exceeding GPU-based providers. The workshop includes a hands-on basic inference call using LangChain and SambaStudio API, and a RAG-based Q&A system for enterprise search that integrates Unstructured for document loading, E5-large-v2 embeddings, ChromaDB vector store, and the high-speed Llama-3 endpoint. Attendees learn to configure prompts with special Llama-3 tags, set chunk size and overlap, and optionally run embeddings on SambaNova's RDU hardware for faster processing.

Prompt Engineering Tactics: Dan Cleary

Feb 5, 2025 · 5:12

Dan Cleary, co-founder of PromptHub, presents three research-backed prompt engineering tactics to improve LLM output consistency and reduce hallucinations. Multipersona prompting, from University of Illinois, calls on multiple AI agents to collaborate on complex tasks like writing a book. The "according to" method, from Johns Hopkins, grounds prompts to a specific source (e.g., "according to Wikipedia") and can reduce hallucinations by up to 20%. The motion prompt, from Microsoft, adds emotional stimuli at the end of prompts, improving output accuracy by 8% to 115% depending on the task. These tactics are available as templates on PromptHub. Cleary emphasizes that even small changes in prompts can have outsized effects, crucial for maintaining user trust in AI-integrated products.

Scaling AI in Education: A Khanmigo case study: Shawn Jansepar

Feb 5, 2025 · 22:39

Shawn Jansepar, Khan Academy's Director of Engineering, details building Khanmigo, an AI tutor and teacher assistant on GPT-4 with OpenAI, arguing generative AI can democratize one-on-one tutoring. He describes a rapid prototyping culture that launched Khanmigo in three months via a company-wide hackathon, replacing traditional agile with a prototype-to-beta-to-launch framework. Technical challenges include math accuracy through a math agent and chain-of-thought prompting, refactoring prompts into a component architecture for testing, and managing scale using multiple models and dedicated Azure compute. Jansepar highlights ethical design: a Socratic tutor that avoids answers, teacher moderation, and a writing coach with revision history. Khanmigo now has 200,000 paid users, half in school districts, with teacher tools sponsored by Microsoft free to US teachers. Plans include releasing a math tutoring benchmark and evaluating smaller models like Phi-3.

Navigating Challenges and Technical Debt in LLMs Deployment: Ahmed Menshawy

Dec 31, 2024 · 16:15

Ahmed Menshawy, VP of AI Engineering at Mastercard, argues that deploying LLMs requires overcoming massive technical debt, with over 95% of the work outside the ML code. He details how Mastercard boosted fraud detection by 300% using LLMs while adhering to seven responsible AI principles. The episode covers RAG as a solution for hallucination, attribution, and access control, noting that 80% of organizational data is unstructured and 71% of companies struggle to manage it. Menshawy highlights that GPU compute for inference now exceeds training, and that fine-tuning open-source models enables joint training of retriever and generator. He also cautions against AGI hype, quoting Ada Lovelace and a Nature article urging focus on current AI risks.

Cooking with fire without burning down the kitchen: Dominik Kundel

Dec 31, 2024 · 20:35

Dominik Kundel, who leads product and design for Twilio's Emerging Tech & Innovation team, explains how the company balances disruptive AI innovation with its existing communications and customer data platform businesses. He distinguishes sustaining innovation (e.g., Apple's AI features) from disruptive innovation (e.g., agents not yet enterprise-ready due to quality and cost), arguing that ignoring disruptive AI is increasingly dangerous because quality improves daily. Kundel shares three key lessons from Twilio's AI journey: first, ship early and often—even rough prototypes—to gather real feedback, which led to the Twilio Alpha sub-brand for setting expectations; second, build a curious, problem-owning team rather than requiring existing AI expertise; third, share learnings internally and externally to avoid operating in a silo and to help customers be thought leaders. He recounts how an initial AI personalization engine was too disruptive for mainstream R&D, so the team iterated on an AI Assistance agent builder, using internal hackathons and dogfooding with low-risk use cases like IT helpdesk to improve quality before broader release.

E-Values Evaluating the Values of AI: Sheila Gulati and Nischal Nadhamuni

Dec 31, 2024 · 28:13

Sheila Gulati (Tola Capital) and Nischal Nadhamuni (Klarity CTO) argue that evaluating AI systems is at a seminal moment, requiring a shift from simplistic benchmarks to multi-faceted, user-centric approaches. Gulati highlights the inadequacy of current leaderboards, citing benchmark hacking and the need for dynamic datasets and value alignment. Nadhamuni shares how Klarity, which raised a $70M Series B, tackles eval challenges in its document-processing product—processing over 500K documents for customers with 15+ LLM use cases. He details practical strategies: front-loading UX risk, thinking backwards from user experience, reducing degrees of freedom by standardizing prompt engineering across customers, and building scrappy future-facing evals. The episode emphasizes that the AI engineer community must innovate benchmarks, bring depth and community sharing to evaluations, and introspect the values instilled in AI systems.

Hiring & Building an AI Engineering Team: Dr. Bryan Bischof

Dec 31, 2024 · 29:07

Dr. Bryan Bischof, Head of AI at Hex, argues that building an AI engineering team requires hiring based on product stage—starting with full-stack engineers and data profiles, then adding designers and later MLEs—while avoiding the 'mythical man month' trap in early AI products. He advocates for data intuition over LeetCode in interviews, using a take-home data exercise to assess candidates' ability to extract meaning from data and give feedback. Key attributes he looks for are curiosity, urgency, and product-mindedness, noting that enthusiasm alone is insufficient. Bischof also recommends working directly with domain experts to model AI behavior and suggests a centralized AI platform team to support multiple product teams, rather than having each team build AI infrastructure independently.

Real ROI: Lessons from Enterprises that have already succeeded with LLMs at Scale: Raza Habib

Dec 31, 2024 · 20:01

Raza Habib, CEO of Humanloop, shares lessons from enterprises like Duolingo, Filevine, and Ironclad that have achieved real ROI with LLMs at scale. He argues that success hinges on centering domain experts (e.g., Duolingo's linguists do all prompt engineering), needing less ML expertise than expected, and breaking down evaluation into small, testable components rather than chasing a single metric. Habib emphasizes collecting end-user feedback (acceptance rates, edits, thumbs up/down) and building tooling for logging, regression testing, and team collaboration. Filevine doubled revenue by launching six LLM-powered products; Ironclad's open-source Rivet logging tool enabled agents to auto-negotiate 50% of contracts. The talk covers how to optimize the four-component chain (base model, prompt, data selection, function calling) and why simpler systems will win as models improve.

Enhancing Quality and Security in CI: Gunjan Patel

Nov 27, 2024 · 18:27

Gunjan Patel, Director of Engineering at Palo Alto Networks, presents Ghost Pilot, an AI-powered CI pipeline that outsources boring software development tasks to enable self-evolving code. Unlike real-time Copilots, Ghost Pilot operates as a 'slow system' in CI, iteratively improving variable names and code comments, then generating unit tests by first listing edge cases and personalizing them via team context files. For security, it simulates three AI roles—Red Team engineer, developer, and engineering manager—who debate identified vulnerabilities, prioritize fixes by risk and effort, and propose changes with citations. Patel shares a reusable CI template with a Bring-Your-Own-LLM option, demonstrated by catching a logical Kubernetes bug missed by static analysis tools.

The AI emperor has no DAUs why most devs still don't use code AI: Quinn Slack

Nov 20, 2024 · 18:45

Quinn Slack, CEO and cofounder of Sourcegraph, argues that despite massive hype, only about 5% of professional developers actually use Code AI tools, with total recurring revenue from Code AI sitting at roughly $300 million ARR—a fraction of Salesforce's $36 billion. He cites GitHub's 1.3 million paid Copilot subscribers and just 935,000 yearly active users receiving suggestions, revealing the gap between perception and reality. Slack warns that the entire AI ecosystem—foundation models, infra, and applications—risks collapse if usage doesn't grow, and most revenue in AI flows to NVIDIA and chip makers, not software. From building Cody, the number two Code AI product, he shares lessons: hype fools everyone, autocomplete is a freakishly good feature that spoils expectations, while chat and agents are harder to vet and adopt. He advises builders to use their own product daily, ignore customer demands for buzzwords like fine-tuning, and manually build explicit interactions before adding magic. Slack concludes that the industry must collectively dehype and focus on real daily active users to turn the potential into sustained enterprise revenue.

Productionizing GenAI Models – Lessons from the world's best AI teams: Lukas Biewald

Oct 23, 2024 · 22:36

Lukas Biewald, founder of Weights and Biases, shares lessons from productionizing GenAI models, emphasizing that while AI is easy to demo, it is hard to productionize. He illustrates this with a personal project building a custom Alexa-like device using LLaMA and Whisper, where accuracy improved from 0% to 98% through prompt engineering, switching to Mistral, and fine-tuning with QLoRA. Biewald argues that tracking experiments (including failures) is critical for reproducibility and collaboration, and that a robust evaluation framework—beyond 'testing by vibes'—is essential for iterating and shipping v2. He notes that 70% of the audience had LLM apps in production, yet many lack solid evaluations, and recommends starting with lightweight prototypes and incorporating end-user feedback.

The era of unbounded products: Designing for Multimodal IO: Ben Hylak

Sep 25, 2024 · 20:32

Ben Hylak, founder of Dawn and former Apple Vision Pro designer, argues that the key to building intuitive AI products in the era of unbounded interfaces is adding structure—highlighting what matters, establishing hierarchy, and leveraging familiarity—lessons from designing VisionOS. He shows how successful AI apps like Dot, Perplexity, and Claude use structure (e.g., Claude pulling code into artifacts) while the Vercel chatbot's inline dynamic UI is an anti-pattern because it disrupts conversation flow. For agents, spreadsheets (like Clay) make unfamiliar multi-step tasks familiar. Looking ahead, Hylak predicts less prompt engineering via sparse autoencoders for millions of ranked, personalized presets, shifting product evaluation from evals to user analytics as apps become increasingly personalized.

10x Development: LLMs For the working Programmer - Manuel Odendahl

Aug 21, 2024 · 1:14:46

Manuel Odendahl presents a workshop on using LLMs as a translation engine to achieve 10x development productivity, arguing that treating models as cultural rather than purely technological artifacts enables novel techniques like world simulation for code reviews and domain-specific language creation. He demonstrates concrete methods: regenerating responses, editing model outputs, clearing context frequently, and summarizing transcripts into actionable formats. Odendahl introduces his tool Prompto for managing code fragments and emphasizes that fundamental programming knowledge remains essential while API-specific details become less critical. The episode focuses on decomposing problems into language translation steps, using examples such as transforming meeting transcripts into GitHub issues and building applications by instructing the LLM to simulate itself as the app. Odendahl advocates for divergent thinking and human-centered output, noting that all generated artifacts should ultimately serve human understanding.

Decoding the Decoder LLM without de code: Ishan Anand

Aug 9, 2024 · 17:08

Ishan Anand demonstrates how decoder-based LLMs like GPT-2 work internally using a fully functional spreadsheet implementation. He explains tokenization (e.g., 're-injury' splitting into 'rain' and 'injury' due to subword units), 768-dimensional embeddings, multi-headed attention where tokens look backward (e.g., 'he' focuses 0.48 on 'Mike'), and the multi-layer perceptron with matrix multiplies (MMULT). He shows residual connections creating an information superhighway and uses logit lens to reveal layer-by-layer predictions (e.g., 'Wednesday' appears early but only rises to top probability at later layers). Finally, he injects a sparse autoencoder feature vector for 'Jedi' into the residual stream, steering GPT-2 to output 'lightsaber' instead of 'phone', and contrasts this with alternative steering methods like representation engineering and activation steering.

How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou

Jul 31, 2024 · 18:42

Kevin Hou, AI engineer at Codeium, argues that embedding-based retrieval is hitting a ceiling for AI code generation and that Codeium breaks through by throwing large language models directly at the problem via a system called MQuery. He outlines the limitations of long context, fine-tuning, and embeddings, noting that embeddings are cheap but fail to reason over multiple documents needed for real-world coding tasks. Codeium developed a product-driven benchmark using recall 50 and commit-message datasets to measure retrieval quality. Because Codeium is vertically integrated—training custom models and building its own infrastructure—it can afford to run thousands of LLM calls in parallel per query, giving users 100x more compute than competitors. MQuery retrieves context from thousands of files in seconds, resulting in more accepted completions and better chat satisfaction. Hou compares this shift to autonomous driving: early heuristics like embeddings are giving way to large models that handle more data directly.

Fixing bugs in Gemma, Llama, & Phi 3: Daniel Han

Jul 31, 2024 · 17:42

Daniel Han of Unsloth details eight bugs found in Llama 3, including double BOS tokens, untrained tokens in the base model, and pad tokens equaling EOS tokens, which cause infinite generations. He explains how Unsloth automatically fixes these issues and offers a free Colab notebook for fine-tuning with Ollama. Han also covers tokenization fixes for Gemma and a sliding window bug in Phi-3, emphasizing the importance of correct chat templates and avoiding double BOS tokens. The episode provides concrete code examples and best practices for fine-tuning open-source LLMs to avoid common pitfalls.

Copilots Everywhere: Thomas Dohmke and Eugene Yan

Jul 26, 2024 · 18:22

GitHub CEO Thomas Dohmke shares the Copilot origin story from 2020 lockdown demos of GPT-3's Codex, its early 72 NPS despite hallucinations, and how it now writes half the code in enabled files. He details Copilot's expansion beyond IDE auto-completion to Chat and the new Copilot Workspace, which takes a GitHub issue and generates a plan and code changes, keeping humans in the loop. Dohmke argues Workspace helps non-developers like product managers see implementation details and size tasks. He defines an agent as an 'AI dishwasher' that autonomously handles drudge work like security backlogs, and urges developers to embrace AI tools to bring fun back into software development.

Lessons From A Year Building With LLMs

Jul 19, 2024 · 35:21

The six authors of the O'Reilly article "Lessons From A Year Building With LLMs" — Bryan Bischof, Jason Liu, Hamel Husain, Eugene Yan, Shreya Shankar, and Swix — argue that the model itself is not a moat and that success comes from continuous improvement centered on evals and data. They stress that AI engineers should treat models as SaaS, quickly swapping for better ones, and focus on product and user interactions. The talk warns against toxic practices like prematurely hiring ML engineers without data or blindly adopting tools, advocating instead for deliberate eval practice and data literacy. On tactics, they compare LLM-as-judge (quick to prototype) vs fine-tuned evaluators (more precise and faster), and emphasize looking at real user data regularly with automated guardrails. Ultimately, they conclude that going from demo to production requires sustained investment in infrastructure and evaluation, echoing MLOps lessons from a decade ago.

Open Challenges for AI Engineering: Simon Willison

Jul 17, 2024 · 18:49

Simon Willison argues the GPT-4 barrier has been broken as GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and open models like LLaMA 3 70B now compete, making GPT-4-class models a commodity. He highlights the AI trust crisis with examples of Dropbox and Slack being falsely accused of training on user data, and notes Anthropic trained Claude 3.5 Sonnet without customer data. Willison warns about prompt injection vulnerabilities, citing the Markdown image exfiltration bug affecting six major chatbots, and defines slop as unreviewed AI-generated content, calling for accountability and responsible use patterns.

The Making of Devin by Cognition AI: Scott Wu

Jul 11, 2024 · 20:04

Scott Wu, co-founder and CEO of Cognition AI, demonstrates Devin, a fully autonomous AI software engineer that builds and deploys working applications from plain English instructions, sharing the early lessons and vision behind the product. Devin built a name game website for the AI Engineer World's Fair in minutes, iterating on feedback to hide names, add a streak counter, and deploy the final version. Cognition uses Devin internally for features like a search bar and dashboards; Wu notes that engineers typically spend 80–90% of time on implementation, but Devin frees them to focus on architecture and problem-solving, making them 5–10x more productive while increasing overall demand for software. He also highlights the UX challenges of agents, which require new interactions beyond text completion, and notes that access to Devin is expanding via a waitlist.

From Text to Vision to Voice Exploring Multimodality with Open AI: Romain Huet

Jul 10, 2024 · 23:39

OpenAI's Romain Huet demonstrates GPT-4o's real-time audio, vision, and screen-sharing capabilities at the AI Engineer World's Fair, showing how the omni-model achieves human-like conversational latency, understands emotion, and can be interrupted naturally. He walks through live demos of ChatGPT Desktop recognizing drawings, reading book pages, and pair-coding a responsive travel app using Tailwind CSS. Huet also previews Sora's text-to-video generation and OpenAI's unreleased Voice Engine, which can clone a voice from seconds of audio and translate it into multiple languages while preserving the original speaker's tone. He outlines OpenAI's four focus areas: advancing textual intelligence, faster/cheaper models (GPT-4o is twice as fast as GPT-4 Turbo at half the price), model customization via fine-tuning, and enabling multimodal agents. The talk emphasizes that today's models are the dumbest they'll ever be and urges developers to build for a future where AI works across text, voice, and video.

How to Become an AI Engineer from a Fullstack Background - Reid Mayo

Feb 2, 2024 · 10:19

Reid Mayo presents a step-by-step syllabus to transition from fullstack engineer to AI engineer, covering generative AI foundations, prompt engineering, LangChain, fine-tuning, and cost-effective open-source model deployment. The syllabus starts with Cohere's LLM overview, then dives into prompt engineering via Elvis Seravia's guide and Learn Prompting org docs. It emphasizes LangChain as the glue layer for modular AI systems, with tutorials from Mayo Ocean. Evals are treated as software tests using OpenAI's cookbook. Fine-tuning is taught via OpenAI's cookbook and then open-source LLaMA 2, with a specific case study showing a $19 fine-tuned LLaMA 2 matching OpenAI's $24,000 model on a target task. The boot camp ends with advanced deep learning courses from FastAI and Hugging Face for further mastery.

Open Questions for AI Engineering: Simon Willison

Nov 25, 2023 · 24:33

Simon Willison recaps the AI industry's past year—from ChatGPT's breakthrough to open-source local models—and poses key open questions for AI engineering. He argues that ChatGPT's chat interface, while popular, is a poor fit for advanced use, urging better UIs like his command-line tool LLM. He celebrates Meta's Llama release as a 'stable diffusion moment' for language models and highlights the rise of small, locally-run models such as Replit's 3B model, asking how small models can remain useful. On security, he warns that prompt injection remains unsolved after 13 months, limiting what can safely be built. He champions ChatGPT's Code Interpreter (which he dubs 'Coding Intern') as the most exciting tool, able to write and compile C code on a phone, and argues that LLMs flatten the learning curve, making programming accessible to more people. He concludes by urging the community to build tools that enable anyone to automate tedious tasks.

Trust, but Verify: Shreya Rajpal

Nov 25, 2023 · 19:41

Shreya Rajpal, CEO of Guardrails AI, argues that large language models require a verification layer to compensate for their non-deterministic nature. She explains that while prototyping works, production apps fail due to hallucinations, prompt injections, and structural errors. Guardrails AI is an open-source framework that wraps LLMs with a validation suite: on output, it checks constraints like provenance (grounding in a source), profanity, and competitor mentions. On violation, it re-asks the model to self-correct or falls back to specified policies. Rajpal demonstrates with a chatbot example where a provenance guardrail catches a hallucinated password setting, then guides the model to a correct answer grounded in help center articles. The framework also supports custom validators, automatic prompt compilation from checks, and integration with external systems like sandboxed SQL databases for code generation.

Harnessing the Power of LLMs Locally: Mithun Hunsur

Nov 22, 2023 · 17:09

Mithun Hunsur presents llm.rs, a Rust library for running large language models locally, arguing it gives developers ownership, lower latency, and privacy compared to cloud APIs. He explains how quantization makes inference viable on consumer hardware, and shows that llm.rs supports architectures like LLaMA and Falcon through a unified interface. Practical code examples demonstrate customization, and community projects like LocalAI and LLMchain illustrate real-world use. Hunsur shares his own date-extraction pipeline, fine-tuning a small model with GPT-3 data to replace expensive cloud calls. He also cautions about hardware requirements, trade-offs between speed and quality, and ecosystem churn from rapid innovation.

Domain adaptation and fine-tuning for domain-specific LLMs: Abi Aryan

Nov 14, 2023 · 25:09

Abi Aryan's talk covers domain adaptation and fine-tuning for large language models, contrasting prompting, RAGs, and three fine-tuning methods: adaptive, behavioral, and parameter-efficient. Adaptive fine-tuning adds small adapter modules (0.15% of parameters) for new domains like biochemical engineering; behavioral fine-tuning optimizes label space for a single task; and parameter-efficient methods like LoRA and QLoRA reduce model size via low-rank adaptation and four-bit precision, ideal for low-resource devices. Aryan emphasizes data quality—deduplication reduces memorization—and practical tips: batch size of 32 or 64, starting with 100 epochs, using Adam optimizer, gradient checkpointing for memory savings, and in-context learning with dynamic examples to handle drift. Evaluation should combine metric-based (Bleu, Rouge), tool-based (Weights & Biases), model-based, and human-in-the-loop approaches, though full pipeline considerations (data collection, base model choice, storage) are critical for robust applications.

Building Reactive AI Apps: Matt Welsh

Nov 9, 2023 · 17:02

Matt Welsh announces AI.JSX, an open-source TypeScript framework he describes as 'React for LLMs,' which lets developers build reactive AI applications by composing JSX components that render to an LLM rather than the DOM. He argues AI.JSX makes LLM app development accessible to JavaScript developers, not just Python back-end engineers, by handling RAG, tool invocation, and real-time voice interaction out of the box. Welsh demonstrates parallel streaming of multiple LLM calls for story generation, a kid-safe wrapper component that rewrites content, and a 10-line RAG implementation. He showcases a live voice demo where an AI takes a donut order with sub-second latency, powered by Fixie's cloud platform for hosting, managed RAG pipelines, and conversational state. He emphasizes the framework's full React integration and ability to generate UI from AI, positioning AI.JSX as a simplification for building sophisticated LLM-powered apps.

[Workshop] AI Engineering 201: Inference

Nov 7, 2023 · 1:43:16

Charles Frye, instructor of the Full Stack LLM Bootcamp, leads a workshop on AI engineering inference, focusing on the build-versus-buy decision between proprietary and open models. He argues that proprietary models like OpenAI's GPT-4 and Anthropic's Claude are currently more capable but expensive, while open models like LLaMA 2 are less capable but offer hackability, though they may catch up if capabilities requirements saturate. Frye covers inference on end-user devices, noting that running models locally avoids network latency but faces tight memory and power constraints—e.g., a 7B parameter model requires 14 GB, too large for phone RAM. He explains inference-as-a-service (e.g., OpenAI, Replicate) versus self-serving on cloud GPUs or serverless platforms like Modal, highlighting that memory bandwidth is the bottleneck: GPUs have 1.5 TB/s memory bandwidth vs 312 TFLOPS compute, so batching is crucial for throughput. Frye also discusses inference arithmetic, custom silicon like TPUs which offer ~30% better efficiency but not drastic gains, and containerization challenges for GPU workloads.

The Hidden Life of Embeddings: Linus Lee

Nov 7, 2023 · 18:15

Linus Lee, a Research Engineer at Notion, presents a tour of embedding visualization and manipulation at the AI Engineer Summit 2023. He demonstrates an encoder-decoder model fine-tuned from T5 that can reconstruct text from embeddings, allowing direct manipulation of features like length and sentiment by moving in latent space. Lee shows that mixing embeddings by splicing dimensions from two texts produces a semantic blend, and a linear adapter can decode text from OpenAI's text-embedding-ada-002 embedding space. Using CLIP, he interpolates between photographic and cartoon images and performs vector arithmetic to modify facial expressions. Lee releases these custom text embedding models on Hugging Face, enabling others to explore and interact with latent spaces. He argues that making model internals visible and manipulable fosters deeper understanding and more humane interfaces to generative AI.

[Workshop] AI Engineering 101

Nov 6, 2023 · 3:02:24

In this hands-on workshop, AI engineer Noah Hein teaches the basics of AI engineering by building five projects with OpenAI's GPT-3, GPT-4, Dall-E, Whisper, and Telegram. Participants create a Telegram bot that uses GPT-3 for chat, implements retrieval-augmented generation (RAG) with embeddings and cosine similarity on MDN docs, generates code with GPT-4 using few-shot prompting, creates images via Dall-E 2, and transcribes voice with Whisper. The session explains core concepts like tokens, chunking, context windows, temperature, and top-p, emphasizing that embeddings and prompt engineering are key to performance. Hein demonstrates that these tools are cheap and accessible—the entire workshop costs about $0.05 in API fees—and shows how AI engineers can integrate multiple models into a single application.

The Intelligent Interface: Sam Whitmore & Jason Yuan of New Computer

Nov 2, 2023 · 18:56

Sam Whitmore and Jason Yuan of New Computer advocate for AI interfaces that adapt to human context by using implicit signals like proximity, gesture, and tone, rather than forcing users to adapt to rigid input modes. They demonstrate a pose-detection system that switches between keyboard and voice input based on user distance from the screen. They explore social gestures for rejecting calls and propose new physics metaphors like blending images by shaking an iPad. They argue that probabilistic materials like generative AI require familiar metaphors from nature and human behavior, and that mixing input and output modalities in real time yields more natural experiences. The demos, all speculative fictions, point toward future hardware where context-aware reasoning becomes the norm.

Building Blocks for LLM Systems & Products: Eugene Yan

Nov 2, 2023 · 17:24

Eugene Yan presents practical patterns for building LLM systems and products, emphasizing evals as the foundation. He argues that automated evals—starting with as few as 40 domain-specific questions—enable faster iteration and safer deployment, while academic benchmarks like MMLU may not fit real tasks. For retrieval-augmented generation, he warns that LLMs perform worse when the answer is in the middle of retrieved documents, and even with perfect retrieval, accuracy tops out at 75%. Guardrails against hallucination can be built via natural language inference at the sentence level or by sampling multiple summaries to check consistency. Finally, he highlights UX design—like Copilot’s accept/reject or Midjourney’s upscale/vary—as a way to collect implicit feedback that builds a data flywheel for evals and fine-tuning.

Building Context-Aware Reasoning Applications with LangChain and LangSmith: Harrison Chase

Nov 1, 2023 · 18:54

Harrison Chase, CEO of LangChain, argues that building context-aware reasoning applications on LLMs requires treating the model as part of a larger system, and he outlines key approaches and challenges. He categorizes context provision as instruction prompting, few-shot examples, retrieval-augmented generation (RAG), and fine-tuning. For reasoning, he describes a spectrum from single LLM calls to chains, routers, agents with cycles, and autonomous agents like Auto-GPT. Chase then details engineering challenges: choosing the right cognitive architecture, data engineering for context, prompt engineering across multi-step systems, evaluation using LLM-assisted metrics and user feedback, and enabling collaboration between technical and non-technical team members. He emphasizes that the field remains early and that LangChain and LangSmith help prototype, debug, and iterate on these systems.

See, Hear, Speak, Draw: Logan Kilpatrick & Simón Fishman

Oct 24, 2023 · 18:43

OpenAI's Simón Fishman and Logan Kilpatrick argue that 2024 will be the year of multimodal models, showcasing how text can currently bridge vision, audio, and image generation. They demo a DALL-E vision loop where GPT-4 with vision describes an image, DALL-E 3 generates a synthetic version, then the models compare and iterate to improve accuracy—all in roughly 50 lines of code. Another demo extracts frames from a video, uses GPT-4 with vision to describe them, combines with Whisper's transcript, and generates a blog post with matching DALL-E images, making video content more accessible. They emphasize that while today's multimodal capabilities are separate, future unified models will directly reason across modalities, eliminating the need for manual integration.

The Future of Work: Toran Bruce Richards, Silen Naihin et al

Oct 24, 2023 · 12:53

Toran Bruce Richards and Silen Naihin present Auto-GPT's vision for AI agents that augment human work, announcing 150k GitHub stars and $12M funding from Redpoint Ventures to build an open-source generalist agent. They unveil a forge template and dev tool built on the Agent Protocol, alongside a benchmark that tracks agent success rates from 35% to 55% over August. Safety is prioritized: prompt injection and agents deleting all JSON files on a laptop highlight the need for 100% reliability. The episode argues code agents are the stepping stone to AGI, with the mission to redefine the future of work by giving humans time back for creative tasks.