GitHub Copilot

On AI and Knowledge — Pablo Castro, Distinguished Engineer & CVP for AI Knowledge, Microsoft

Jul 17, 2026 · 17:35

Pablo Castro, Distinguished Engineer and CVP for AI Knowledge at Microsoft, argues that AI agents require three knowledge types—intrinsic (model training), extrinsic (RAG and grounding), and learned (optimization loops)—and demonstrates how Microsoft's Foundry IQ, Azure AI Search, and Agent Optimizer deliver them. He shows that combining retrieval methods outperforms individual ones, citing Azure AI Search evaluations where combined methods improve evidence recall and answer completeness. He introduces Foundry IQ as a layered system with agentic retrieval that reflects on data to satisfy information needs before returning results. Castro demonstrates creating a knowledge base that connects unstructured data, structured Parquet tables, and the web, then linking it to an agent via MCP server. He also showcases the Agent Optimizer, which uses hill climbing on baseline evaluations to automatically refine agent instructions, tool definitions, and skills based on runtime traces, enabling continuous learning loops that capture organizational differentiation.

From Writing Code to Designing Systems: How the Developer Role is Changing — Chris Noring, Microsoft

Jul 11, 2026 · 23:05

Chris Noring from Microsoft argues that the developer role is shifting from writing code to designing systems and orchestrating AI agents, using tools like GitHub Copilot and Claude. He proposes a workflow starting with the CLI rather than the editor, employing agents.md for high-level guidance, skills for repeatable tasks, and custom agents for orchestration. Noring demonstrates scaling by delegating tasks via /delegate in the CLI or assigning issues to agents in the GitHub UI, allowing developers to become 20x more productive. He stresses that guardrails are essential to prevent agents from producing 'slop', and that human-in-the-loop oversight remains critical. The episode emphasizes that developers must encode standards and constraints into their workflows to maintain consistency and quality at scale.

Building Interactive UIs in VS Code with MCP Apps — Marlene Mhangami & Liam Hampton, GitHub

Jun 6, 2026 · 16:06

Marlene Mhangami and Liam Hampton from GitHub explain MCP Apps, which let server tools return rich interactive UI components inside VS Code Chat via sandboxed iframes, replacing plain text responses. They demonstrate a flame graph profiler that profiles a Go app running bubble sort and Fibonacci, rendering an interactive flame graph directly in the chat window. The MCP server returns data and a resource reference to bundled HTML; VS Code fetches and renders it, enabling live interaction like clicking and querying without leaving chat. Use cases include data exploration, e-commerce checkouts (Shopify), and interactive diagrams (Excalidraw). The iframe is sandboxed for security—like putting a hamster in a cage to prevent it from modifying VS Code settings or calling external APIs. Marlene and Liam walk through building an MCP App from scratch using a skill from the model context protocol repository.

Let's Talk About FOMAT: Fear of Missing Agent Time — Michael Richman, Cmd+Ctrl

May 24, 2026 · 16:17

Michael Richman introduces FOMAT (Fear of Missing Agent Time), the anxiety of wasting agent productivity when you're away from your dev machine. He built Cmd+Ctrl, a system that sends push notifications when an agent stalls or finishes, lets you respond from phone or watch, and allows starting new sessions remotely. The demo shows Claude Code mirrored on an iPhone, with real-time interaction and a standup dashboard summarizing all sessions. The open-source daemon layer works with Claude Code, Cursor, Codex, Gemini CLI, and others, aggregating sessions across machines into a single pane of glass. Richman argues this alleviates the cognitive load of managing multiple agents and supports the new flow of agent choreography.

Cooking with Agents in VS Code — Liam Hampton, Microsoft

May 21, 2026 · 17:04

Liam Hampton from Microsoft demonstrates how to run three AI agents simultaneously in VS Code using GitHub Copilot: a local agent with Claude Opus for iterative unit test writing, a background agent using git work trees to build a front end from a GitHub issue with minimal oversight, and a cloud agent in GitHub Actions to add documentation and open-source files. He argues that VS Code serves as a single entry point for all three agent types—local for hands-on iteration, background for large tasks where partial involvement is acceptable, and cloud for tasks the developer doesn't need to touch. The talk walks through a live demo where the three agents work in parallel on one codebase, solving different problems (testing, UI, documentation) without interference, and explains the underlying infrastructure: cloud agents run securely in GitHub Actions with MCP servers and built-in safeguards, while local and background agents leverage VS Code's chat customizations and third-party extensions. Hampton also highlights the new modal for managing agents, custom instructions, skills, and MCP servers, positioning Copilot as a unified control plane for diverse AI workflows.

How Building with AI Can Double the Throughput of Your Engineering Team — Brian Scanlan, Intercom

May 15, 2026 · 21:49

Brian Scanlan of Intercom explains how treating Claude Code like a new hire—onboarding it to a 15-year-old Rails monolith, writing skills for every recurring task, and connecting it to production systems—doubled engineering PR throughput in under a year. By going all in on one platform instead of letting everyone choose their own tool, Intercom achieved a 17.6% automatic PR approval rate with SOC 2 sign-off, and its CI infrastructure collapsed under the volume. The key principle: give agents problems, not tasks. Scanlan recounts a security incident where Claude automatically pulled files, ran analysis, and handed back next steps in two minutes using a skill he didn't know existed. The talk emphasizes that all engineering work—debugging, testing, planning—should be agent-first, and that companies must invest in platform-level adoption and continuous skill improvement.

Mind the Gap (In your Agent Observability) — Amy Boyd & Nitya Narasimhan, Microsoft

May 14, 2026 · 1:20:07

Amy Boyd and Nitya Narasimhan of Microsoft explain how to close the gap between agent behavior and requirements using Microsoft Foundry's observability stack. They demonstrate tracing via OpenTelemetry, built-in evaluators for quality, safety, and agentic metrics (e.g., intent resolution, task adherence), and red teaming where a second AI attacks the agent to reveal vulnerabilities. The showcase is the observe skill: pointed at an agent with no eval data, it generates a dataset, runs batch evaluations, optimizes the prompt, compares versions, and rolls back to the best one—all from a single prompt. The skill surfaces failures developers didn't know existed, accelerating the optimize loop with human-in-the-loop guidance.

Software Engineering Is Becoming Plan and Review — Louis Knight-Webb, Vibe Kanban

May 2, 2026 · 20:23

Louis Knight-Webb, founder of Vibe Kanban, argues that software engineering is becoming all about planning and reviewing AI-generated code, not writing it. He presents two approaches: plan-heavy (spending time on specs to minimize review) and review-heavy (letting agents run and correcting output), noting five minutes of planning saves thirty minutes of review. As agents run longer—crossing the five-minute threshold—engineers must parallelize multiple agents and manage workflows, which Vibe Kanban aimed to facilitate. Knight-Webb announces shutting down Vibe Kanban despite 30,000 monthly active users because it resold tokens to enterprises and struggled to make money, advising that enterprise sales are crucial. He reflects on the journey positively, emphasizing learning to work with great people and the importance of hard work.

How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer

Apr 21, 2026 · 26:42

Gergely Orosz explains how AI is reshaping software engineering, from token maxing at Meta, Microsoft, and Salesforce—where engineers use AI tools to inflate token counts out of fear of layoffs and performance reviews—to the broader shift in the engineer's role toward orchestrating agents rather than managing people. He notes that big tech companies like Uber, Airbnb, and Shopify are building custom internal AI infra (MCP gateways, coding agents) to stay ahead, even tolerating high churn for a six-month competitive edge. Shopify secured GitHub Copilot a year early by offering feedback from 3,000 engineers. Gergely also shares how The Pragmatic Engineer hit product-market fit: 100 paid subscribers before publishing, reaching 1,000 in six weeks, and becoming the #1 paid tech newsletter by focusing on two deep-dive articles per week for two years.

Full Workshop: Build Your Own Deep Research Agents - Louis-François Bouchard, Paul Iusztin, Samridhi

Apr 20, 2026 · 1:57:03

Louis-François Bouchard, Samridhi Vaid, and Paul Iusztin built a combined system of a deep research agent and a deterministic writing workflow to automate high-quality technical content creation, trading flexibility for control. The research agent uses an MCP server (FastMCP) with tools for Google search, YouTube analysis, and report compilation, powered by Gemini and orchestrated by Claude Code. The writing workflow is a static pipeline that loads a user guideline, writing profiles (structure, terminology, character), and few-shot examples into a system prompt, then applies an evaluator-optimizer loop: a reviewer outputs structured pydantic objects with profile/location/comment to flag violations, and the editor applies them in priority order (guideline > research > profile). They stress that research demands autonomy while writing needs constraint, and show how to avoid AI slop by banning specific words and using profiles. For observability, they use Opik to capture all traces, and for evals they built a 20-sample dataset from real LinkedIn posts, split into train/dev/test, to calibrate an LLM judge (binary pass/fail) that they run on dev and test splits to measure F1 and detect…

Agentic Engineering: Working With AI, Not Just Using It — Brendan O'Leary

Apr 7, 2026 · 27:03

Brendan O'Leary, a Developer Relations Engineer at Kilo Code, argues that agentic engineering means shifting from using AI as a tool to working with it as a collaborator—like an energetic but inexperienced junior developer. He explains that context engineering—managing what goes into the model's context window—is critical, as too much or bad context degrades output quality. O'Leary recommends a research-plan-implement loop: first understand the system without writing code, then create a detailed plan, and finally execute in a fresh session to keep context lean. He advocates for configuring agents with role-based modes (Ask, Architect, Code), along with agents.md and skills files for persistent rules and reusable workflows. O'Leary also cautions against overusing MCP servers, which add token cost and can confuse the agent, and suggests isolating agent work via Git for easy review. The episode emphasizes that AI cannot replace thinking—it only amplifies the human's preparation and judgment.

Automating Large Scale Refactors with Parallel Agents - Robert Brennan, OpenHands

Jan 8, 2026 · 1:16:21

Robert Brennan, CEO of OpenHands, and colleague Calvin explain how parallel agents can automate large-scale code refactors, demonstrating that breaking tasks into agent-sized batches with human oversight achieves 90% automation. They trace the evolution from context-unaware snippets to autonomous coding agents and now agent orchestration, where multiple agents work in parallel on tasks like CVE remediation — one client saw a 30x improvement in time-to-resolution. Calvin shows a pipeline using OpenHands' SDK to batch code files, run a verifier to detect code smells, and spin up fixer agents that generate focused pull requests. Brennan details strategies: task decomposition into single-PR-sized chunks, dependency ordering, and context sharing via agent.md files. The episode closes with a live-coding walkthrough building a parallel agent script that scans for vulnerabilities and opens individual PRs.

Can you prove AI ROI in Software Eng? (Stanford 120k Devs Study) – Yegor Denisov-Blanch, Stanford

Dec 11, 2025 · 16:40

Stanford researcher Yegor Denisov-Blanch presents research based on 120,000+ developers across 600+ companies, arguing that AI ROI in software engineering is often negative despite apparent productivity gains. The study finds a median 10% productivity lift from AI, but a widening gap between top and bottom performers. Key drivers include codebase hygiene (environment cleanliness index with R² 0.40) and AI usage quality over volume. A company case study shows pull requests increased 14% but code quality dropped 9% and rework rose 2.5x, yielding no net effective output gain. Denisov-Blanch proposes a measurement framework using a primary metric (engineering output via ML model) paired with guardrail metrics, and emphasizes that companies can retroactively measure impact via git history without waiting for experiments.

Developing Taste in Coding Agents: Applied Meta Neuro-Symbolic RL — Ahmad Awais, CommandCode

Nov 24, 2025 · 20:52

Ahmad Awais, founder of Langbase, introduces CommandCode, a coding agent that learns a developer's personal taste and preferences to generate code that feels like their own. Built on Langbase's infrastructure (deploying over 350K agents and 1.2B monthly runs), CommandCode uses a Meta Neuro-Symbolic architecture with reinforcement learning to continuously capture a programmer's invisible architecture of choices—such as preferring TypeScript, PNPM, TSUP, and Commander over other tools—without requiring explicit rule files. Awais demonstrates how CommandCode automatically applies these learned preferences when building a CLI, contrasting it with Claude Code which generates generic output. He argues that taste models represent the next frontier, enabling agents to evolve with the developer and even share tastes across teams or borrow preferences from other developers like design engineers.

Real World Development with GitHub Copilot and VS Code — Harald Kirschner, Christopher Harrison

Aug 3, 2025 · 1:19:33

Harald Kirschner demonstrates three stages of Vibe Coding with GitHub Copilot and VS Code—YOLO, Structured, and Spectrum—arguing that developers can move from rapid prototyping to maintainable, scalable AI-assisted development. He shows how to use Agent mode, auto-approve settings, and new workspace scaffolding to build a hydration tracking app with React Vite and Material Design without looking at code. The talk covers custom instructions, MCP servers (Playwright, Jest Pad, Perplexity), and custom modes like TDD that enforce test-driven development by writing failing tests first. Kirschner explains how to constrain tools for deterministic behavior using tool sets and recommends committing often, pausing AI when needed, and iterating on instructions. He also discusses spec-driven development, critiquing specs with AI, and the importance of well-structured codebases for AI productivity.

Piloting agents in GitHub Copilot - Christopher Harrison, Microsoft

Jul 26, 2025 · 59:07

Christopher Harrison (Microsoft) demonstrates GitHub Copilot's agent capabilities for AI Engineers, arguing context is key to effective pair programming. He explains Copilot's modes—completion, chat, edit, local agent, and coding agent—and how Model Context Protocol (MCP) enables access to external tools like databases. Harrison shows how Copilot-instructions.md files provide project-specific guidance, and how coding agent runs within ephemeral GitHub Actions environments with strict security (no external access, only write to its own branch). He emphasizes that AI doesn't change DevOps fundamentals: tests still run locally and via PR workflows, and code review remains essential. The lab covers assigning issues to Copilot, configuring MCP servers, and using .instructions files for reusable patterns, with a repeat session at 3:30 PM.

Software Development Agents: What Works and What Doesn't - Robert Brennan, OpenHands

Jul 25, 2025 · 16:46

Robert Brennan, creator of the open-source coding agent OpenHands (formerly OpenDevin), argues that while coding is going away, software engineering remains about critical thinking, not typing. He explains that agents like OpenHands operate via a loop between an LLM and tools like a diff-based editor, terminal, and web browser, all sandboxed in Docker. Brennan advises starting small with rote tasks like merge conflicts (99% success) or PR feedback, being specific about frameworks and files, and always reviewing AI-generated code to avoid tech debt. He shares that 90% of his code now goes through the agent, with 10% requiring manual intervention, and warns against auto-merging without human accountability—OpenHands now assigns PRs to the human who triggered them. Key use cases include fixing failing tests, infrastructure changes, database migrations, and greenfield internal apps where vibecoding is acceptable.

Monetizing AI — Alvaro Morales, Orb

Jul 23, 2025 · 18:18

Alvaro Morales, CEO of Orb, argues that traditional subscription-based SaaS pricing fails for AI products due to rapid cost fluctuations, margin pressures, and customer demand for clear ROI. He presents three frameworks: decide between direct monetization (add-on, bundled) or indirect (free to drive upsells) using examples like GitHub Copilot, Notion AI, and Expedia; select a value metric from resource-based tokens (Vercel's v0) to task-based (Zapier) to outcome-based pricing (Intercom's Fin at $0.99 per resolved ticket); and embrace continual experimentation, as AI pricing cycles shrink from years to months. To reduce pricing guesswork, Orb built Simulations, a tool that lets companies backtest alternate pricing strategies on real usage data from closed betas before launch, helping teams like Replit and Perplexity optimize revenue without risking customer trust.

UX Design Principles for Semi Autonomous Multi Agent Systems — Victor Dibia, Microsoft

Jul 21, 2025 · 20:28

Victor Dibia, Principal Research Software Engineer at Microsoft Research, presents four UX design principles for semi-autonomous multi-agent systems: capability discovery, observability and provenance, interruptibility, and cost-aware action delegation. He demonstrates these through Blender LM, a multi-agent system built from scratch that translates natural language into 3D Blender scenes, featuring a planner and verify agent. Dibia stresses eval-driven design—starting with a non-AI baseline, then iterating with agents—and warns that only a small subset of tasks truly benefit from multi-agent autonomy, advocating for careful ROI assessment. He also showcases AutoGen Studio, a low-code tool for composing multi-agent workflows, and shares code and resources for further learning.

Production software keeps breaking and it will only get worse — Anish Agarwal, Traversal.ai

Jul 10, 2025 · 18:13

Anish Agarwal and Matthew Schoenbauer of Traversal.ai argue that as AI writes more code, production troubleshooting will become vastly harder, requiring a new approach combining causal machine learning, reasoning models, and agentic swarms to autonomously resolve incidents in minutes. They explain that traditional AI ops generates too many false positives, LLMs can't handle petabyte-scale data, and simple agents depend on deprecated runbooks. Their Traversal AI orchestrates thousands of parallel agentic tool calls to sift through trillions of logs and metrics, identifying root causes and citing observability data. A case study with DigitalOcean shows a 40% reduction in mean time to resolution (MTTR), with the system delivering findings in about five minutes. The episode details how this approach turns frantic incident Slack channels into autonomous, cited root-cause analysis, freeing engineers to focus on system design.

[Full Workshop] Vibe Coding at Scale: Customizing AI Assistants for Enterprise Environments

Jun 27, 2025 · 1:20:38

Harold from the VS Code team demonstrates "Vibe Coding at Scale" at the AI Engineer World's Fair, presenting three stages—YOLO, structured, and spectrum vibes—for customizing AI assistants in enterprise environments. He live-codes a hydration tracking app using GitHub Copilot's agent mode, auto-approve, and new workspace scaffolding, then shows how to enforce design principles via Copilot instructions and reusable prompts. Harold introduces custom modes (e.g., TDD mode), tool sets, and MCP servers like Playwright and Perplexity for research and browser testing. He emphasizes iterating on instructions, committing often, and using spec-driven development to balance speed with reliability, arguing these techniques enable true flow-state collaboration even on complex codebases.

Unlocking AI Powered DevOps Within Your Organization — Jon Peck, GitHub

Jun 27, 2025 · 22:13

Jon Peck, a developer advocate at GitHub, explains how organizations can unlock AI-powered DevOps by integrating tools like GitHub Copilot into their workflows, achieving up to 1.5x feature throughput and 30% average improvement in successful builds. He emphasizes starting with brownfield (existing code) rather than greenfield, using agent mode for iterative planning and narrowed context, and codifying team best practices via copilot-instructions.md files. Peck also covers governance: safety, privacy, org-wide policies, and excluding sensitive files. Autonomous AI use cases include auto-generating PR descriptions, code review, and assigning issues to Copilot, always keeping a human in the loop with isolated branches. Finally, MCP servers allow agents to interact with GitHub and other systems, automating commits and PR creation.

Vibe Coding at Scale: Customizing AI Assistants for Enterprise Environments - Harald Kirshner,

Jun 27, 2025 · 15:26

Harald Kirshner, a Principal Product Manager at Microsoft working on VS Code and GitHub Copilot, argues that 'vibe coding' must evolve from ad-hoc experimentation to structured, spec-driven approaches for enterprise environments. He breaks vibe coding into three stages: YOLO (rapid prototyping with auto-accept and voice dictation), structured (using templates, custom instructions, and tools like MCP for consistency), and spec-driven (iterating from a plan for scale and reliability). Kirshner demonstrates VS Code Insiders' agent mode, new workspace flow, and auto-approve features, emphasizing that developers should commit often, review agent pauses, and leverage workspace instructions to maintain quality. He advocates for standardizing AI usage through team-defined prompts and modes, preserving developer autonomy while ensuring maintainable code even in complex, polyglot codebases.

Collaborating with Agents in your Software Dev Workflow - Jon Peck & Christopher Harrison, Microsoft

Jun 27, 2025 · 1:04:06

GitHub's Christopher Harrison and Jon Peck explain how to collaborate with Copilot Coding Agent as a peer programmer, emphasizing that context—through clear code, comments, instructions files, and MCP servers—is key to success. They walk through a lab where Copilot works on assigned issues in ephemeral GitHub Actions environments without internet or write access to the main repo, creating draft PRs that require human review. Harrison advises against being passive-aggressive, advocating specific prompts and instructions files to guide the agent. The episode covers MCP's role in providing external tools, the need for code review and security checks even with AI, and pricing: $39.99/month for Enterprise tier with coding agent. They also highlight .instructions files for reusable patterns and note that coding agent currently supports only tools, not resources.

GraphRAG methods to create optimized LLM context windows for Retrieval — Jonathan Larson, Microsoft

Jun 27, 2025 · 15:09

Jonathan Larson of Microsoft Research presents GraphRAG methods, arguing that LLM memory with structure is a key enabler for effective AI applications, and that agents paired with these structures amplify their power. He demonstrates GraphRAG for Code, which uses graph-based understanding to outperform regular RAG on code comprehension, translation from Python to Rust, and even multi-file feature addition like adding jumping to the original DOOM codebase. Larson announces Benchmark QED, an open-source evaluation framework for measuring local and global query quality across comprehensiveness, diversity, empowerment, and relevance. He introduces Lazy GraphRAG, which beats vector RAG on 92% of local and 90% of global queries at one-tenth the cost of million-token context windows, and is being integrated into Azure Local and Microsoft Discovery.

Mastering Engineering Flow with Windsurf - Eashan Sinha, Windsurf

Jun 27, 2025 · 15:20

Eashan Sinha, a Deployed Engineer at Windsurf, explains how engineering teams can master agentic IDEs using Windsurf's Cascade agent. He argues that developers should treat AI as a teammate rather than a tool to abuse, emphasizing Windsurf's flow awareness—tracking user actions, edits, and terminal commands to infer intent. Sinha details Windsurf's evolution from copilots to agents, combining collaborative chat with autonomous tool calling, and introduces features like state-of-the-art context engine, workflows for deterministic steps, rules, memories, and simultaneous Cascades. He provides a practical four-step framework—explore, plan, build, test—for maximizing agentic flow, including using planning files, checkboxes, and automated test generation. The episode delivers a concrete guide to integrating AI into development without sacrificing quality, recorded at the AI Engineer World's Fair in San Francisco.

Building Protected MCP Servers — Den Delimarsky and Julia Kasper, MCP Steering Committee & Microsoft

Jun 20, 2025 · 20:13

Den Delimarsky and Julia Kasper, Microsoft engineers on the MCP Steering Committee, explain how the new MCP authorization spec simplifies securing remote MCP servers by separating the resource server from the authorization server, eliminating the need for developers to build custom token factories. They argue that developers shouldn't need to be security experts, so the spec offloads token minting and lifecycle management to standard identity providers like Entra ID, Okta, or Auth0. The client now handles the full OAuth 2.0 dance, while the server only validates tokens via a Protected Resource Metadata (PRM) document. A live C# SDK demo shows a server configured with minimal code, and an Azure API Management deployment demonstrates production-ready remote MCP servers with OAuth proxy and consent flows, including integration with VS Code and GitHub Copilot. The session also previews transforming REST APIs into MCP tools via Azure API Management.

The Many Ends of Programming - Ray Myers

Jun 10, 2025 · 27:12

Ray Myers, Chief Architect at All Hands AI, argues that the future of software engineering is not predetermined and we have a say, categorizing competing visions into six distinct endgames: extreme completion, dev apocalypse (innovator's paradise), abstraction leap, uncharted waters, review economy, and infinite pile of garbage. He illustrates extreme completion with a real example where an AI agent generated a 48-file pull request from a Slack message, and abstraction leap via domain-specific languages (DSLs) shown in the Parcel paper’s 61 prompts producing a 220-line Lisbon interpreter. Myers warns that dev apocalypse is desirable but not yet feasible, while the review economy is a pit stop where manual review bottlenecks must be managed using Theory of Constraints. He cites contradictory studies: UpLevel found developers with coding assistants had higher bug rates, while GitHub Copilot claimed improved quality; he notes 2024 was the first year copy-pasted code exceeded refactored code, a red flag for tech debt. Ultimately, Myers urges deliberate choices: whether we want no programmers or everyone to be a programmer, and whether we prioritize quality over quantity.

Creating Agents that Co-Create — Karina Nguyen, OpenAI

Apr 30, 2025 · 24:22

Karina Nguyen, an AI researcher at OpenAI and former Anthropic researcher, discusses two major AI scaling paradigms—next-token prediction (pretraining) and reinforcement learning on chain of thought—that shift AI from narrow tools to collaborative agents that co-create. Pretraining builds world understanding by predicting the next token, but hard tasks like math and creative writing require chain of thought reasoning, scaled with OpenAI’s O1 model. Post-training via RLHF and synthetic data enables rapid iteration, and the next stage is co-innovators: agents with reasoning, tool use, long context, and creativity. Nguyen shares product lessons from developing ChatGPT and Claude: 100K context via file uploads, ChatGPT Tasks for scheduled reminders that scale with model capabilities, and Canvas as a flexible interface for co-writing, coding, and research that can morph into an IDE, a tutor, or a data scientist. She envisions a future of invisible software creation where AI generates personalized, multimodal outputs on the fly, reducing reliance on clicking links, and the interface becomes a blank canvas that adapts to user intent.

Self Coding Agents — Colin Flaherty, Augment Code

Apr 21, 2025 · 17:23

Colin Flaherty, founding researcher at Augment Code, describes how the company built an AI coding agent that wrote over 90% of its own 20,000-line codebase with human supervision. The agent autonomously added third-party integrations like Google Search and Linear by searching its own codebase and even used its own Google Search tool to look up Linear API docs. It profiled and optimized itself, replacing synchronous file hashing with a process pool after adding print statements and running sub-copies. Flaherty argues that agent capabilities improve with better context engines, code execution environments, and test harnesses—adding test-driven iteration boosted a bug-fixing benchmark by 20% versus 4% from just upgrading the foundation model. He cautions that agents are general-purpose, not category-specific, and have different strengths than humans, making knowledge bases crucial for onboarding. With code becoming cheap, the focus shifts to product insights and design, while better tests enable greater autonomy and accelerate agent self-improvement.

WTF do people use Open Models for??

Feb 22, 2025 · 28:01

Eugene Cheah of Featherless.ai breaks down how individuals and enterprises actually use open-source AI models, based on platform data. DeepSeek R1 dominates individual usage, but Mistral Nemo 8B remains the top enterprise model due to production stickiness and Apache 2.0 licensing. Creative writing and roleplay account for 30–40% of all traffic, with over 60% of users in that segment being women; coding copilots and agents make up 20–30%, driven by 'vibe coding' and token-hungry workflows like Kline. RAG and ChatGPT clones represent 20%, while agentic workflows (10–20%) succeed with human-in-the-loop designs. Cheah advises enterprises to aim for 80% automation with escape hatches, and warns against chasing 100% reliability. He concludes by introducing Quirky, a post-transformer hybrid built for $100k.

Your AI Agent Isn't an Engineer: The Art of Thoughtful Anthropomorphism

Feb 22, 2025 · 20:20

Rizel Scarlet, a staff developer advocate at Block, argues that marketing AI agents as "software engineers" creates unrealistic expectations and alienates developers. She explains that such anthropomorphism—attributing human traits to AI—stems from lazy marketing to executives, but it backfires by making developers feel threatened and discrediting products when AI fails to meet human-level performance. Citing a Wired article, she notes 52% of game companies have adopted generative AI, yet 30% of surveyed developers expressed negative sentiment. To build trust, she proposes a framework: understand how AI works (e.g., the agentic loop), use non-human names like "copilot" (pioneered by GitHub), focus on augmentation over replacement, be transparent, give developers control, show real demos (including live troubleshooting), provide clear documentation, and foster open collaboration (e.g., Cursor.Directory). The episode reframes AI as a tool to work in parallel, not a replacement.

How Coding Agents change Software Development Forever - Hailong Zhang

Feb 22, 2025 · 8:50

Hailong Zhang presents how coding agents, particularly the unit test agent Gru, transform software development by automating routine tasks while humans focus on creative work. In the future workflow, synchronous agents like GitHub Copilot and Cursor assist in real-time, while asynchronous agents like Gru autonomously handle tasks such as writing tests, fixing bugs, and submitting pull requests. Gru, built to boost unit tests, detects code changes from pull requests, generates and runs tests, and submits a new PR with a summary and coverage improvement. In production, over 50% of Gru's PRs are accepted by humans, and it handles 80% of unit tests in its own repo, making it the top contributor. To build such agents, Zhang emphasizes defining a clear problem (e.g., unit tests, not generic software engineering), creating datasets and evaluation harnesses, selecting and fine-tuning LLMs per stage, building task-specific context from environment data, and developing an agent operating system to reuse common infrastructure across different tasks.

[Full Workshop from Microsoft] Github Copilot - The World's Most Widely Adopted AI Developer Tool

Feb 7, 2025 · 1:19:45

This workshop from the AI Engineer World's Fair features GitHub's Christina Warren, Dave, Alex, and Harald presenting GitHub Copilot, the world's most widely adopted AI developer tool. They demonstrate Copilot's three interaction modes: ghost-text completions using GPT-3.5 for speed, inline chat (also GPT-3.5) for quick code edits, and the chat panel powered by GPT-4 Turbo for deeper conversations. Speakers emphasize prompt crafting—being specific, providing examples, and keeping relevant files open—to improve results. They show how to use slash commands (/fix, /explain, /test) and the new attach button for explicit context. The workshop uses GitHub Codespaces preconfigured with Python 3.11 and Copilot extensions; a coupon provides a 7-day free trial. Harald explains the trade-off between model quality and latency, and notes upcoming features like automatic agent delegation and workspace integration for cross-file edits.

GitHub Copilot: The World's Most Widely Adopted AI Developer Tool

Feb 6, 2025 · 29:49

GitHub Senior DevOps Advocate Dave Bernason demonstrates how GitHub Copilot has evolved from an AI code generator to a comprehensive developer assistant with chat, enterprise knowledge bases, pull request summaries, and third-party extensions, emphasizing that Copilot keeps developers in flow and requires human oversight. He shows Copilot Chat explaining code, refactoring, debugging, and generating unit tests. Copilot Enterprise adds Bing search, knowledge bases from Markdown files, and repo indexing for accurate answers, exemplified by updating a hard-coded sales tax function to use the Avalara API. Copilot Extensions integrate third-party tools like Octopus Deploy for deployment dashboards within chat. Bernason also covers prompt engineering tips—specificity improves suggestions—and highlights Copilot's support for any language, even COBOL, for modernization.

Cooking with fire without burning down the kitchen: Dominik Kundel

Dec 31, 2024 · 20:35

Dominik Kundel, who leads product and design for Twilio's Emerging Tech & Innovation team, explains how the company balances disruptive AI innovation with its existing communications and customer data platform businesses. He distinguishes sustaining innovation (e.g., Apple's AI features) from disruptive innovation (e.g., agents not yet enterprise-ready due to quality and cost), arguing that ignoring disruptive AI is increasingly dangerous because quality improves daily. Kundel shares three key lessons from Twilio's AI journey: first, ship early and often—even rough prototypes—to gather real feedback, which led to the Twilio Alpha sub-brand for setting expectations; second, build a curious, problem-owning team rather than requiring existing AI expertise; third, share learnings internally and externally to avoid operating in a silo and to help customers be thought leaders. He recounts how an initial AI personalization engine was too disruptive for mainstream R&D, so the team iterated on an AI Assistance agent builder, using internal hackathons and dogfooding with low-risk use cases like IT helpdesk to improve quality before broader release.

AI Platform Engineering: Patrick Debois

Dec 31, 2024 · 28:18

Patrick Debois, who coined DevOps in 2009, argues scaling GenAI requires an AI platform team, mirroring the cloud and DevOps pattern. He lists shared infrastructure: model access, vector databases, RAG connectors, version control, proxy, observability, monitoring, caching, feedback services, and stresses enablement through prototyping and frameworks. He warns of pitfalls like chasing GenAI without use case, overfocus on fine-tuning, and cost obsession. Citing the Ironies of GenAI Automation, he notes that as engineers shift from producing to reviewing code—copying co-pilot suggestions more on weekends—they risk losing situational awareness. For governance, he advises awareness programs, opt-out training, license checks, and layered guardrails: central rules plus team-specific overlays. He recommends combining cloudops, secops, devx, data platform, and AI platform teams for collaboration.

Real ROI: Lessons from Enterprises that have already succeeded with LLMs at Scale: Raza Habib

Dec 31, 2024 · 20:01

Raza Habib, CEO of Humanloop, shares lessons from enterprises like Duolingo, Filevine, and Ironclad that have achieved real ROI with LLMs at scale. He argues that success hinges on centering domain experts (e.g., Duolingo's linguists do all prompt engineering), needing less ML expertise than expected, and breaking down evaluation into small, testable components rather than chasing a single metric. Habib emphasizes collecting end-user feedback (acceptance rates, edits, thumbs up/down) and building tooling for logging, regression testing, and team collaboration. Filevine doubled revenue by launching six LLM-powered products; Ironclad's open-source Rivet logging tool enabled agents to auto-negotiate 50% of contracts. The talk covers how to optimize the four-component chain (base model, prompt, data selection, function calling) and why simpler systems will win as models improve.

Enhancing Quality and Security in CI: Gunjan Patel

Nov 27, 2024 · 18:27

Gunjan Patel, Director of Engineering at Palo Alto Networks, presents Ghost Pilot, an AI-powered CI pipeline that outsources boring software development tasks to enable self-evolving code. Unlike real-time Copilots, Ghost Pilot operates as a 'slow system' in CI, iteratively improving variable names and code comments, then generating unit tests by first listing edge cases and personalizing them via team context files. For security, it simulates three AI roles—Red Team engineer, developer, and engineering manager—who debate identified vulnerabilities, prioritize fixes by risk and effort, and propose changes with citations. Patel shares a reusable CI template with a Bring-Your-Own-LLM option, demonstrated by catching a logical Kubernetes bug missed by static analysis tools.

The AI emperor has no DAUs why most devs still don't use code AI: Quinn Slack

Nov 20, 2024 · 18:45

Quinn Slack, CEO and cofounder of Sourcegraph, argues that despite massive hype, only about 5% of professional developers actually use Code AI tools, with total recurring revenue from Code AI sitting at roughly $300 million ARR—a fraction of Salesforce's $36 billion. He cites GitHub's 1.3 million paid Copilot subscribers and just 935,000 yearly active users receiving suggestions, revealing the gap between perception and reality. Slack warns that the entire AI ecosystem—foundation models, infra, and applications—risks collapse if usage doesn't grow, and most revenue in AI flows to NVIDIA and chip makers, not software. From building Cody, the number two Code AI product, he shares lessons: hype fools everyone, autocomplete is a freakishly good feature that spoils expectations, while chat and agents are harder to vet and adopt. He advises builders to use their own product daily, ignore customer demands for buzzwords like fine-tuning, and manually build explicit interactions before adding magic. Slack concludes that the industry must collectively dehype and focus on real daily active users to turn the potential into sustained enterprise revenue.

How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou

Jul 31, 2024 · 18:42

Kevin Hou, AI engineer at Codeium, argues that embedding-based retrieval is hitting a ceiling for AI code generation and that Codeium breaks through by throwing large language models directly at the problem via a system called MQuery. He outlines the limitations of long context, fine-tuning, and embeddings, noting that embeddings are cheap but fail to reason over multiple documents needed for real-world coding tasks. Codeium developed a product-driven benchmark using recall 50 and commit-message datasets to measure retrieval quality. Because Codeium is vertically integrated—training custom models and building its own infrastructure—it can afford to run thousands of LLM calls in parallel per query, giving users 100x more compute than competitors. MQuery retrieves context from thousands of files in seconds, resulting in more accepted completions and better chat satisfaction. Hou compares this shift to autonomous driving: early heuristics like embeddings are giving way to large models that handle more data directly.

Lessons From A Year Building With LLMs

Jul 19, 2024 · 35:21

The six authors of the O'Reilly article "Lessons From A Year Building With LLMs" — Bryan Bischof, Jason Liu, Hamel Husain, Eugene Yan, Shreya Shankar, and Swix — argue that the model itself is not a moat and that success comes from continuous improvement centered on evals and data. They stress that AI engineers should treat models as SaaS, quickly swapping for better ones, and focus on product and user interactions. The talk warns against toxic practices like prematurely hiring ML engineers without data or blindly adopting tools, advocating instead for deliberate eval practice and data literacy. On tactics, they compare LLM-as-judge (quick to prototype) vs fine-tuned evaluators (more precise and faster), and emphasize looking at real user data regularly with automated guardrails. Ultimately, they conclude that going from demo to production requires sustained investment in infrastructure and evaluation, echoing MLOps lessons from a decade ago.

The Making of Devin by Cognition AI: Scott Wu

Jul 11, 2024 · 20:04

Scott Wu, co-founder and CEO of Cognition AI, demonstrates Devin, a fully autonomous AI software engineer that builds and deploys working applications from plain English instructions, sharing the early lessons and vision behind the product. Devin built a name game website for the AI Engineer World's Fair in minutes, iterating on feedback to hide names, add a streak counter, and deploy the final version. Cognition uses Devin internally for features like a search bar and dashboards; Wu notes that engineers typically spend 80–90% of time on implementation, but Devin frees them to focus on architecture and problem-solving, making them 5–10x more productive while increasing overall demand for software. He also highlights the UX challenges of agents, which require new interactions beyond text completion, and notes that access to Devin is expanding via a waitlist.

Trust, but Verify: Shreya Rajpal

Nov 25, 2023 · 19:41

Shreya Rajpal, CEO of Guardrails AI, argues that large language models require a verification layer to compensate for their non-deterministic nature. She explains that while prototyping works, production apps fail due to hallucinations, prompt injections, and structural errors. Guardrails AI is an open-source framework that wraps LLMs with a validation suite: on output, it checks constraints like provenance (grounding in a source), profanity, and competitor mentions. On violation, it re-asks the model to self-correct or falls back to specified policies. Rajpal demonstrates with a chatbot example where a provenance guardrail catches a hallucinated password setting, then guides the model to a correct answer grounded in help center articles. The framework also supports custom validators, automatic prompt compilation from checks, and integration with external systems like sandboxed SQL databases for code generation.

Harnessing the Power of LLMs Locally: Mithun Hunsur

Nov 22, 2023 · 17:09

Mithun Hunsur presents llm.rs, a Rust library for running large language models locally, arguing it gives developers ownership, lower latency, and privacy compared to cloud APIs. He explains how quantization makes inference viable on consumer hardware, and shows that llm.rs supports architectures like LLaMA and Falcon through a unified interface. Practical code examples demonstrate customization, and community projects like LocalAI and LLMchain illustrate real-world use. Hunsur shares his own date-extraction pipeline, fine-tuning a small model with GPT-3 data to replace expensive cloud calls. He also cautions about hardware requirements, trade-offs between speed and quality, and ecosystem churn from rapid innovation.

Building Blocks for LLM Systems & Products: Eugene Yan

Nov 2, 2023 · 17:24

Eugene Yan presents practical patterns for building LLM systems and products, emphasizing evals as the foundation. He argues that automated evals—starting with as few as 40 domain-specific questions—enable faster iteration and safer deployment, while academic benchmarks like MMLU may not fit real tasks. For retrieval-augmented generation, he warns that LLMs perform worse when the answer is in the middle of retrieved documents, and even with perfect retrieval, accuracy tops out at 75%. Guardrails against hallucination can be built via natural language inference at the sentence level or by sampling multiple summaries to check consistency. Finally, he highlights UX design—like Copilot’s accept/reject or Midjourney’s upscale/vary—as a way to collect implicit feedback that builds a data flywheel for evals and fine-tuning.