Devin

Can LLMs generate Enterprise Quality Code? — Prasenjit Sarkar, Sonar

May 31, 2026 · 15:12

Prasenjit Sarkar from Sonar evaluates whether LLMs generate enterprise-quality code using SonarQube analysis of 4,444 Java assignments across 53 models. While Gemini 3.1 Pro High achieves 84.17% pass rate, GPT-5.4 Pro High generates 1.2 million lines (high bloat), and Claude Sonnet 4.6 has 300 security issues per million lines. The ACDC framework addresses these gaps: Guide (Sonar Sweep and Context Augmentation), Verify (SonarQube Agentic Analysis in 1–5 seconds pre-commit), and Solve (Remediation Agent that fixes issues and checks regressions before presenting fixes). Sonar's leaderboard at sonar.com/leaderboard provides detailed pass rates, cyclomatic/cognitive complexity, bug density, and security metrics per model.

Scaling Agents on Kubernetes with acpx and ACP — Onur Solmaz, OpenClaw

May 21, 2026 · 19:00

Onur Solmaz, a founding engineer at textcortex and OpenClaw maintainer, built acpx, a headless CLI for the Agent Client Protocol (ACP), to automate processing of 300–500 daily AI-generated pull requests on OpenClaw. He replaces manual PTY scraping with structured agent-to-client communication, driving Codex sessions through a node-based workflow: reproduce the bug, judge the implementation, check for conflicts, run a review loop, and emit structured JSON. Solmaz runs parallel Codex sessions from five Discord channels while traveling, one channel per task. He also presents an open-source Kubernetes operator called Spritz (from textcortex) that provisions disposable full-compute pods per task, wires them into Slack, and tears them down when work is done, advocating for on-demand disposable agents over persistent chat apps.

Agents for Everything Else — swyx

May 1, 2026 · 14:10

swyx, co-founder of AI Engineer Conferences, details how his team of nine full‑time people uses coding agents like Devin and Town to run the conference, moving from a non‑AI stack to agents that handle everything from Figma‑to‑website conversion and schedule management to buying a lobster for the event. He claims these agents save “yak shaving” and increase human productivity because employees enjoy faster feedback cycles, leading to more work like animations and polish. The team now manages a 1,000‑person conference with plans to handle 6,000 in San Francisco without growing headcount. Swyx also advocates for “agents for everything else,” predicting that the primary user of software will shift from humans to bots, making APIs and MCPs more important than dashboards. He urges attendees to adopt agents and “prescribe” them to skeptical coworkers.

Automating Large Scale Refactors with Parallel Agents - Robert Brennan, OpenHands

Jan 8, 2026 · 1:16:21

Robert Brennan, CEO of OpenHands, and colleague Calvin explain how parallel agents can automate large-scale code refactors, demonstrating that breaking tasks into agent-sized batches with human oversight achieves 90% automation. They trace the evolution from context-unaware snippets to autonomous coding agents and now agent orchestration, where multiple agents work in parallel on tasks like CVE remediation — one client saw a 30x improvement in time-to-resolution. Calvin shows a pipeline using OpenHands' SDK to batch code files, run a verifier to detect code smells, and spin up fixer agents that generate focused pull requests. Brennan details strategies: task decomposition into single-PR-sized chunks, dependency ordering, and context sharing via agent.md files. The episode closes with a live-coding walkthrough building a parallel agent script that scans for vulnerabilities and opens individual PRs.

How Claude Code Works - Jared Zoneraich, PromptLayer

Dec 26, 2025 · 1:05:43

Jared Zoneraich from PromptLayer dissects how Claude Code works, arguing its success stems from a radical simplification: a single-threaded master loop with tool calls, relying on capable models rather than complex DAG-based architectures. He breaks down the core internal tools—Bash, FileEdit, Grep—and the to-do planning mechanism that is purely prompt-based and not deterministically enforced. Zoneraich contrasts Claude Code with other coding agents: Codex (Rust core, kernel-level sandboxing), Amp (handoff context management, model-agnostic), and Cursor Composer (UI-first, distilled model). He emphasizes that Bash is the most important tool, acting as a universal adapter, and that context management via sub-agents and summarization is critical. Key takeaways include trusting the model, simple design, rigorous tool testing, and that different agent philosophies (not one winner) will serve different use cases.

No More Slop – swyx

Dec 22, 2025 · 9:15

In this keynote at the AI Engineer Summit, host swyx declares war on slop — low-quality, inauthentic, inaccurate work produced by both humans and AI — arguing that the AI engineering community must elevate taste and accountability. He introduces Swix's law of anti-slop: the taste needed to fight slop scales with the plummeting cost of generating tokens. Swyx demonstrates how to combat slop using AI itself, citing examples like AI News (which tells readers to skip slow days), prompting techniques to avoid slop, and using sub-agents against context rot. He calls for rejecting autonomy without accountability and urges the audience to say "no more slop" to bosses demanding more lines of code, untested releases, and engagement bait.

Agent Reinforcement Fine Tuning – Will Hang & Cathy Zhou, OpenAI

Dec 9, 2025 · 16:55

Will Hang and Cathy Zhou of OpenAI's fine-tuning team introduce Agent Reinforcement Fine-Tuning (RFT), a method to improve AI agents by training them end-to-end on tasks involving tool calls and reasoning. They define an agent as a model that interleaves reasoning with external tool interactions, unlike regular models. The hierarchy of optimization moves from prompt engineering to task optimization to RFT, which changes model weights based on a custom reward signal. New features allow models to call tools via public endpoints and use custom rewards hosted externally. Case studies show concrete gains: Cognition improved code edit planning by 10 points with 1,000 examples and reduced tool call steps from 8–10 to 4; Codto's deep research agent boosted recall by 6% while cutting long-tail tool calls (over 15) down to 2–4; Cosine achieved state-of-the-art on enterprise code benchmarks by using strict graders that reward only pass-tested code; and Macco wrote GPU kernels from just 100 PyTorch prompts, beating SOTA by 72% after addressing reward hacking with seven edge-case detectors. Four principles for success: define tasks unambiguously, mirror production traffic in train/eval sets,…

What Data from 20m Pull Requests Reveal About AI Transformation — Nick Arcolano, Jellyfish

Nov 24, 2025 · 17:57

Nick Arcolano, head of research at Jellyfish, presents data from 20 million pull requests across 200,000 engineers at 1,000 companies to reveal what AI transformation actually looks like. Median AI adoption of interactive coding tools (Copilot, Cursor, Claude Code) jumped from 22% in June 2024 to nearly 90% today, driving a 2x increase in PR throughput and 24% faster cycle times on average. Autonomous agents like Devin and Codex remain negligible—only 44% of companies have tried them, contributing less than 0.2% of PRs. PRs grow 18% larger with AI, but bug rates and reverts show no significant quality impact. Crucially, code architecture determines success: centralized and balanced repos see 4x throughput gains, while highly distributed architectures (many repos per engineer) show no correlation with AI adoption, likely due to cross-repo context limitations. Arcolano advises focusing on adoption first, then addressing architectural barriers to unlock productivity.

How to Improve your Vibe Coding — Ian Butler

Aug 3, 2025 · 7:30

Ian Butler (CEO of BizMyth) presents benchmark data showing that popular coding agents like Devin and Cursor have less than 10% true positive rates for finding bugs, with Cursor producing a 97% false positive rate across 100+ repos and 1,200+ issues. He argues that these agents overrun codebases with false positives, causing alert fatigue and bugs reaching production. To improve vibe coding, he recommends adding bug-focused rules (e.g., OWASP Top 10, explicit bug classes like auth bypasses), managing context by feeding diffs and preventing file summarization, and using thinking models which were significantly better at finding deeper bugs. However, even thinking models show high variability in which bugs they find across runs, indicating agents never holistically examine a file. The talk concludes with a plug for BizMyth.sh, which automates PRs and vulnerability scanning.

Software Development Agents: What Works and What Doesn't - Robert Brennan, OpenHands

Jul 25, 2025 · 16:46

Robert Brennan, creator of the open-source coding agent OpenHands (formerly OpenDevin), argues that while coding is going away, software engineering remains about critical thinking, not typing. He explains that agents like OpenHands operate via a loop between an LLM and tools like a diff-based editor, terminal, and web browser, all sandboxed in Docker. Brennan advises starting small with rote tasks like merge conflicts (99% success) or PR feedback, being specific about frameworks and files, and always reviewing AI-generated code to avoid tech debt. He shares that 90% of his code now goes through the agent, with 10% requiring manual intervention, and warns against auto-merging without human accountability—OpenHands now assigns PRs to the human who triggered them. Key use cases include fixing failing tests, infrastructure changes, database migrations, and greenfield internal apps where vibecoding is acceptable.

Devin 2.0 and the Future of SWE - Scott Wu, Cognition

Jul 25, 2025 · 16:13

Scott Wu of Cognition AI describes an AI 'Moore's Law' for coding agents, with capability doubling every 70 days, propelling Devin from simple migrations to autonomous project work. He traces Devin's evolution through tiers: first repetitive migrations using Playbooks for instruction-following, then isolated bugs with VM snapshots, then multi-file changes with codebase intelligence, and finally backlog killing with self-testing. Each tier required solving different bottlenecks—memory, language server integration, iterative planning, and confidence calibration. Wu argues that the interface and key capabilities shift every 2-3 months, and predicts another 16-64x improvement in the next year.

Training Agentic Reasoners — Will Brown, Prime Intellect

Jul 7, 2025 · 19:17

Will Brown of Prime Intellect argues that reasoning and agents are fundamentally the same, and reinforcement learning (RL) is the key to advancing both. He explains that RL now works at scale, as shown by DeepSeek's GRPO and OpenAI's o3, and that agentic tasks like tool calling are natural RL environments. Brown warns against reward hacking and emphasizes designing evals that are harder to game than the task itself. He introduces his open-source toolkit 'verifiers' (now on pip) which lets users build trainable agent loops with a simple API, and demonstrates training a 7B Wordle agent in a few turns on just a couple GPUs.

Claude Code & the evolution of agentic coding — Boris Cherny, Anthropic

Jul 4, 2025 · 18:12

Boris Cherny, creator of Claude Code and Anthropic member of technical staff, argues that the product stays intentionally unopinionated and minimal because the model's coding capabilities are improving exponentially and the right UX remains unknown. He traces programming UX from 1950s punch cards to 1980s Smalltalk's live reload, Eclipse's static-analysis autocomplete, Copilot, and Devin's natural-language paradigm. Claude Code's terminal-first approach works in any terminal, over SSH, and inside VS Code or Cursor, with a GitHub integration that keeps data on user compute and a programmatic SDK for custom UIs. Key tips include using Claude for code-based Q&A (shortening onboarding from weeks to days), teaching it tools via `--help` and Claude MD files, leveraging TDD with visual iteration, and running multiple Claude instances in parallel via terminal tabs or GitHub Actions. Today's launch adds a plan mode triggered by Shift+Tab, which makes a plan and waits for approval before writing code.

From Mixture of Experts to Mixture of Agents with Super Fast Inference - Daniel Kim & Daria Soboleva

Jun 27, 2025 · 53:15

Daria Soboleva and Daniel Kim of Cerebras explain how Mixture of Experts (MoE) architectures enable scaling large language models efficiently by replacing monolithic feedforward networks with specialized experts, a technique used by GPT-4 and Claude. They then introduce Mixture of Agents (MoA), which combines multiple LLMs with custom prompts to outperform frontier models like GPT-4o on complex tasks, reducing a 293-second reasoning problem to 7.4 seconds using Cerebras' ultra-fast inference. The workshop guides participants to build their own MoA system, configure agents for bug fixing and performance optimization on a Python function, and achieve scores up to 120/120. Daniel details Cerebras' wafer-scale chip with 900,000 cores and distributed memory that eliminates memory bandwidth bottlenecks, enabling linear scaling and 15.5x faster inference on Llama 3.3-70B versus GPUs. Daria discusses ongoing research in diffusion models and sparsity, while Daniel notes plans for multimodal APIs and LoRA fine-tuning support.

The Agent Native Company — Rick Blalock, Agentuity

Jun 3, 2025 · 20:58

Rick Blalock of Agentuity argues that an agent-native company, built from the ground up with AI agents at the core of product, operations, and culture, is fundamentally different from an AI-enhanced one that merely uses AI as a tool. He contrasts the two: removing agents from an agent-native company would halt productivity, while an AI-enhanced business would just become less efficient. Blalock describes the agent-native workday, where humans oversee agents that handle routine tasks, and notes the rise of roles like 'Agent Manager' and the importance of AI fluency in hiring. He shares how his 7-person team built an entire agentic cloud infrastructure in weeks using agents like Devin, arguing that this paradigm shift requires founders to rethink org charts, roles, and skills. The episode concludes that businesses must decide whether they are just using AI or ready to be built around it.

The Devops Engineer Who Never Sleeps — Diamond Bishop, Datadog

Apr 22, 2025 · 16:18

Diamond Bishop, Director of AI Engineering at Datadog, explains how his team builds AI agents—the On-Call Engineer and Software Engineer—that automate DevOps tasks like incident investigation, remediation, and postmortem writing. He details how the On-Call Engineer wakes up for alerts, reads runbooks, queries logs and metrics, and suggests fixes to let human engineers sleep. The Software Engineer proactively fixes errors by generating code diffs and pull requests. Bishop shares four key lessons: scope tasks and evaluate rigorously, assemble teams of optimistic generalists and UX experts, adapt UX for human-agent collaboration, and treat observability as critical for debugging multi-step agent workflows. He predicts that within five years, AI agents will surpass humans as primary users of SaaS platforms like Datadog, urging builders to design for agent consumers.

Building and evaluating AI Agents — Sayash Kapoor, AI Snake Oil

Apr 17, 2025 · 20:00

Sayash Kapoor argues that current AI agents fall far short of their claimed performance due to flawed evaluation and a gap between capability and reliability. He cites failures like Do Not Pay (fined by FTC), LexisNexis (hallucinations in up to a third of cases), and Sakana AI (agent hacked reward functions, claiming 150x speedup that exceeded H100's theoretical max). Princeton's CoreBench shows best agents reproduce under 40% of papers. He emphasizes that agent benchmarks like SWE-bench mislead VC funding—Cognition's Devin succeeded on only 3 of 20 real-world tasks. Kapoor calls for cost-aware, multi-dimensional evaluation (e.g., Holistic Agent Leaderboard with Pareto frontiers) and a shift from capability to reliability engineering, drawing parallels to ENIAC's vacuum tube failures.

The Price of Intelligence - AI Agent Pricing in 2025

Feb 22, 2025 · 20:38

Shitej, co-founder and CTO of Orbe, argues AI agent pricing must continuously evolve, citing Intercom's 99 cent per resolution outcome model, Clay's prospecting credits, and Cursor's tiered usage limits. He stresses aligning pricing with target audience—SMB vs. enterprise—and maintaining simplicity and predictability. Cost structure is key: Character.AI optimized inference to support 100M DAUs, while Jasper leveraged a model decision engine to offer unlimited credits. Shitej emphasizes flexibility, noting OpenAI's price drops force repricing, and predicts 2025 will see more unlimited plans, outcome-based pricing with SLAs, and greater investment in pricing R&D for usage visibility.

Your AI Agent Isn't an Engineer: The Art of Thoughtful Anthropomorphism

Feb 22, 2025 · 20:20

Rizel Scarlet, a staff developer advocate at Block, argues that marketing AI agents as "software engineers" creates unrealistic expectations and alienates developers. She explains that such anthropomorphism—attributing human traits to AI—stems from lazy marketing to executives, but it backfires by making developers feel threatened and discrediting products when AI fails to meet human-level performance. Citing a Wired article, she notes 52% of game companies have adopted generative AI, yet 30% of surveyed developers expressed negative sentiment. To build trust, she proposes a framework: understand how AI works (e.g., the agentic loop), use non-human names like "copilot" (pioneered by GitHub), focus on augmentation over replacement, be transparent, give developers control, show real demos (including live troubleshooting), provide clear documentation, and foster open collaboration (e.g., Cursor.Directory). The episode reframes AI as a tool to work in parallel, not a replacement.

The Making of Devin by Cognition AI: Scott Wu

Jul 11, 2024 · 20:04

Scott Wu, co-founder and CEO of Cognition AI, demonstrates Devin, a fully autonomous AI software engineer that builds and deploys working applications from plain English instructions, sharing the early lessons and vision behind the product. Devin built a name game website for the AI Engineer World's Fair in minutes, iterating on feedback to hide names, add a streak counter, and deploy the final version. Cognition uses Devin internally for features like a search bar and dashboards; Wu notes that engineers typically spend 80–90% of time on implementation, but Devin frees them to focus on architecture and problem-solving, making them 5–10x more productive while increasing overall demand for software. He also highlights the UX challenges of agents, which require new interactions beyond text completion, and notes that access to Devin is expanding via a waitlist.

From Text to Vision to Voice Exploring Multimodality with Open AI: Romain Huet

Jul 10, 2024 · 23:39

OpenAI's Romain Huet demonstrates GPT-4o's real-time audio, vision, and screen-sharing capabilities at the AI Engineer World's Fair, showing how the omni-model achieves human-like conversational latency, understands emotion, and can be interrupted naturally. He walks through live demos of ChatGPT Desktop recognizing drawings, reading book pages, and pair-coding a responsive travel app using Tailwind CSS. Huet also previews Sora's text-to-video generation and OpenAI's unreleased Voice Engine, which can clone a voice from seconds of audio and translate it into multiple languages while preserving the original speaker's tone. He outlines OpenAI's four focus areas: advancing textual intelligence, faster/cheaper models (GPT-4o is twice as fast as GPT-4 Turbo at half the price), model customization via fine-tuning, and enabling multimodal agents. The talk emphasizes that today's models are the dumbest they'll ever be and urges developers to build for a future where AI works across text, voice, and video.