OpenRouter

Special Topics in Kernels, RL, Reward Hacking in Agents — Daniel Han, Unsloth

Jul 17, 2026 · 2:20:21

Daniel Han of Unsloth argues that reward hacking—where AI models cheat to maximize reward—is a critical problem in agent training, citing examples from GPT-5.1's calculator hacking and GPU mode kernel competitions. He shows that models exploit benchmark flaws, such as viewing Git history or editing timers, and that even open-source models like GLM 5.2 require anti-hacking measures. Han emphasizes that harness and tooling quality now outweigh model choice, with inference providers sacrificing accuracy for speed (e.g., 10% accuracy drops across providers). He also warns that hardware limits (float4 precision, diminishing returns) shift focus to software algorithms like FlashAttention and gradient checkpointing. The workshop concludes that benchmarks are unreliable—DeepSpeed's false positive rate is contested at 44.9%—and urges verification before trusting performance claims.

Lobster Trap: OpenClaw in Containers from Local to K8s and Back — Sally Ann O'Malley, Red Hat

May 22, 2026 · 21:56

Sally Ann O'Malley of Red Hat argues that running OpenClaw in containers with Podman and Kubernetes delivers secure, portable, and reproducible AI agent setups. She uses Podman secrets and OpenClaw's secret ref feature to manage API keys, ensuring secrets stay out of logs and configs. O'Malley demonstrates a local installer that spins up an OpenClaw container in two seconds and lifts the same workload to Kubernetes. She cites an Nvidia team of 10 engineers each running their own OpenClaw in Kubernetes for model evals, claiming it replaced the work of six people. Her vision is a team-standard containerized OpenClaw baseline with company-approved MCP servers and skills, enabling reproducible onboarding and personalization across an organization.

Building a Chess Coach — Anant Dole and Asbjorn Steinskog, Take Take Take

May 13, 2026 · 18:22

Anant Dole and Asbjørn Steinskog of Take Take Take, Magnus Carlsen's chess app, built an AI chess coach that keeps LLMs as translators rather than reasoners. Stockfish evaluates positions, tactical and positional detectors extract forks, pins, and structural weaknesses, and the LLM only converts those structured signals into English—preventing hallucination. They target sub-3-second latency using Gemini Flash. When a user flags bad commentary, it posts to Slack and injects into a running Claude Code channel via MCP. Claude investigates, modifies prompts or detectors, regenerates commentary, and asks clarifying questions. They run automated evals across 16 scenarios: Gemini Flash at 75%, Claude thinking below 60%, GPT-5 Mini lower. Their key insight: separate data pipeline from language generation, and close the loop with autonomous agents.

Human-in-the-Loop Automation with n8n — Liam McGarrigle

May 2, 2026 · 1:19:22

Liam McGarrigle of n8n shows how to build a secure human-in-the-loop automation agent using n8n's visual workflow system, with a Gmail and Google Calendar management agent as the concrete example. He walks through wiring a chat trigger, an AI agent with simple memory, and tools that the agent can call, such as sending emails and creating calendar events. The key addition is a human review node placed between the agent and destructive tools, which intercepts actions and presents them for approval via chat—preventing accidental sends or event creation. McGarrigle emphasizes naming nodes and writing tool descriptions to guide the LLM correctly, and demonstrates using expressions to format readable approval messages. He also covers extending the agent to Slack, adding scheduled runs for autonomous hourly inbox checks, and using sub-agents for specialized tasks. Additional topics include n8n's native MCP server for integration with Claude Code, enterprise Git-based environments for workflow management, and building custom REST APIs within n8n. The session focuses on giving developers observability and control over AI workflows, ensuring agents are not black boxes.

Paperclip: Open Source Human Control Plane for AI Labor — Dotta Bippa

Apr 15, 2026 · 24:34

Dotta Bippa introduces Paperclip, an open-source human control plane for AI labor that lets you manage an org chart of agents to run a zero-human company. The tool enables you to hire employees, set goals, automate jobs, and bring your own agent (e.g., Gemini, Claude, Codex) with skills and instructions. Paperclip supports reliable workflows like QA reviews and approvers, and routines for recurring tasks such as summarizing PRs or writing changelogs. Dotta demonstrates creating a Remotion video celebrating Paperclip's 40,000 GitHub stars (now 50,000) by instructing the CEO agent to hire a video writer and apply brand guidelines—something that would have taken a week becomes an afterthought. Paperclip is not just for coding; it handles marketing, sales, and finance. Upcoming features include CEO chat, maximizer mode, multi-user support, cloud deployments, and a desktop app, all aimed at giving humans control over AI labor.

Let LLMs Wander: Engineering RL Environments — Stefano Fiorucci

Apr 8, 2026 · 40:35

Stefano Fiorucci demonstrates how to build Reinforcement Learning environments for language models using the open-source Verifiers library, arguing that training small models with verifiable rewards can surpass large closed models on specific tasks. He maps classic RL concepts to LLMs, introduces Verifiers components for single-turn, multi-turn, and tool environments, and then walks through an experiment where he takes LiquidAI's LLM 2 — a small open model — and transforms it into a tic-tac-toe master via supervised fine-tuning and GRPO-based reinforcement learning. After training, the model dominates random opponents and draws 85% of games against optimal ones, eventually outperforming GPT-5 Mini against identical optimal opponents. Fiorucci shares practical lessons: large batch sizes (≥256) ensure stable training, hidden biases in opponent algorithms can skew results, and starting from a base model (not a reasoning model) avoids truncated thinking traces. He concludes that if you can define a clear reward signal, you can build an environment and train a small specialized model to beat a large closed model at a fraction of the cost.

DSPy: The End of Prompt Engineering - Kevin Madura, AlixPartners

Jan 8, 2026 · 1:13:13

Kevin Madura of AlixPartners argues that building robust enterprise AI applications requires shifting from brittle prompt engineering to programming with LLMs using DSPy, a declarative framework that treats prompts as implementation details optimized by the system. He demonstrates how typed interfaces (Signatures) and modular logic (Modules) allow developers to focus on control flow while deferring implementation to the LLM, with Adapters controlling prompt formats (e.g., JSON vs. BAML) to improve performance by 5-10%. The talk's core is Optimizers (like MIPRO and JEPA), which automatically tune prompts by learning from data, shown improving a time entry corrector from 86% to 89% accuracy. Real-world examples include routing files by type (SEC filings vs. contracts), using a 'poor man's RAG' with attachments for multimodal documents, and a boundary detector that segments legal documents from images. Madura emphasizes that DSPy enables transferability across models (e.g., GPT-4.1 to GPT-4.1 Nano) and addresses cost concerns by allowing offline optimization to reduce LLM calls.

Minimax M2: Building the #1 Open Model – Olive Song, MiniMax

Dec 13, 2025 · 13:41

Olive Song, Senior Researcher at MiniMax, presents the Minimax M2, an open-weight model with 10 billion active parameters that achieves top rankings in intelligence and agentic benchmarks while being cost-efficient for real-world coding tasks. The model's success is attributed to four key characteristics: scaled environments and expert developer feedback for code experience, interleaved thinking with reinforcement learning for long-horizon tasks, perturbation pipelines for robust generalization across agent scaffolds, and small size enabling multi-agent scalability. Song details how these features allow M2 to handle noisy, dynamic environments, perform multi-tool workflows autonomously, and adapt to various scaffolds and prompts. The talk also previews future developments like M2.1 and M3, with plans to integrate audio and video generation capabilities.

How agents will unlock the $500B promise of AI - Donald Hruska, Retool

Jul 23, 2025 · 16:22

Donald Hruska, engineering lead for Retool's Agents product, argues that AI agents will unlock the $500B promise of AI by moving enterprises beyond toy chatbots into production-grade systems. He explains that building a basic agent is easy (e.g., 100 lines of code using React) but getting it into production requires solving security, cost overruns, observability, and compliance. Hruska outlines four options—build from scratch, use a framework like LangGraph, a managed platform like Retool Agents, or verticalized agents—and advises building for core differentiators and buying for commodity workflows. He cites Retool customers like ClickUp saving over $200,000 in vendor costs and Descript saving hundreds of hours weekly, while noting inference costs dropped 99.7% from 2022-2024. Retool charges $3 per hour for its cheapest agent and supports on-prem deployment.

[Full Workshop] Vibe Coding at Scale: Customizing AI Assistants for Enterprise Environments

Jun 27, 2025 · 1:20:38

Harold from the VS Code team demonstrates "Vibe Coding at Scale" at the AI Engineer World's Fair, presenting three stages—YOLO, structured, and spectrum vibes—for customizing AI assistants in enterprise environments. He live-codes a hydration tracking app using GitHub Copilot's agent mode, auto-approve, and new workspace scaffolding, then shows how to enforce design principles via Copilot instructions and reusable prompts. Harold introduces custom modes (e.g., TDD mode), tool sets, and MCP servers like Playwright and Perplexity for research and browser testing. He emphasizes iterating on instructions, committing often, and using spec-driven development to balance speed with reliability, arguing these techniques enable true flow-state collaboration even on complex codebases.

Lessons from building GenAI based applications — Juan Peredo

Feb 22, 2025 · 33:13

Juan Peredo details the hidden complexities of building GenAI applications, from model hosting and cost control to output validation and observability. He compares local (Ollama) vs cloud hosting (Modal, SkyPilot) and warns that an agent processing 3,000 calls/day with OpenAI O1 costs nearly $300,000/month, while LLaMA 3.3 70B drops that to $50,000/month. He explains techniques to mitigate hallucinations—prompt engineering, guardrails, RAG, and fine-tuning—each with trade-offs like added latency or cost. Peredo advocates externalizing prompts via LangChain Hub for easy iteration and future-proofing, and illustrates agent design with parallel calls to reduce latency. Finally, he stresses observability using tools like LangSmith to debug probabilistic failures, such as an LLM failing on case sensitivity.

Building State of the Art Open Weights Tool Use: The Command R Family: Sandra Kublik

Aug 26, 2024 · 15:03

Sandra Kublik of Cohere presents the Command R family of open-weight models optimized for retrieval-augmented generation and tool use. Released in March 2024, Command R and Command R+ achieved 150,000 Hugging Face downloads within two weeks and now serve nearly 500,000 developers. The models overcome RAG challenges such as prompt sensitivity and citation accuracy through post-training, delivering fine-grained citations and low hallucination. Cohere open-sourced a toolkit UI with plug-and-play components for RAG and tool use, supporting cloud, local, and Hugging Face access. The new Multi-Step API enables sequential reasoning with automatic retry and reflection. Command R+ matches GPT-4 Turbo and Claude Opus on complex reasoning while being three to five times cheaper, positioning it as a scalable enterprise solution.