Mistral

Dark Factory: OpenClaw Ships Faster Than You Can Read the Diff — Vincent Koc, OpenClaw

Jun 5, 2026 · 16:44

Vincent Koc, a core maintainer at OpenClaw, explains how the open-source project ships code faster than humans can read diffs by running 60–70 autonomous coding agents across swim lanes. In a single night, he and Peter Steinberger executed 2,700 commits touching 82% of the core codebase, launching a plugin architecture and changing close to a million lines of code. Koc describes managing agents as managing people: the key skill is reading reasoning tokens to detect when an agent is bullshitting, gained through sheer volume of token maxing. He argues 2025 was about token maxing; 2026 is about not wasting them, with agent-in-the-loop processes and opinionated swim lanes replacing blind Ralph looping. The episode details his agent development environment using .skills files, Git work trees, and a semantic graph for triaging 60K PRs, emphasizing that the bottleneck shifts from engineering to taste and process.

Reachy Mini: the $300 open source robot you can actually hack — Andres Marafioti, Hugging Face

May 29, 2026 · 21:16

Andres Marafioti from Hugging Face presents Reachy Mini, a $300 open source robot shipped to 7,500 people unassembled, and explains that its most popular app is voice conversation. To make that app responsive, his team optimized Qwen3-TTS from a below-real-time 0.8x factor to 5.8x real time by adding streaming, switching to a static KV cache, and enabling CUDA graph captures, cutting time to first audio to under 200 milliseconds. The full voice pipeline runs Parakeet transcription every 150 milliseconds with partial results, feeds Qwen 3.5 27B for the LLM, and uses the optimized TTS, while infrastructure round trips are handled by a load balancer that separates LLM endpoints from conversation nodes. Marafioti argues that expensive humanoid robots limit creativity and access, whereas Reachy Mini's hackable, repairable design invites hackers, students, and dreamers to build new interactions. He demonstrates the robot's ability to take photos, show emotions, and even be pet, and emphasizes that all models and software are open source.

Does GenAI "belong" to data scientists? — Phil Hetzel, Braintrust

May 25, 2026 · 18:54

Phil Hetzel of Braintrust argues that generative AI development should not be isolated to data scientists or ML engineers, but instead requires a diverse team including product engineers, systems engineers, and non-technical domain experts. Because models are already built by OpenAI and Anthropic, the remaining work is prompt and context engineering, distributed systems, human annotation, and functional evaluation—not traditional training pipelines. Data scientists add value through rigorous testing and LLM-as-judge evaluation, but must move beyond precision/recall toward broader functional metrics. Traditional enterprises often mistakenly hand GenAI to ML platform teams, while AI natives use small cross-functional groups with closer problem proximity. The ideal mix combines technical roles for implementation and system design with non-technical experts for prompt engineering and human annotation, keeping the agent relevant through continuous feedback.

What Breaks When You Build AI Under Sovereignty Constraints - Bilge Yücel, deepset GmbH

May 19, 2026 · 19:09

Bilge Yücel, Senior Developer Relations Engineer at deepset, argues that sovereign AI requires explicit control over data flow, model choice, infrastructure, and operations, and retrofitting these pillars breaks existing systems in predictable ways. Replacing frontier APIs with self-hosted models forces re-evaluation from scratch, moving private data across jurisdictions creates multi-database search problems, replacing managed infra reveals vendor lock-in, and adding observability exposes black-box systems. She presents a sovereign architecture with guardrails, MCP tools, and Haystack's swappable components to mitigate these issues. The closing checklist asks whether you can swap models without changing application logic, have compliant run logs, and respond to incidents without calling a hyperscaler.

Why TTS Models Now Look Like LLMs — Samuel Humeau, Mistral

May 9, 2026 · 22:26

Mistral AI scientist Samuel Humeau explains why text-to-speech models now resemble LLMs, using an autoregressive transformer with neural audio codecs to compress 200 kbps audio into ~500 tokens per second. He demos Mistral's open-weight TTS model, cloning a voice from a few seconds of reference audio and generating speech with 17 ms first-token latency via frame-by-frame streaming. The talk covers the codec-backbone-decoder pipeline, the model's 4B-parameter backbone generating 37 tokens per 80 ms frame via a diffusion decoder, and a live voice agent answering conference schedule questions. Humeau notes the next latency win is handling streaming text input from an LLM, with no settled architecture yet, while keeping the encoder for voice cloning proprietary.

State of Startups and AI 2025 - Sarah Guo, Conviction

Aug 2, 2025 · 23:52

Sarah Guo of Conviction argues that AI's value creation is massive and early, with companies like Cursor reaching $100M ARR in 12 months and Harvey exceeding $70M ARR. She predicts that by end of 2026, AI agents will ship code directly to production, voice AI will replace text for most business communication, and inference costs will drop below a cent per million tokens. Reasoning is a new scaling vector unlocking higher-stakes use cases, and agent startups have increased 50% in the last year. Multimodal models from HeyGen and Eleven are already rocketing past $50M ARR. The model market is more competitive than ever, with GPT-4 costs falling from $30 to $2 per million tokens in 18 months and open-source like DeepSeek competing. Guo advises builders to focus on thick wrappers around LLMs, leveraging domain and workflow knowledge, and warns against generic text boxes: 'The prompt is a bug, not a feature.' Execution, not first-mover advantage, is the moat.

2025 in LLMs so far, illustrated by Pelicans on Bicycles — Simon Willison

Jul 9, 2025 · 18:30

Simon Willison reviews the past six months of LLM releases — including AWS Nova, Llama 3.3 70B, DeepSeek R1, Mistral Small 3, Claude 3.7 Sonnet, GPT 4.5, Gemini 2.5 Pro, GPT-4o, Llama 4, GPT 4.1, O3/O4 Mini, and Claude 4 — using his 'pelican on bicycle' SVG benchmark to argue that local models have become good enough to run GPT-4 class models on a laptop and that combining tools with reasoning is the most powerful technique in AI engineering, while noting risks like prompt injection and the 'lethal trifecta'. He tracks 30 significant model releases, highlighting that Mistral Small 3 (24B) matches Llama 3 70B's performance, which itself matched the 405B model, enabling local inference. DeepSeek's R1 caused a $500B+ Nvidia stock drop on January 27. GPT 4.1 Nano is the cheapest model yet at a fraction of a cent per pelican. He also examines bugs: ChatGPT's sycophantic 'shit-on-a-stick' incident and Claude 4's tendency to snitch to authorities when given ethical instructions and email tools. Willison concludes that while the pace is accelerating, control over context and security remain critical.

Ship it! Building Production Ready Agents — Mike Chambers, AWS

Jun 27, 2025 · 19:37

Mike Chambers, a developer advocate at AWS, demonstrates how to take a simple local agent—built with a Llama 3.1 8B model and a dice-rolling tool—and ship it to production at cloud scale using Amazon Bedrock Agents. He breaks down the essential components of an agent: model, prompt, loop, history, and tools. Then he live-deploys the agent with Bedrock Agents, configuring instructions and an action group wired to an AWS Lambda function that handles the dice roll. The fully managed service automates scaling, infrastructure, and the agentic loop. Chambers also highlights free courses on DeepLearning.AI and invites attendees to discuss MCP servers and the new open-source SDK for model-first agents.

From Mixture of Experts to Mixture of Agents with Super Fast Inference - Daniel Kim & Daria Soboleva

Jun 27, 2025 · 53:15

Daria Soboleva and Daniel Kim of Cerebras explain how Mixture of Experts (MoE) architectures enable scaling large language models efficiently by replacing monolithic feedforward networks with specialized experts, a technique used by GPT-4 and Claude. They then introduce Mixture of Agents (MoA), which combines multiple LLMs with custom prompts to outperform frontier models like GPT-4o on complex tasks, reducing a 293-second reasoning problem to 7.4 seconds using Cerebras' ultra-fast inference. The workshop guides participants to build their own MoA system, configure agents for bug fixing and performance optimization on a Python function, and achieve scores up to 120/120. Daniel details Cerebras' wafer-scale chip with 900,000 cores and distributed memory that eliminates memory bandwidth bottlenecks, enabling linear scaling and 15.5x faster inference on Llama 3.3-70B versus GPUs. Daria discusses ongoing research in diffusion models and sparsity, while Daniel notes plans for multimodal APIs and LoRA fine-tuning support.

Navigating AI’s Frontier in 2025 - Grace Isford, Lux Capital

Mar 13, 2025 · 17:55

Grace Isford, partner at Lux Capital, argues that while 2025 is a 'perfect storm' for AI agents with reasoning models like O3 and R1, cheaper inference, and billions in infrastructure (e.g., Stargate, DeepSeek), agents still fail due to cumulative errors—decision, implementation, heuristic, and taste—exemplified by OpenAI Operator booking a flight incorrectly. She prescribes five strategies: curating proprietary and agent-generated data, building personalized evals for non-verifiable domains (e.g., seat preference), designing scaffolding that prevents cascading failures (citing Ramp's approach), treating UX as the moat (e.g., Codium, Harvey, TLDraw), and building multimodally with voice, smell via Osmo, and touch for embodiment. The talk, recorded at the AI Engineer Summit 2025 in NYC, closes with a call to reframe perfection through visionary product experiences.

Insights from Snorkel AI running Azure AI Infrastructure: Humza Iqbal and Lachlan Ainley

Feb 8, 2025 · 20:46

Humza Iqbal of Snorkel AI explains how the company uses Azure AI infrastructure powered by NVIDIA GPUs to fine-tune foundation models for enterprise customers, achieving better performance per dollar by switching from A100s to H100s. He details their distributed training stack (PyTorch, Horovod, NFS) and lessons learned such as balancing node count for batch size and monitoring GPU utilization to avoid networking or data-loading bottlenecks. A cost comparison found two H100s outperformed four A100s on both training and inference, enabling faster iteration through more synthetic data. Azure's dedicated VMs, reliable NFS throughput, and flexible capacity allowed Snorkel to scale experiments from single-node to dozens of GPUs. Future work includes programmatic preference signals and multimodal retrieval algorithms, all planned on Azure.

Lessons from the Trenches: Building LLM Evals That Work IRL: Aparna Dhinkaran

Feb 6, 2025 · 18:49

Aparna Dhinakaran, co-founder of Arize AI, distinguishes between model evals (e.g., Hugging Face leaderboard) and task evals for real-world LLM systems, arguing that production applications need component-level evaluations like router and parameter evals. She demonstrates a chat-to-purchase app where a router function call misidentifies user intent, showing how Phoenix open source tool traces errors and provides explanations to iterate. Dhinakaran advises using categorical over numeric LLM-as-judge scores because numeric outputs tend to be binary (0 or 10) and lack granularity. Presenting needle-in-haystack research, she notes GPT-4 struggles retrieving facts placed early in large context windows, and in retrieval-with-generation tasks, Anthropic’s Claude 2.1 outperforms GPT-4 due to verbose reasoning, a gap closed by prompting GPT-4 to explain itself first.

No more bad outputs with structured generation: Remi Louf

Oct 14, 2024 · 15:32

Rémi Louf, CEO of .txt and co-maintainer of Outlines, argues that structured generation—guiding LLMs to produce valid regex, JSON, or context-free grammar outputs—eliminates parsing errors and hallucinations while adding negligible overhead. Outlines masks tokens that violate the target structure, enabling Mistral 7B to achieve 99.9% valid JSON (vs. 17% without). It also accelerates inference: a chatty 50-token ChatGPT answer shrinks to 8 tokens, and structured generation boosts open models beyond GPT-4—Phi-3 Medium hits 96.5% on Berkeley function calling (GPT-4 gets 93.5%). With one shot matching eight shots in accuracy, Louf contends that most text is structured and that structured generation should be the default for non-chatbot workflows.