Vertex

What if the network was the sandbox? — Remy Guercio, Tailscale

Jun 1, 2026 · 24:29

Remy Guercio from Tailscale argues that standard sandboxing conflates execution isolation with access control, proposing Aperture—an LLM gateway built on Tailscale's WireGuard identity network—which gives every connection verified identity (user, tag, or group) so agents get placeholders instead of real API keys, making exfiltration impossible. Aperture provides visibility into every tool call, bash command, and MCP request without instrumentation inside the container; internally at Tailscale, bash dominates over structured tool calls. Access permissions are configured via Tailscale's grants and ACLs, supporting quotas, cost controls across providers, and webhooks for tool calls. The gateway works at the LLM layer, capturing even non-tool-call agent behaviors like direct code execution, and is available on Tailscale's free plan.

Build & deploy AI-powered apps — Paige Bailey, Google DeepMind

Apr 29, 2026 · 1:03:20

Paige Bailey, engineering lead for developer relations at Google DeepMind, demonstrates how the company's latest multimodal models and AI Studio platform enable rapid prototyping of AI-powered apps at minimal cost. She showcases Gemini 3.1 Flash Lite analyzing YouTube dinosaur videos for under a penny, the new Build feature constructing a fully functional bookshelf cataloger with Firebase database and Google login, and Gemini Live providing real-time multilingual screen interaction. Other demos include Genie 3 generating playable world models from text prompts, NanoBanana 2 performing cost-effective image editing, Veo 3.1 Lite producing stock footage (e.g., a "vegan basketball food truck" featuring Chef Curry), Lyria 3 creating custom music tracks, and Gemma 4 offering open-source multimodal models that run on mobile devices. Bailey emphasizes that the entire stack—grounded by Google Search and code execution—allows developers to ship ideas that would have been startups years ago, with all code exportable for production use.

Gemma 4 Deep Dive — Cassidy Hardin, Researcher, Google DeepMind

Apr 27, 2026 · 19:03

Cassidy Hardin, a researcher at Google DeepMind, details the Gemma 4 family of open-source models, claiming they set a new precedent for small-scale performance. The family includes two on-device effective models (E2B and E4B) with Per Layer Embeddings (PLE) stored in flash memory, and two larger models: a 26B mixture-of-experts (MoE) with 128 total experts (8 active) and a 31B dense model that ranked #3 on the LM arena leaderboard, outperforming models 20x its size. Architectural innovations include interleaved local/global attention (5:1 ratio, sliding windows of 512/1024 tokens), grouped query attention (8 queries per key-value in global layers), and native multimodal support with variable aspect ratios and resolutions for vision encoders (550M params for large, 150M for small) plus a 305M parameter conformer for audio. All models are released under Apache 2.0 license, available for self-hosting on Hugging Face and Ollama, or cloud deployment on Vertex AI.

2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI

Aug 6, 2025 · 19:14

John Dickerson, CEO of Mozilla AI and former co-founder of Arthur AI, argues that 2025 is finally the year for AI evaluation (evals) due to three converging forces: ChatGPT making AI tangible to the C-suite, enterprise budget freezes that funnelled funding into GenAI pet projects, and the rise of agentic systems acting autonomously. He traces how ML monitoring existed pre-2022 but lacked C-suite attention, with Jamie Dimon’s JPMC spending only $100 million on AI from 2017 to 2021. The ChatGPT launch on November 30, 2022, unlocked CEO discretionary budgets for GenAI, leading to 2023 science projects, 2024 production deployments, and 2025 scaling. Now, as agents reason and act, connecting evals to downstream business KPIs—risk mitigation, revenue gains—is a first-class discussion for CEOs, CFOs, CISOs, and CTOs. Dickerson notes that evaluation companies have shifted to multi-agent system monitoring, and while LLM-as-judge is popular, biases require human validation, with companies like Mercor hiring experts at $50–200/hour to lockstep multi-agent outputs.

Fun stories from building OpenRouter and where all this is going - Alex Atallah, OpenRouter

Jun 25, 2025 · 18:47

Alex Atallah, founder of OpenRouter, tells the story of how he launched the LLM aggregator in early 2023 after observing the open-source race sparked by Meta's Llama 1 and Stanford's Alpaca distillation under $600. He argues the inference market is not winner-take-all, citing OpenRouter's data showing Google Gemini growing from 2-3% to 34-35% of tokens over 12 months. The platform evolved from a simple model collection into a marketplace with over 400 models and 60 providers, solving architecture challenges like 30-millisecond latency, cancelable streams, and a middleware plugin system for web search and PDF parsing. Atallah explains that OpenRouter grew 10-100% month-over-month for two years, and he predicts future additions like transfusion models that generate images and more powerful geographic routing for enterprise optimization.

Agents reported thousands of bugs, how many were real? - Ian Butler and Nick Gregory

Jun 3, 2025 · 18:39

Ian Butler and Nick Gregory introduce SM-100, a benchmark of 100 real-world bugs across 84 repositories, and present their agent Bismuth as outperforming existing agents in detecting complex bugs—finding 10 needles vs 7 for the next best. However, even top performers struggle: Bismuth's true positive rate is only 25%, and Codex leads at 45%, while most agents (Cursor, Debian) report thousands of false positives (97% for basic loops). They argue that agents universally exhibit narrow thinking, missing simple bugs like a form state issue that only Bismuth and Codex caught, and that high SWE-bench scores do not translate to maintenance tasks. The episode concludes that software maintenance remains an unsolved problem requiring deeper reasoning and targeted search.

Anthropic in the Enterprise — Alexander Bricken & Joe Bayley

Apr 13, 2025 · 20:55

Alexander Bricken and Joe Bayley from Anthropic's Applied AI team argue that enterprise AI implementation often fails due to overengineering, poor data infrastructure, or lack of testing—but industry leaders achieve transformative results with Claude. They detail Anthropic's deployment models (API, cloud partnerships, enterprise solutions) and real-world case studies like Intercom's Fin agent, which solved 86% of support volume using Claude. Best practices include building evals early as intellectual property, identifying intelligence/cost/latency trade-offs based on use-case stakes, and avoiding premature fine-tuning by trying prompt caching, contextual retrieval, and agentic architectures first. They also highlight interpretability research and the Model Context Protocol for reliable AI deployments.