ClickHouse

From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization - May Walter, Hud

Jul 19, 2026 · 22:46

May Walter, CTO of Hud, presents a case study on using coding agents with runtime intelligence to automate continuous performance optimization in production. The system analyzes real production traces, queries, and latencies to surface high-ROI fixes like N+1 queries and missing indexes, scoring them by complexity and impact. It runs weekly via GitHub Actions and Claude Code, generating reports with verified fixes and evidence of impact, such as P90 latency improvements after deployment. Key challenges included 'plausible unverified' fixes, complex ClickHouse queries, and lazy fixes that only mask exceptions rather than root causes. Walter emphasizes the need for prod-to-code context mapping, scoring guardrails, and human review to achieve reliable automation. The talk concludes that context over cleverness works, and agents can automate the investigation phase that teams rarely do proactively, turning dormant performance debt into actionable merged PRs.

How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust

May 28, 2026 · 20:43

Phil Hetzel of Braintrust argues that agent observability differs fundamentally from traditional observability, which only answers 'is the system up?' Agents are non-deterministic, with individual traces exceeding a gigabyte and spans reaching 20 megabytes, packed with unstructured text that traditional tools like Datadog or Grafana cannot handle. Braintrust built a custom database from scratch—featuring a write-ahead log for instant visibility, analytical indexes for filtering, and a forked Rust-based Tantivy index for full-text search—to ingest and query this data in real time. Unlike traditional observability, agent observability involves non-technical experts (e.g., clinicians, lawyers) who grade traces and write justifications that become training signals for automated scoring functions. Braintrust also runs lightweight LLMs on traces to perform clustering and topic modeling, surfacing user intent, sentiment, and failure modes to accelerate the iteration loop between production issues and fixes. The episode also covers Braintrust's shift from ClickHouse to a custom database due to ClickHouse's limitations in text indexing.