Snowflake

Why Can't Anyone Answer Questions About the Business? — Garrett Galow, WorkOS

Jun 11, 2026 · 19:06

Garrett Galow from WorkOS built Studio, an internal workspace where anyone can ask natural language questions against Snowflake, Linear, and Notion, and get reusable widgets instead of filing a request. The LLM generates declarative JavaScript widgets that call data sources directly, making subsequent runs deterministic and cheap. Three techniques made it reliable: preflight sequencing injects schema context only when a tool is invoked, a layering rule tells the model to distrust its own knowledge about WorkOS and use primary sources, and query validation catches valid SQL that returns zero rows before hardcoding it into a widget.

Everything You Need To Know About Agent Observability — Danny Gollapalli & Zubin Koticha, Raindrop

May 7, 2026 · 50:25

Zubin Koticha and Danny Gollapalli of Raindrop argue that agent observability must shift from evals to production monitoring because agent failures are non-deterministic and unbounded. They break down explicit signals (tool error rate, latency, cost) and implicit signals (user frustration, refusals, task failure) detected by trained classifiers and regex, emphasizing that aggregate patterns even from imperfect regex are valuable. Experiments let teams ship changes to a percentage of users and compare semantic signal rates, with statistical relevance often reached after a few hundred events. Self-diagnostics—a simple tool and system prompt—enable agents to report their own failures, capability gaps, and even self-correction behavior, as demonstrated in a live coding agent demo where a disabled write tool caused the agent to use bash and then report the bypass. The episode covers alerting, trace visualization, data export to BigQuery/Snowflake, and the challenge of managing fast-paced experimentation at scale.