AI Engineering Philosophy

"The engineer of the future is the person who is able to choose what is worth doing." — Addy Osmani

Jul 14, 2026 · 18:26

Addy Osmani argues that the engineer of the future must shift from code production to accountability, judgment, and ownership as AI agents automate more work. He introduces Alpha (the gap between human and model capabilities) and Decay (the clock on that gap), warning that skills like speed and recall erode quickly, while taste—the ability to judge without objective metrics—decays more slowly but still fades. Osmani warns against cognitive debt (losing understanding of your codebase) and cognitive surrender (blindly accepting AI outputs, noting 73% of people felt more sure when AI was wrong). He advocates for an agency ladder where engineers move from flagging problems to discerning which paths deserve ownership, and proposes an operational rule: "explain it or don't ship it." Ultimately, he predicts that lower costs of building will unlock latent demand, moving the bottleneck from "can we build this" to "should this exist" and "can we answer for it."

The Golden Age of AI Engineering — Alexander Embiricos & Romain Huet & Peter Steinberger, OpenAI

Jul 9, 2026 · 25:13

OpenAI's Alexander Embiricos, Romain Huet, and Peter Steinberger reveal how AI engineering is entering its golden age, with model releases accelerating from every 15 months to roughly every 6 weeks and frontier-level intelligence now available at $1 per million input tokens and $6 per million output tokens. They demonstrate Codex's evolution from a model that couldn't test its own code to one generating 750 tokens per second, enabling a 10-second pull request, and emphasize that the product is built on the same open-source stack—responses API, AGENTS.md format, and harness—that ships to developers. The trio argue that the future is not about automating engineers but empowering them via value maxing over token maxing, with agents moving from local/cloud distinctions to persistent, long-running managers that orchestrate work and only require human steering at key decision points, as illustrated by an automated open-source issue resolution workflow.

Your LLM Deception Monitor Is Broken. The Fix Is in the Training Data - Sachin Kumar, LexisNexis

Jul 8, 2026 · 13:58

Sachin Kumar (LexisNexis) presents his peer-reviewed IJCNN paper showing that standard LLM deception monitors—behavioral tests and joint cross-model features (crosscoders)—fail to detect sleeper-agent backdoors. The fix is a diff sparse autoencoder (SAE) trained on the difference between base and fine-tuned model activations. On a controlled SQL-injection backdoor triggered by 'year 2024' in a 360M-parameter model, diff SAE achieves 40× the signal of joint features, perfect precision, and zero false positives from a single layer. Recall is about 25%, so multiple features should be ensembled. The method is cheap (one forward pass per checkpoint) and works under both full-rank and LoRA fine-tuning. Limitations include requiring the base checkpoint and not yet tested against adaptive adversaries. Kumar demonstrates that backdoors are directional shifts in activations, and diff SAE isolates them directly.

Text Diffusion — Brendan O’Donoghue, Google DeepMind

Jun 4, 2026 · 28:03

Brendan O'Donoghue, a research scientist at Google DeepMind, explains that text diffusion models generate tokens 10x faster than autoregressive models by performing 24 denoising steps to produce 256 tokens, dramatically reducing memory transfers. Unlike GPT-4o and Gemini 2.5 Flash, which incorrectly answered 40 and 42 on a math problem, Gemini Diffusion used bidirectional attention to self-correct from 60 to 49 to 39. The model adaptively allocates compute: 4 steps for memorized digits of π, 31 for quantum mechanics, and automatically stops when satisfied. Text diffusion also enables in-place editing, demonstrated by fixing code bugs or adding paragraphs. However, lower throughput on large batches makes it expensive to serve at scale today. O'Donoghue showcases low-latency applications: a fully generated Wikipedia, a Reddit clone with AI text and images, an on-the-fly operating system, and a to-do app built in 15 seconds by voice.

Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs

May 4, 2026 · 1:21:26

Angelos Perivolaropoulos from ElevenLabs walks through building a small GPT-2-like LLM from scratch on a local machine, demonstrating that the core techniques used by major labs are accessible in a few hundred lines of PyTorch code. The workshop uses character-level tokenization (65 tokens) on a Shakespeare dataset to enable fast training with limited compute. The model architecture includes multi-head self-attention, MLP layers, residual connections, and layer normalization, totaling 10 million parameters across six transformer blocks with a 256-token context window. The training loop employs next-token prediction with a warm-up cosine decay learning rate schedule and validation loss to detect overfitting. Inference uses temperature sampling (default 0.7) and top-k sampling to improve creativity. Perivolaropoulos explains that audio and multimodal models share the same transformer foundation but differ in tokenization (e.g., mel-spectrograms for audio) and use specialized losses like L2 or KL divergence, while reasoning models result from post-training base models with high-quality chain-of-thought data.

Cognitive Exhaust Fumes, or: Read-Only AI Is Underrated — Šimon Podhajský, Head of AI, Waypoint

Apr 8, 2026 · 11:31

Šimon Podhajský argues that read-only AI systems that analyze personal digital exhaust without the ability to write back are more valuable than agentic AI that acts on users' behalf. He built a system ingesting six read-only sources (email, journal, tasks, CRM, browser sessions, notes) that surfaces insights like intention-action gaps, attention drift, and relationship decay via cross-source pattern detection—things no single source reveals. For example, a weekly reflection skill in Claude synthesizes a brutal review of his week, and a cross-source query maps his recent reading to contacts in his CRM using Vivaldi SQLite and Clay MCP. He emphasizes risk asymmetry: read-only errors cost nothing, while write errors can be unbounded. He also acknowledges security risks like the mosaic effect and Simon Willison's lethal trifecta, noting that shell access still allows exfiltration, but argues that examined risk is better than ignorance.

No More Slop – swyx

Dec 22, 2025 · 9:15

In this keynote at the AI Engineer Summit, host swyx declares war on slop — low-quality, inauthentic, inaccurate work produced by both humans and AI — arguing that the AI engineering community must elevate taste and accountability. He introduces Swix's law of anti-slop: the taste needed to fight slop scales with the plummeting cost of generating tokens. Swyx demonstrates how to combat slop using AI itself, citing examples like AI News (which tells readers to skip slow days), prompting techniques to avoid slop, and using sub-agents against context rot. He calls for rejecting autonomy without accountability and urges the audience to say "no more slop" to bosses demanding more lines of code, untested releases, and engagement bait.

On Engineering AI Systems that Endure The Bitter Lesson - Omar Khattab, DSPy & Databricks

Aug 6, 2025 · 19:12

Omar Khattab, research scientist at Databricks and creator of DSPy, argues that engineering AI systems that endure the rapid pace of model and technique changes requires decoupling high-level task definitions from low-level, swappable components. He reinterprets the Bitter Lesson—that scaling search and learning beats hand-coded domain knowledge—as a warning against premature optimization, which he equates to hard-coding at lower abstraction levels than justified. Khattab criticizes prompts as a poor programming abstraction that entangles task specs, formatting, and model-specific tricks, violating separation of concerns. Instead, he advocates for investing in three orthogonal concerns: natural language specs for things that can't be otherwise expressed, evals to capture true objectives, and code for reliable structure and control flow. DSPy realizes this philosophy through declarative signatures that define what an AI system should do, independent of which LLM, inference strategy (e.g., chain-of-thought, agent), or optimizer (e.g., reinforcement learning, prompt optimization) is used. The takeaway: avoid hand engineering at lower levels than current abstractions allow, and ride…

Why you should care about AI interpretability - Mark Bissell, Goodfire AI

Jul 27, 2025 · 21:11

Mark Bissell of Goodfire AI argues that mechanistic interpretability—reverse engineering neural networks—has moved from research to practical use cases that AI engineers can apply today, demonstrated through Goodfire's Ember platform for neural programming. He shows how Ember enables debugging and steering models at the neuron level, such as turning up a 'sensitive information' feature to make LLaMA refuse to reveal an email, or dynamically injecting a Coca-Cola recommendation when a beverage feature activates. For image models, the Paint with Ember demo (paint.goodfire.ai) lets users paint concepts like pyramid or lion face directly onto a canvas and steer sub-features (e.g., lion minus mane becomes tiger). Beyond interfaces, interpretability powers model diffs to detect unwanted changes after fine-tuning, guardrails for production systems, and scientific extraction: Goodfire works with the ARC Institute to uncover biological principles from the superhuman genomics model Evo2. Efficiency gains also emerge by pruning unnecessary weights for specialized tasks, making interpretability a critical tool for reliable AI engineering.

Small AI Teams with Huge Impact — Vik Paruchuri, Datalab

Jul 15, 2025 · 17:36

Vikas Paruchuri, CEO of Datalab, argues that small teams can outperform large ones, sharing how his team of 4 achieved 40k GitHub stars, 7-figure ARR, and 5x revenue growth since January by training state-of-the-art models like surya OCR 3. Drawing from his experience scaling DataQuest to 30 people and then cutting to 7, he explains that layoffs increased productivity due to fewer specialists, less meeting overload, and more senior generalists. He advocates hiring senior generalists who work across the stack, using simple tech (e.g., server-rendered HTML over React), and minimizing bureaucracy with high trust and in-person collaboration. By leveraging AI to handle low-leverage tasks and training models to replace forward-deployed engineers, Datalab maintains tight feedback loops and fast iteration. Paruchuri emphasizes scaling productivity, not headcount, and offers a three-step hiring process: a peer chat, a paid 10-hour project, and a culture fit check, with a 40% hire rate.

How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS - Ishan Anand

Jul 13, 2025 · 1:41:34

Ishan Anand shows that GPT-2 small implemented in 600 lines of vanilla JavaScript makes LLMs understandable for web developers without ML backgrounds. He explains tokenization via byte-pair encoding, 768-dimensional embeddings representing semantic meaning via co-occurrence, and the Transformer's attention mechanism that lets tokens share context. The multi-layer perceptron learns next-token prediction through backpropagation, while the language head converts embeddings to token probabilities using softmax. Anand demonstrates each step—tokenization, embedding lookup, positional encoding, attention, MLP, and output—in a browser debugger, and notes that GPT-2's architecture underpins ChatGPT, with innovations like scale, supervised fine-tuning, and RLHF. The workshop provides an intuitive mental model of Transformers, turning perceived AI magic into understandable machinery.

The New Code — Sean Grove, OpenAI

Jul 11, 2025 · 21:36

Sean Grove of OpenAI argues that specifications, not code, are becoming the fundamental unit of programming, with the most valuable skill being precise communication of intent. He presents OpenAI's Model Spec—a collection of versioned Markdown files—as a living specification that aligns both humans and models around shared values and intentions. Grove illustrates how the Model Spec served as a trust anchor during the 4.0 sycophancy bug, where shipped behavior contradicted the spec's explicit 'don't be sycophantic' clause, leading to a rollback. He explains deliberative alignment, where the spec is used as training and eval material to embed policy into model weights, moving from inference-time prompting to muscled memory. Drawing parallels to the US Constitution, Grove positions specifications as executable, testable artifacts that compose like code, and suggests future IDEs will become 'integrated thought clarifiers.' He closes by calling for help in aligning agents at scale, noting OpenAI's new agent robustness team.

CI in the Era of AI: From Unit Tests to Stochastic Evals — Nathan Sobo, Zed

Jun 27, 2025 · 14:50

Nathan Sobo, co-founder of Zed, explains how his team adapted continuous integration for AI-powered features in their Rust-based code editor. Zed's traditional CI eliminates non-determinism with simulated schedulers and deterministic tests, but LLMs forced a shift to stochastic evaluation. Their process starts with broad data-driven evals, then drills into focused stochastic unit tests run 100–200 times with pass thresholds, and finally refines into deterministic tests. Specific challenges like parsing streaming edits, XML tag mismatches, indentation normalization, and strange escaping were solved by robust algorithms—e.g., fuzzy matching and indent delta detection—rather than complex ML. Sobo emphasizes that rigorous empirical testing and traditional software engineering skills remain essential, even when embracing probability over binary pass/fail.

The Coherence Trap: Why LLMs Feel Smart (But Aren’t Thinking) - Travis Frisinger

Jun 3, 2025 · 20:47

Travis Frisinger argues that large language models are not intelligent but coherent, introducing 'coherence reconstruction' as a mental model for understanding their utility. He explains that hallucinations are a feature, not a bug, as models fill gaps predictably to maintain pattern completion. Prompts act as force vectors navigating latent space, and the key to reliable outputs is a 'frame, generate, judge, iterate' loop. Frisinger presents a three-layer model: latent space, execution layer (tools and RAG), and conversational interface. He advises engineers to design for emergence rather than control, using dense context as anchors to steer generation, and to watch for breakdowns in tone as early signs of lost coherence.

Frontier Feud: Anthropic, Google DeepMind, Meta FAIR, Thinking Machines — Barr Yaron, Amplify

Apr 19, 2025 · 22:26

Teams from Anthropic, Google DeepMind, Meta FAIR, and Thinking Machines compete in a Frontier Feud game hosted by Barr Yaron at the AI Engineer Summit 2025. Surveyed AI engineers name Ilya Sutskever as the most influential AI researcher, with Andrej Karpathy, Jeff Hinton, and Yann LeCun also on the board. Cost is the top consideration when choosing a model, followed by latency, eval scores, and open vs. closed source. The buzzword everyone is tired of hearing is AGI, with RAG and prompt engineering trailing. In the fast money round, Cursor tops favorite AI tools, 'Attention Is All You Need' wins most influential paper, and hardware failure is the biggest 2 a.m. nightmare. The winning team, Rocco's Basilisk, takes home a massive llama and other prizes.

AI Engineering Without Borders — swyx

Oct 30, 2024 · 10:32

In this talk from the AI Engineer World's Fair, host swyx argues that AI inherently disrespects human-made borders—it is naturally multilingual, multimodal, and indifferent to copyright or ground truth. He challenges the field to define its own laws, distinguishing constants (e.g., humans speak at 80 wpm vs. read at 200 wpm) from contingent facts (e.g., Apple Intelligence's 30 tokens/sec baseline). Reflecting on one year of the 'Rise of the AI Engineer,' he notes that the conference tracks—RAG, code gen, agents, multimodality—are arbitrary constructs we created, not natural categories. He proposes that AI Engineering sits between software engineering and real engineering: it must apply natural sciences for humanity's benefit. The talk concludes with a call to 'disagree more'—with your own conclusions, each other, and the status quo—and to transform the Shoggoth of raw AI into mass transit tools for society.

Decoding the Decoder LLM without de code: Ishan Anand

Aug 9, 2024 · 17:08

Ishan Anand demonstrates how decoder-based LLMs like GPT-2 work internally using a fully functional spreadsheet implementation. He explains tokenization (e.g., 're-injury' splitting into 'rain' and 'injury' due to subword units), 768-dimensional embeddings, multi-headed attention where tokens look backward (e.g., 'he' focuses 0.48 on 'Mike'), and the multi-layer perceptron with matrix multiplies (MMULT). He shows residual connections creating an information superhighway and uses logit lens to reveal layer-by-layer predictions (e.g., 'Wednesday' appears early but only rises to top probability at later layers). Finally, he injects a sparse autoencoder feature vector for 'Jedi' into the residual stream, steering GPT-2 to output 'lightsaber' instead of 'phone', and contrasts this with alternative steering methods like representation engineering and activation steering.

Open Challenges for AI Engineering: Simon Willison

Jul 17, 2024 · 18:49

Simon Willison argues the GPT-4 barrier has been broken as GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, and open models like LLaMA 3 70B now compete, making GPT-4-class models a commodity. He highlights the AI trust crisis with examples of Dropbox and Slack being falsely accused of training on user data, and notes Anthropic trained Claude 3.5 Sonnet without customer data. Willison warns about prompt injection vulnerabilities, citing the Markdown image exfiltration bug affecting six major chatbots, and defines slop as unreviewed AI-generated content, calling for accountability and responsible use patterns.