Why Agent Engineering — swyx

Hi, good morning everyone.

Good morning!

Love that, love that. Um, I'm going to getright into it. One of the challenges we have with Summit is that we actually ask our speakers to do very short talks. So I, as the leader of Summit, I have to do even shorter talks.

So let's go. Uh, you can see a lot of these—there will be a lot of show notes and homework, you can see it on the livestream. How is AI Engineering doing? Uh, it's pretty good. We have an O'Reilly book that's pretty cool.

Um, uh, Chip is actually a good friend and she's actually speaking at, uh, she's giving our keynote for the workshop session, uh, tomorrow. It's just pretty cool. Uh, Garner hates us. Garner thinks we've, we've hit the peak. So it's only downhill from here, guys.

I'm sorry to inform you that AI Engineering is over. Uh, there's no, there's no way out else, uh, else to go but down. Um, a lot of, uh, what I try to do with these a- with these, uh, talks that I do at, at each conference is to try to landmark the se- the state of the art, or the state of the industry.

Um, so with Latent.Space, I, I did the rise of the AI Engineer. With the first AI Engineer Summit, we talked about the three types of AI Engineer. And with last year's AI Engineer World's Fair, we talked about how the discipline of AI Engineering was maturing and spreading across different disciplines.

Um, I think this is starting to get a little sale. Uh, by now a few million people have seen this and, like, you know, uh, used this to form their teams, and I think that was the intended effect.

What I am encountering these days is the two resistance from two sides of the AI Engineer spectrum. Uh, if you come from an MLE point of view, you think that the AI Engineer is just, like, mostly an MLE plus a few prompts.

If you come from a software engineering point of view, you think that it's mostly software engineering and, uh, calling a few LLM APIs. Um, I think over time, it, the AI Engineer is going to basically emerge as its own discipline, and it's still not there yet.

It's still very, very early. I still say things like, "Oh yeah, AIE is 90% software engineering, 10% AI." I think that will grow over time. And I think this is the year when it starts to spread out. And that's, that's what I'm here to talk about a little bit today.

Um, so for example, I, I think, like, what I try to do with AIE is also, like, it's a, it's a work in anthropology. Like, how people describe themselves from groups, from identities, and from industries. Uh, so MLE, you know, it leaks out in your language.

Um, they say test time compute, because the only reason to run inference is to test it. Uh, AIE will maybe say inference time compute, because we actually really care about inference. Um, software engineers may be reasoning. Um, I, I, I think you see these differences, and I'm trying to articulate them over time.

Um, part of what I want to do here to set context is to explain why we have kind of pivoted AI Engineer Summit to be the Agent Engineering conference. Um, uh, it's not a decision that we made lightly.

Because, um, we're saying no to all these things. We're saying no to RAG, we're saying no to open models, uh, GPUs, and we're just saying, uh, you know, th-this is the only thing that we're going to do today.

Um, and, but, like, closing all those doors actually opens up others. So when we put out the call for speakers, we, uh, made up all this list of, uh, you know, other ag-agent engineering disciplines. And, and I soon realized we didn't have to.

I'll talk about this in a bit. Um, I also looked at last year's top performing talks on YouTube, and you guys told us, uh, that, you know, you really wanted all the, all the agentic things. Now, the only problem with this is that we only got speakers who basically made agent frameworks for a living.

Uh, and everyone's asking the, the, the real question, who's putting this in production? So we had a new rule this year of, "Allright, no more vendor pitches." Um, you know, you, you complain about, yeah, let's, uh, thank you.

Uh, as a, as a curator, it makes it so much infinitely harder because, uh, basically the people that you're about to see have no incentive to come on stage and share what they're sharing. Uh, but somehow we talked them into it.

So, uh, I hope you're looking forward to that. Uh, the other thing also I realized that, like, everything plus agent works, basically. So agent plus RAG works, agent plus cogen works, agent plus search works. Um, and this is kind of like the simple formula for, like, making money in 2025.

Uh, most of these, most of these names you'll see in the talks that are, uh, that will follow, uh, in, in the sessions. Um, some of you, if you heard this one before, 2025 is the year of agents,right?

If you say it often enough, it might be true. Uh, I think that when people make predictions, oftentimes they confuse what they want to happen for what will actually happen. Um, so maybe you believe Satya Nadella, maybe you believe Raman, maybe you believe Greg Brockman, maybe you believe Sam Altman.

All of them want you to believe that 2025 is the year of agents. Uh, and I'll be very honest, uh, me and my co-host Alessio, I think I saw you over there, hey! Um, uh, we were pretty skeptical as well.

We were on the record being skeptical. Actually, actually all of you are being on the record, because last, yesterday, uh, Barr played, uh, Family Feud with, with our, with our audience. And the number two, uh, buzzword that everyone is tired of hearing is agents.

Um, but fortunately you guys are not tired enough, because you came to today. I have you for one more day of, uh, of agents talk. Uh, but we're on record, March 2024, with David Lohan, uh, the former VP of Eng of OpenAI, uh, saying that we, we tell people to take agents off of their branding.

Uh, now we tell them to put it back on. So, okay. There, um, I, I, I think I'm, I'm doing this as a public service. To start any agents conference, we have to define the word agents. Are you guys ready?

Allright. I actually have one. I, I, it's a monumental task. I could do it in one slide. Um, so if you talk again, this is, this is a very POV, sort of anthropological point of view. The machine learning people will talk about some kind of reinforcement learning environments.

They want to talk about actions, achieving goals, and all that. Um, AIE, we don't know what they, what they want yet. Uh, the, the software engineers are very reductive. They're just, you know, put it in a for loop.

Um, okay, you, it seems like you agree. Um, so, uh, fortunately, you know, I think every AIE conference needs to invoke the name of Simon Willison. Uh, he is our, uh, patron saint. Um, he's actually gone and crowdsourced 300, uh, definitions of what an agent is.

So I didn't have to survey all of you. I, I was thinking about asking every single speaker to start with, "What is your definition?" Uh, it doesn't matter. Uh, there's, here's six of them,right? You, it's either about goals, it's about tools, it's about control flow, it's about long-running processes, it's about delegated authority, uh, and it's about multi-step task completion.

Yeah, I see all the phones coming out. Don't worry, it's on the livestream,right? There's like 20,000 people, uh, watching along. Um, and then there's, there's a bunch of other things. Uh, I think, I think the last one on the bottom left, bottomright, is, uh, is an interesting one.

Like, just have some things that everyone defines, uh, agrees as an agent and make sure that they're sort of, your agent definition is passing those things. Um, except, so that was my one slide. That was my slide, uh, of, of, like, what, what defining an agent.

And then yesterday, OpenAI went and dropped a new agents definition, uh, on the livestream, uh, that you can watch yesterday as well. Um, so this is something that they're obviously going to work with. Um, and, uh, I, I think you should definitely pay attention to, to this, because they're, they're building on top of this, uh, new definition as well.

So that's defining agents. Why now? Why, why are agents working now when they did not work a year ago, two years ago? Um, I have a rough idea. So the people are talking about capabilities, and so, uh, you can see that capabilities, even, even on a trajectory of 2023, 2025, um, have been, have been really growing, and they started to run to hit human baselines, uh,right about now.

Um, and I also have a map of other cap, uh, reasons as well. So I'll just bring you through each of them. Most people will say, "Oh yeah, we have better reasoning now, we have better tool use now, we have better tools," um, including MCP, which, which we're doing a workshop on, uh, tomorrow.

Uh, but I think there are some other less appreciated things, which I'm going to bring up to youright now. Model diversity,right? Uh, the OpenAI market share has gone from, like, let's say 95% two years ago, now, now to 50%.

It's a much more diverse, uh, uh, landscape. Including, like, this, this, this past week, um, two frontier model labs that are possible challenges to OpenAI have emerged, and which I think, which I think is, um, really exciting for 2025.

We, we don't actually know what it's going to shake out to it by the end of the year. Uh, the second thing is, uh, that the cost of intelligence is super Moore's law, is what I call it. Um, it's, it's gone, uh, the cost of GPT-4 level intelligence has gone down 1,000 times in the last 18 months.

Um, and you can see the same curve starting for the O1 level intelligence. Um, uh, and also we now start to have RL fine-tuning options. Um, I have zero experience in this area, but fortunately one of our speakers will, uh, is going to tell us, talk to us, uh, later today about this, about this.

Um, so we have all these reasons. We have, uh, I have a few more. Uh, you know, in our conversation with Brad Taylor, um, he talked about, uh, cost charging for outcomes instead of, uh, instead of cost. Um, there's a lot of work on multi-agents as well as, uh, faster inference as well.

That's coming out from the, the better hardware that we have. Um, there's more homework there if you want. Uh, this is all sourced and, uh, you know, has, has, has some backing in our, in our latent space conversations, uh, but I don't really have time for that.

Okay, so one last thing for you guys on agent use cases. So, uh, I think most people agree with, like, Bar, um, Barry's, uh, building effective agents talk. Um, he, he's going to talk about how coding agents and support agents have product market fit.

I think now it's fair to say deep research has PMF. Um, but also I will say up and coming are some of these use cases, uh, some of which you're, you're going to see in the, the talks later.

But I also want to offer anti-use cases. Can we please stop demoing agents that book flights? Yeah? No more flight booking agents. Um, I want to book my own flights. Thank you very much. I want to, I want to book my own Instacart orders.

And also please don't AstroTurf Reddit. Right. Uh, okay. So, uh, one that, uh, yeah, and I think the reason that the tell that, uh, you know, this is, this is the headline that I saw yesterday. I had to put this in.

Um, OpenAI reported 400 million users, uh, which is a 33% growth from three months ago. Um, and then you can ask Deep Research to research OpenAI and draw this chart of ChatGPT growth, uh, going from, uh, zero to, uh, 400 million users in two years, in two and a half years.

Um, so, uh, I, I, I remember this chart very well because Open, uh, ChatGPT spent a year not growing. And why did it spend a year not growing? Because they didn't ship any, any agentic models. Um, and if you actually just look at the, uh, the sort of weekly active user chart and stretch it out, you actually get this chart, uh, which is actually super interesting because it basically shows that one, one, um, the sort of O1 models have doubled ChatGPT usage.

And if you stretch it out, um, ChatGPT is going to hit a billion users by the end of this year, this year. Uh, it's basically going to quintuple the number of users it had, uh, as of September of last year.

Um, and so, like, the, the, the, the growth of ChatGPT and the growth of any AI product is going to be very, very tied to reasoning capabilities and the amount of agents that you can ship for your users.

Um, it is, it is real. It is, it is, uh, huge, huge numbers. This is one-eighth of the world population that's going to be using ChatGPT by the end of this year. And I think there's a lot of money left on the table for everyone else.

So, um, I hope you enjoy doing that. Um, I'm well past time, so I'm going to skip all this. But basically I, I think that the job of AIE is now evolving towards building agents in the same way that MLEs build models, software engineers build software.

Um, so, uh, I'm going to skip all that. You can see all, you can see all that on, on the, on the livestream. Uh, but we're actually, uh, you know, just here to welcome you to the show. Um, and, uh, I'm really excited to introduce you to everyone.

So, um, thank you, and I hope you enjoy.