AI Engineering Without Borders — swyx

I've been thinking a lot about borders recently, for no particular reason. And borders are a very innately human thing: if I don't have theright piece of paper, I cannot cross this line in the sand. Like, it's a very, very real problem that I face, and many, many people face every single day.

And borders are just one kind of constraint that humans just make up. And I think that's very interesting that we respect borders so much, but AI does not. AI is a border disrespecter. It is very, very easily multilingual.

So if you trained an LLM on mostly English text corpus, it's going to learn other languages just as a side effect. It's going to be very natively multimodal, because you can turn LLaMA into a vision language model with just like 100 bucks of just post-training.

It's very disrespecting of ground truth borders, because it can just doesn't know the difference between hallucination and memorizing from a world model. And it also doesn't respect copyright, which is a whole other topic that we won't get into today, but it's also super fascinating.

And how does that relate to do with AI Engineering,right? Like, I think a lot of you here are here because you are interested in that concept at least. Maybe you identify as an AI Engineer, maybe you're trying to hire an AI Engineer.

So there are a lot of definitions floating around, and I confess that, you know, I've contributed to that. Is AI Engineering an API line,right? That's the line that a lot of people have, and that's come under some debate recently.

And yeah, that's one form of AI Engineering. And I think that is useful to some people for understanding, like, where the responsibilities in a team might stop and end and start with the other people in the team. Or maybe it's there's different subtypes,right?

Like last AI Engineer Summit, I talked about the three types of AI Engineer that I was seeing emerge: the AI Enhanced Engineer, the AI Products Engineer, and the non-human agentic AI Engineer. Or it could be a job description that you try to sort of list out.

And this is something that, on the Latent Space podcast we recently went through with Elicit, we're talking about the different roles that they see within their teams as well. So, okay, if I broadly have any of these three things, do I have I nailed down a good definition of AI Engineer that is workable?

Yes,right. But is that something that we're happy with? Is it something that we can is there nothing left to explore? I think the answer is no. I think there's more to explore. I think the very easy cop-out as well for people discussing this is that you have your opinion, I have my opinion, you come from your point of view, I come from my point of view, we agree to disagree, or we agree that, you know, different strokes, different folks, and we move on.

I don't really like that, just because there's no shared agreement on the things that is ground truth to everybody. So I want to raise that challenge a little bit more. I was in a podcast with Raza Habib, who is one of the speakers today, talking about what this conference is and why this conference is does what it does.

And I always say that AI Engineer conferences are effectively my highest stakes expression of what I think the state of AI Engineering is. So this time last year, 2023, AI years are two times of human years. We had a few tracks, and we had a few topics that were up for debate.

We had RAG, code gen, and then agents and multimodality. All those tracks are repeated here today. I have some speakers illustrated here, just for illustrative purposes. These obviously are not every not everyone involved. But I think just like the inside-out metaphor that I've been thinking a lot about, as the AI Engineer matures, so does the number of concerns that you have to juggle in your head.

So this year, you know, after you're a competent AI Engineer, this year you're now faced with, like, okay, I have to migrate to open models, I have to build up my evals, maybe I should have done that first, but that's a whole topic of discussion.

Maybe I should scale up my inference, or maybe I should deploy it to the Fortune 500. And maybe on the management side of things, I should be hiring teams of AI Engineers and managing AI strategy for my company.

I think the last track, you know, like I talked about the nine tracks in AI Engineers, it's always about the network, the community, the network that we're building. That's probably the single most important part of this conference, and that's the part that we cannot sell.

We cannot I cannot put on the website, "Hey, we have a good community," because no one will believe us. You have to come and see for yourself. But please, for those of you who've been uploading to the Google Photos album, who've been tweeting out your photos and sharing them on LinkedIn, please keep doing that.

That's a way for myself and everyone else who is not at the conference to try to join in on the fun. The reason I'm not comfortable with any of these tracks is because I know how they were made, because I made them up.

And I know that because I was looking at the original document for the Rise of the AI Engineer. It's we're celebrating the one-year anniversary today. And just down in the document somewhere, I just listed out the, you know, disciplines that I thought the AI Engineer would have, and those eventually became mostly mapping to the tracks that I have been exploring in these conferences and the meetups that I do.

And it's arbitrary. Like, why is there a separate agents track from code gen? Why is there a separate RAG track from open models? Like, these are all related. You know, they're all of a kind, and obviously as a competent AI Engineer, you should be familiar with all of these things.

And that brings us back to this mindset of having boundaries and borders,right? These are all made up by someone. I made them up for this one. But, you know, you're going to live in a world where your boss made it up at your company.

And these are not reflective of how reality actually has to operate,right? If you struck away all the rules and a different group of people came in, would they agree on the same rules? Probably not, just because they're made up.

But the laws of nature are hard to make up because reality actually works that way. If you had an alien civilization come down to Earth, they would discover that gravity works the same, the sun the energy from the sun works the same, the magnetic field works the same.

So what are the laws of AI Engineering? We'll come to that via definitions of software engineering and real engineering. So I went and looked up definitions of software engineering. And IEEE talks about the application of engineering to software, with the design, implementation, testing, and documentation of software.

Google Developers talks about mostly the same thing. They also mention software lifecycle management. Okay, like, really reasonable definitions of a kind. Until you look at real engineers. Real engineers talk about applying natural sciences utilizing them for benefit of humanity.

Both the IEEE and the National Association of Engineers agree on that. And it's curiously missing from software engineering. Like, the benefit in humanity part, the understanding natural laws part, completely missing from software engineering. So my proposal to you is that AI Engineering is somewhere in between,right?

The software engineering, but then encountering a lot of the more of the real-world constraints than you would in a typical software engineer career. So if we know the laws of Earth, and they are independently derived, they cannot no matter what point of view you're looking at, they all are the same laws.

Then what are the equivalent laws of AI Engineering? I have a few. You can come up with more, but I'm just going to propose some to start off the debate. There's constants. So, for example, if you're designing for humans, you should respect the fact that humans only speak at 80 words per minute, but they read at 200 words per minute,right?

So there's an inherent disparity there. There's also contingent facts. There are things that are true for now instead of true forever. And true for now facts are, for example, like the Apple Intelligence. When they ship a local model on every phone, then that inference speed of 30 tokens per second that they're advertising becomes the baseline speed limit, or speed barrier, of what intelligence that is too cheap to meter should look like.

And these things, they're not set in stone. They're not actual physical laws. So they also trend over time due to forces and momentum. And I want to establish a little bit, like, I think it's very beneficial for AI Engineers to understand what the Moore's laws of AI is, so that you can plan for them, so that you don't have to make the bad bets that are not going to last, just obviously just because of overwhelming evidence.

The first bet is the improving of context,right? A year ago, I was interviewing Mosaic and talking about MPC-7B with their whopping 60,000, 70,000 token context, with a lot of loss. Today, sitting in the audience, we have people who have trained million token context windows.

And we've also have from Anthropic, which just released Claude 3.5 last week, the fact that, you know, we have complete utilization. It's not just about the length of context, it's also about the utilization of context. And I think Greg, who's sitting in the audience as well, would be very happy with how Claude is improving on their utilization of their very, very long context windows.

There's also the cost of intelligence, the commodification of intelligence. So in the past two years, we've seen a 99.55% decline in the cost of GPT-3 level intelligence. The cost of GPT-4 level intelligence has probably come down maybe 90%, maybe 80%, from GPT-4 to LLaMA 3 and all the newer models that.

Finally, I think it's worth commenting a little bit on where AI Engineering stands in contrast to other AI philosophies. There's EA versus EAC. And maybe we're in the middle. Like, we care about safety, but we also want to accelerate,right?

It's kind of like a weird combination of the two things. My proposal is that that one dimension isn't enough to express how AI Engineer differs from the other philosophies. And actually, you need to add a second dimension to talk about utility.

We are utility maxis above all else. We see what's out there, and we want to use it to benefit humanity. So my message to everyone at the World's Fair is to try to disrespect borders a little bit. Try to avoid your own dogmatic beliefs, lazy consensus of other people, or passive reactions.

And in other words, try to disagree. Disagree more. Disagree with your own conclusions. Disagree with each other productively. And disagree with the status quo. And I think there you'll find that this conference becomes more of a useful landmark in your careers rather than just a party, which it can very well be.

My final analogy, which I really like, is that AI Engineers are the kind of person that looks at Shoggoth and sees, instead of a monster that cannot be tamed, they want to turn them into mass rapid transit. And the kind of person that looks at that, looks at, you know, force of nature, and wants to turn it into tools that are useful for people, is the kind of AI Engineer that I would love to speak to and welcome at conferences like this one.

So that's my view of what borders and AI Engineering without borders should look like. I very much encourage you to jump between tracks, to jump between friend groups, to jump between disciplines and modalities, because here's the one place that you can do that, outside of your work, and to mingle with everyone else that we've gathered.

So I hope you enjoyed doing that. I wish I was there in person, but just share them online, and I hope to see you in person at the next one. Bye.