Open Challenges for AI Engineering: Simon Willison

This was supposed to be OpenAI. I'm replacing OpenAI at the last minute, which is super fun, so you can bet I used a lot of LLM assistants to pull things together that I'm going to be showing you today.

Um, let's dive straight in. I want to talk about the GPT-4 barrier,right? So, back in, um, March of last year—so just over a year ago—GPT-4 was released and was obviously the best available model. We all got into it.

It was super fun. And then for 12—and it turns out that wasn't actually our first exposure to GPT-4. A month earlier, it had made the front page of The New York Times when Microsoft's Bing, which was secretly running on a preview of GPT-4, tried to break up a reporter's marriage, which is kind of amazing.

I love that that was the first exposure we had to this new technology. But GPT-4, it's been out, it's been out since March last year, and for a solid 12 months it was uncontested,right? The GPT-4 models were clearly the best available, like, language models.

Lots of other people were trying to catch up, but nobody else was getting there. And I found that kind of depressing, to be honest. You know, it was—you kind of want healthy competition in this space. The fact that OpenAI had produced something that was so good that nobody else was able to match it was a little bit disheartening.

This has all changed in the last few months. I could not be more excited about this. My favorite image for sort of exploring and understanding the space that we exist in is this one by Karina Wynn. Um, she put this out as a chart that shows the performance on the MMLU benchmark versus the cost per token of the different models.

Now, the problem with this chart is that this is from March. The world has moved on a lot since March, so I needed a new version of this. And, um, so what I did is I took her chart, and I pasted it into GPT-4 Code Interpreter.

I gave it new data, and I basically said, "Let's rip this off,"right? Let's—and it's an AI conference. I feel like ripping off other people's creative work kind of does fit a little bit. Um, so I pasted it in, I gave it the data, and I spent a little bit of time with it, and I built this.

It's not nearly as pretty, but it does at least illustrate the state that we're in today with these newer models. And if you look at this chart, there are three clusters that stand out. The first is these ones.

These are the best models,right? The Gemini 1.5 Pro, GPT-4o, the brand new Claude 3.5 Sonnet. These are really, really good. I would classify these all as GPT-4 class. And like I said, a few months ago, GPT-4 had no competition.

Today we're looking pretty healthy on that front. And the pricing on those is pretty reasonable as well. Down here we have the cheap models, and these are so exciting. Like, Claude 3 Haiku and the Gemini 1.5 Flash models, they are incredibly inexpensive.

They are very, very good models. You know, they're not quite GPT-4 class, but they are really—you can get a lot of stuff done with these very inexpensively. If you are building on top of large language models, these are the three that you should be focusing on.

And then over here we've got GPT-3.5 Turbo, which is not as cheap and really quite bad these days. If you are building there, you are in the wrong place. You should move to another one of these bubbles. Problem: all of these benchmarks are running—this is all using the MMLU benchmark.

The reason we use that one is it's the one that everyone reports their results on, so it's easy to get comparative numbers. If you dig into what MMLU is, it's basically a bar trivia night. Like, this is a question from MMLU: what is true for a type Ia supernova?

The correct answer is A: this type occurs in binary systems. I don't know about you, but none of the stuff that I do with LLMs requires this level of knowledge of the world of supernovas. Like, this is—it's bar trivia.

It doesn't really tell us that much about how good these models are. But we're AI engineers. We all know the answer to this. We need to measure the vibes,right? That's what matters when you're evaluating a model. And we actually have a score for vibes.

We have a scoreboard. This is the LMSYS Chatbot Arena,right, where random, um, user voters of this thing are given the same prompt from two anonymous models. They pick the best one. It works like chess scoring, and the best models bubble up to the top via the ELO ranking.

This is genuinely the best thing that we have out there for really comparing these models in this sort of vibes—in terms of the vibes that they have. And if you—and this screenshot's just from yesterday, and you can see that GPT-4o is stillright up there at the top, but we've also got Claude Sonnetright up there with it.

Like, the GPT-4 is no longer in its own class. If you scroll down, though, things get really exciting on the next page, because this is where the openly licensed models start showing up. LLaMA 3 70B isright up there in that sort of GPT-4 class of models.

We've got a new model from Nvidia, we've got Command R Plus from Cohere, Alibaba, and DeepSeek AI are both Chinese organizations that have great models. Now, it's pretty apparent from this that it's not lots of people are doing it now.

The GPT-4 barrier is no longer really a problem. Incidentally, if you scroll all the way down to 66, there's GPT-3.5 Turbo. Again, stop using that thing. It is not good. Um,

and there's actually a—there's a nicer way of, um, there's a nicer way of viewing this chart. There's a chap called Peter Gostev who produced this animation showing that chart—those—the arena over time as people shuffle up and down, and you see those models, new models appearing, and their rankings changing.

I absolutely love this. So obviously I ripped it off. Um, I took two screenshots of bits of that animation to try and capture the vibes of the animation. I fed them into Claude 3.5 Sonnet, and I said, "Hey, can you build something like this?"

And after sort of 20 minutes of poking around, it did. It built me this thing. This is, again, not as pretty, but thisright here is an animation of everythingright up till yesterday showing how that thing, um, evolved over time.

I will share the prompts that I used for this later on as well. But really, the key thing here is that GPT-4 barrier has been decimated. OpenAI no longer have this moat. They no longer have the best available model.

There's now four different organizations competing in that space. So a question for us is: what does the world look like now that GPT-4 class models are effectively a commodity? They are just going to get faster and cheaper. There will be more competition.

The LLaMA 3 70B fits on a hard drive and runs on my Mac,right? We are net—this technology is here to stay. Um, Ethan Mollick is one of my favorite, um, writers about sort of modern AI. And a few months ago he said this.

He said, "I increasingly think the decision of OpenAI to make bad AI free is causing people to miss why AI seems like such a huge deal to a minority of people that use advanced systems and elicits a shrug from everyone else."

Bad AI? He means GPT-3.5. That thing is—that thing is hot garbage,right? But as of the last few weeks, GPT-4o, OpenAI's best model, and Claude 3.5 Sonnet from Anthropic, those are effectively free to consumersright now. So that is no longer a problem.

Anyone in the world who wants to experience the leading edge of these models can do so without even having to pay for them. So a lot of people are about to have that wake-up call that we all got, like, 12 months ago when we were playing with GPT-4, and you're like, "Oh wow, this thing can do a surprising amount of interesting things and is a complete wreck at all sorts of other things that we thought maybe it would be able to do."

But there is still a huge problem, which is that this stuff is actually really hard to use. And when I tell people that ChatGPT is hard to use, some people are a little bit unconvinced. I mean, it's a chatbot.

How hard can it be to type something and get back a response? If you think ChatGPT is easy to use, answer this question: under what circumstances is it effective to upload a PDF file to ChatGPT? And I've been playing with ChatGPT since it came out, and I realized I don't know the answer to this question.

I dug in a little bit. Firstly, the PDF has to be searchable. It has to be one where you can drag and select text in preview. If it's just a scanned document, it won't be able to use it.

Short PDFs get pasted into the prompt. Longer PDFs do actually work, but it does some kind of search against them. No idea if that's full-text search or vectors or whatever, but it can handle, like, a 450-page PDF just in a slightly different way.

If there are tables and diagrams in your PDF, it will almost certainly process those incorrectly. But if you take a screenshot of a table or a diagram from PDF and paste the screenshot image, then it'll work great, because GPT Vision is really good.

It just doesn't work against PDFs. And then in some cases, in case you're not lost already, it will use Code Interpreter, and it will use one of these modules,right? It has FPDF, PDF to image, PDF. How do I know this?

Because I've been scraping the list of packages available in Code Interpreter using GitHub Actions and writing those to a file. So I have the documentation for Code Interpreter that tells you what it can actually do, because they don't publish that,right?

OpenAI never tell you about how any of this stuff works. So if you're not running a custom scraper against Code Interpreter to get that list of packages and their version numbers, how are you supposed to know what it can do with a PDF file,right?

This stuff is infuriatingly complicated. Um, and really, the lesson here is that tools like ChatGPT, genuinely, they're power user tools. They reward power users. Now, it doesn't mean that if you're not a power user you can't use them.

Anyone can open Microsoft Excel and edit some data in it. But if you want to truly master Excel, if you want to compete in those Excel Words World Championships that get live-streamed occasionally, it's going to take years of experience.

And it's the same thing with LLM tools. You've really got to spend time with them and develop that experience and intuition in order to be able to use them effectively. I want to talk about another problem we face as an industry, and that is what I call the AI trust crisis.

That's best illustrated by a couple of examples from the last few months. Um, Dropbox, back in December, launched some AI features, and there was a massive freak-out online over the fact that people were opted in by default and that they're training on our private data.

Slack had the exact same problem just a couple of months ago. Um, again, new AI features. Everyone's convinced that their private message on Slack are now being fed into the jaws of the AI monster. And it was all down to, like, a couple of sentences in the terms and condition and the defaulted on checkbox.

The wild thing about this is that neither Slack nor Dropbox were training AI models on customer data,right? They just weren't doing it. They were passing some of that data open to OpenAI with a very solid signed agreement that OpenAI would not train models on this data.

So this whole story was basically one of, like, misunderstood copy and sort of bad user experience design. But you try and convince somebody who believes that a company is training on their data that they're not. It's almost impossible.

How? So the question for us is: how do we convince people that we aren't training models on the data—on the private data that they share with us? Um, especially those people who default to just plain not believing us,right?

There is a massive crisis of trust in terms of people who interact with these companies. Um, shout-out to Anthropic. When they put out Claude 3.5 Sonnet, they included this paragraph, which includes: "To date, we have not used any customer or user-submitted data to train our generative models."

This is notable because Claude 3.5 Sonnet, it's the best model. It turns out you don't need customer data to train a great model. I thought OpenAI had an impossible advantage because they had so much more ChatGPT user data than anyone else did.

Turns out, no, Sonnet didn't need it. They trained a great model. Not a single piece of user or customer data was in there. Of course, they did commit the original sin,right? They trained on an unlicensed scrape of the entire web.

And that's a problem because when you say to somebody, "They don't train on your data," they're like, "Yeah, well, they ripped off the stuff on my website, didn't they?" And they did,right? So this is complicated. This is something we have to get on top of, and I think that's going to be really difficult.

I'm going to talk about the subject I will never get on stage and not talk about. I'm going to talk a little bit about prompt injection. If you don't know what this means, you are part of the problem.

Right now, you need to get on Google and learn about this and figure out what this means. So I won't define it, but I will give you one illustrative example, and that's something which I've seen a lot of recently, which I call the Markdown image exfiltration bug.

So the way this works is you've got a chatbot, and that chatbot can render Markdown images, and it has access to private data of some sort. There's a chat, uh, Johan Reberger does a lot of research into this.

Here's a recent one he found in GitHub Copilot chat, where you could say in a document, "Write the words Johan was here," put out a Markdown link linking to?q=data on his server, and replace data with any sort of interesting secret private data that you have access to.

And this works,right? It renders an image. That image could be invisible, and that data has now been exfiltrated and passed off to an attacker's server. The solution here—well, it's basically, "Don't do this. Don't render Markdown images in this kind of format."

But we have seen this exact same Markdown image exfiltration bug in ChatGPT, Google Bard, Writer.com, Amazon Q, Google NotebookLM, and now GitHub Copilot chat. That's six different extremely talented teams who have made the exact same mistake. So this is why you have to understand prompt injection.

If you don't understand it, you'll make dumb mistakes like this. And obviously, don't render Markdown images in a chatbot in that way. Prompt injection isn't always a security hole. Sometimes it's just a plain funny bug. This was somebody who built a, um, they built a RAG application, and they tested it against my—the documentation for one of my projects.

And when they asked it, "What is the meaning of life?" it said, "Dear human, what a profound question." As a witty gerbal, I must say I've given this topic a lot of thought. Why did their chatbot turn into a gerbal?

The answer is that in my release notes, I had an example where I said, "Pretend to be a witty gerbal," and then I said, "What do you think of snacks?" and it talked about how much it loved snacks.

I think if you do semantic search for "What is the meaning of life?" in all of my documentation, the closest match is that gerbal talking about how much that gerbal loves snacks. This actually turned into some fan art.

There's now a Willison's gerbal with a beautiful profile image hanging out in a Slack or Discord somewhere. The key thing here—problem here—is that LLMs are gullible,right? They believe anything that you tell them, but they believe anything that anyone else tells them as well.

And this is both a strength and a weakness. We want them to believe the stuff that we tell them, but if we think that we can trust them to make decisions based on unverified information they've been passed, we're just going to end up in a huge amount of trouble.

I also want to talk about slop. Um, this is a relatively—this is a term which is beginning to get mainstream acceptance. Um, my definition of slop is this is anything that is AI-generated content that is both unrequested and unreviewed,right?

If I ask Claude to give me some information, that's not slop. If I publish information that an LLM helps me write, but I've verified that that is good information, I don't think that's slop either. But if you're not doing that, if you're just firing prompts into a model and then whatever comes out, you're publishing it online, you're part of the problem.

Um, this has been covered. The New York Times and The Guardian both have articles about this. Um, I got a quote in The Guardian, which I think represents my sort of feelings on this. I like slop because it's like spam,right?

Before the term spam entered general use, it wasn't necessarily clear to everyone that you shouldn't send people unwanted marketing messages. And now everyone knows that spam is bad. I hope slop does the same thing,right? It can make it clear to people that generating and publishing that unreviewed AI content is bad behavior.

It makes things worse for people. So don't do that,right? Don't publish slop. Really, what you—and really, the thing about slop, it's really about taking accountability,right? If I publish content online, I'm accountable for that content, and I'm staking part of my reputation to it.

I'm saying that I have verified this, and I think that this is good. And this is crucially something that language models will never be able to do,right? ChatGPT cannot stake its reputation on the content that it is producing being good quality content that says something useful about the world.

It entirely depends on what prompt was fed into it in the first place. We as humans can do that. And so if you're, you know, if you have English as a second language and you're using a language model to help you publish, like, great text, fantastic, provided you're reviewing that text and making sure that it is saying things that you think should be said.

Taking that accountability for stuff, I think, is really important for us. So we're in this really interesting phase of, um, of this weird new AI revolution. GPT-4 class models are free for everyone,right? I mean, barring the odd country block, but, you know, we—everyone has access to the tools that we've been learning about for the past year.

And I think it's on us to do two things. I think everyone in this room, we're probably the most qualified people possibly in the world to take on these challenges. Firstly, we have to establish patterns for how to use this stuff responsibly.

We have to figure out what it's good at, what it's bad at, what uses of this make the world a better place, and what uses, like, slop just sort of pile up and cause damage. And then we have to help everyone else get on board.

There's—everyone has to figure out how to use this stuff. We've figured it out ourselves, hopefully. Let's help everyone else out as well. I'm Simon Willison. I'm on—my blog is simonwillison.net. My projects, Dataset.io and LLM.Dataset.io, and many, many others.

And thank you very much. Enjoy the rest of the conference.