The Model Isn’t Wrong—You’re Just Bad at Prompting

Hey everyone, how's it going? Dan here, so excited that you're joining today. We're going to be talking about all things related to prompt engineering, and so some very quick background on myself: I'm the co-founder of PromptHub, the GitHub for prompts, based in New York.

I'm a Lakers fan, more specifically a LeBron fan, but that just means February has been a great month for me, and I'm a marathon runner. And so today we'll be covering a lot of ground, a lot of templates, a lot of free stuff that should be hopefully pretty helpful that you can go and take and start messing around with.

And so we'll talk about why prompt engineering is still important, why Chain of Thought prompting has been so revolutionary, especially when it comes to test-time compute, few-shot prompting, using LLMs to help you write prompts, via Meta Prompting, how prompt engineering with reasoning models is actually very different, and a bunch of free resources templates to grab.

So I usually include a slide like this in any of my talks, and I waiver from doubling down and including more of these and completely removing it. But, you know, I think the meme in the beginning was that, you know, why do you even—why is prompt engineering even a term you can just tell the model to do?

But I think anyone who's ever actually shipped an LLM-based feature has known it's much more nuanced than that. Even just trying to understand what you want the model to do is a challenge. And I think it's just a really good starting point for folks, of course.

It's the easiest and most accessible way to get better outputs from LLMs, and it's a part of the med a greater system,right? We all have access to the same models, but the prompts, the architecture, everything around that is how we can also have a competitive advantage in our product, whatever we're building.

I think, you know, something that was mentioned in Anthropic's recent paper or blog post about agents is that going for the simplest solution, I think, is really key to keep in mind. It's really easy to kind of run away with these things when you're working with LLMs and think about everything you can do, and that's great.

But, you know, just spending an hour trying to mess around with a prompt and then determining that, you know, it's impossible to solve whatever you're trying to do via prompt engineering, and you need to do more complex RAG or things along those lines, I think is not super smart.

You need to give it some time here, because if you can do it, it's much simpler to manage. And so we'll cover just two main methods: Chain of Thought and Few Shot Prompting, just because I think they're the most effective and most topical.

There are a bajillion more. A lot of them fall underneath the umbrella of general reasoning prompts. We've covered most, if not all, of these, and they're all available as templates in PromptHub as well. They can check it out for free.

So what is Chain of Thought Prompting? It's basically when you instruct the model to reason or think about the problem or a solution before actually jumping into whatever that answer is. And so it breaks down problems into sub-problems.

You get a glimpse into how the model is thinking, which can be helpful for troubleshooting. It's widely applicable, of course. You can use it kind of with any model. It's easy to implement as well. And it's so powerful that it's now kind of being built into these reasoning models, and so you don't even really need to do it for those reasoning models.

And so the classic kind of zero-shot way to do this is just to add something to your prompt that will make the model think a little bit before just giving you the output. You know, you want it to just generate some sort of kind of reasoning token beforehand and think step by step what's the classic one.

You know, take a breath and think it through, things along those lines. Another very popular way to do this is by having few-shot examples of those reasoning steps. And so if I'm having a prompt to solve math problems, I can include another math problem in the prompt and show the reasoning steps I went into solving that.

And of course, you can use LLMs to generate these reasoning chains as well. So there's something called just automatic chain of thought, which is a framework that's a little bit more involved. There's another one called AutoReason, which is just a single prompt here where you pass your task or question, and it will generate reasoning chains.

It actually has few-shot examples of reasoning chains in there as well. And you can try this out in PromptHub. And even the training template that DeepSeek used for its R1 model basically did this. It had to generate its thinking process within think tags and then use all these outputs, these generated reasoning chains, to train the model to be really good at Chain of Thought.

And so this is also available inside PromptHub. You can input your task, get a reasoning chain, and kind of be on your way. And that's totally free if you want to try it out. And as I mentioned, we have a ton of these in the platform, a ton of reasoning chains that you can go and check out.

Some of them are Chain of Thought. Some of them are other type of reasoning or verification methods as well.

But they're all pretty helpful, I would say, especially if you want to use them when you're dealing with, like, complex problems. So moving on to few-shot prompting. That's generally when you include examples of what you want the model to kind of mimic or do or to understand about your problem.

And essentially, you're doing a show rather than tell. And so in this example here, I'm telling the model that I have this client. We need to, like, generate some content for it. Here's a brief. Here's the related content.

Here's a brief. Here's the related content. And then I say, "Here's the brief," and then the model will fill in this content here. And so rather than trying to encapsulate my client's tone or style by sending an input and output example, a brief, and a piece of content, I can kind of teach the model exactly what I want.

The great part of this is that you get most of the gains from just, like, an example or two. Almost all the graphs kind of look like this when you're looking at number of examples versus performance. And sometimes performance can even degrade once you have, like, a bunch.

But it's great for builders because you only need, I'd say, one or two. You kind of want to have them be diverse and cover your bases of different inputs you could expect that the model to handle. But yeah, you don't need many of them.

And so next up is Meta Prompting. You know, I think it'd be silly as some people who are working with LLMs to not use LLMs for this part of the process. So Meta Prompting is basically just using an LLM either to create a prompt, refine a prompt, improve a prompt, whatever that might be.

There are a ton of frameworks for this out there. Some of them require you to have coding knowledge. Some of them don't. There are a bunch of free tools as well, which, of course, are very user-friendly. Anthropic has a great one.

OpenAI has one inside of their playground, and then we also have one in PromptHub. The difference with ours is you can select which model provider you are using, and it will run a different Meta Prompt because a prompt that is good for OpenAI models might not be the same as Anthropic.

And so we tailor it a little bit for you as well. And then we also have a way that you can kind of iteratively work with a kind of like a co-pilot inside PromptHub. It's built off very similar things to TextGrad, where you can run prompts, give feedback.

So this is another free tool that you have to your disposal as well, because prompt engineering is something that we can use help with. So why not leverage LLMs? And so moving on to kind of to stuff that's much more apparent now and more recent is that reasoning models are very different, both in terms of how they work and how you prompt them.

So Microsoft released a paper earlier this year about their Med Prompt framework. It's not super important, but basically they ran a prompt engineering framework with O1 and found that adding examples led to worse performance. And the researchers at DeepSeek when building R1 found this as well.

The few-shot degraded performance. And OpenAI kind of mentioned this when they first released O1 preview, saying that you need to be careful when providing additional context because it can kind of overcomplicate things and confuse the model.

And so you got to be careful with examples. But if you need to want to increase performance, there's been a lot of research that has shown that the more reasoning a model does, the better the output could be.

So in that same Med Prompt paper, they had a prompt that was, you know, quick response, and then a prompt that prompted the model to think more. And they saw that better result when the model was thinking more from extended reasoning.

And the folks at DeepSeek saw this as well. So as they continued to train the model, the length of the response and the thought process increased, and then also this will, in turn, increase accuracy and performance as well.

And so overall, when you're using reasoning models specifically, minimal prompting, nothing can really. Like a really good, clear task description. I would encourage more reasoning if you're having trouble kind of getting maybe that last bit of performance. Encouraging the model to reason more can be helpful.

Avoid few-shot prompting if you're going to do it. Start with, like, one, maybe only two examples. And then you don't really need to instruct the model on how to reason. It's kind of built in there. So doing that can actually hurt performance as well.

And so, as I mentioned, a bunch of free resources. We run a substack called Prompt Engineering Substack. We write on our blog. There's a bunch of prompts in the community from us and from other people. And so I hope this was helpful.

I hope you have a great time at the summit and have a great day.