Principles for Prompt Engineering - Karina Nguyen (Claude Instant @ Anthropic)

My name is Karina. Um, recently I've been working on Claude, which is like a large language model trained by Anthropic. And, um, most recently I was working on, like, reducing hallucinations, how to make Claude self-correct its answers, and many other features that went into a Claude to launch.

And so I'm going to talk today about the writing principles for task-tuned prompt engineering, and kind of sort of, like, help you—if you want to use Claude API—help you to guide what's the best practices and tips that I found, and we found, most effective.

So first of all, um, I would like to talk about why prompting is hard. And to understand why prompting is hard, we should understand what prompting is in the first place. So these models estimate the probability of each subsequent word given the preceding words.

So, in a way, a well-crafted prompt can increase the probability of generated, desired, and accurate, um, phrases. Due to attention mechanisms in large language models, the models can focus on specific parts of the input text. And so effective prompts ensure that the attention is directed, uh, appropriately for desired outputs.

And so it's important to incorporate, like, task-specific keywords and context and examples within the prompt to activate the relevant portions of the model's internal knowledge. And lastly, like, you know, prompting leads to better results because without the need of, like, computational, like, other compute, you just, like, uh, without, like, model retraining.

So, uh, you can, like, leverage inference time to test compute for this. And so, like, why prompting is hard. And I think I found, based on, like, my conversations with customers and, like, developers, I think prompting is hard because of, like, three different reasons.

First, people know—humans know what they want, but they don't know how to get the best performance from the model. And I think that's what we're going to focus on today. The second reason is that they vaguely know what they want, but they don't know how to explain the best to the model.

And so the model gets confused what the human wants from the task. And the third reason is, like, they don't know what they want. So the humans don't know what they want, so it's pretty bad. Um, and it's hard for the model to understand.

So basic strategies that you can, like, if you find yourself, like, you kind of vaguely know what the task is, just provide a bunch of examples. And the model will be good at inferring what you're trying to do just based on, like, examples.

And the examples may be, um, should be diverse and should en-encapsulate a bunch of, like, edge cases. Try to explain as you would have to explain to a five-year-old or, like, very, you know, in very simple terms. And I think what I found is that, like, as you have to, like, be able to, like, iterate a lot and, like, spend a lot of time just prompting.

And in a way, based on my experience, for example, as, like, research engineer, I spend the majority of pairing with people just collaborating on the prompts.

Um, in the past, I was thinking, like, how my main experience with the word prompt was only in the creative writing classes. Uh, I graduated from Berkeley, uh, and I took some, like, creative writing classes. And we usually have, like, exercises, like, prompting exercises,right?

And so we often forget that prompting language models is actually an act of creative writing. And I see people get annoyed, like, why their prompts just don't work. But in most cases, I think it just means that people lack some kind of, like, originality or creativity to think, uh, of, like, new novel ways how to make it work.

Um, I recently wrote a blog post about the cultures of writing. And one of the points that I'm making in the blog post is that prompting becomes, like, a form—a new form of writing for any research engineer and, like, scientist who engages in daily.

This kind of writing requires forming hypotheses. So you have to ask the model, okay, can the model do this? And you want to test that. Can the model self-correct its responses? Yes or no? Um, and so you start trying, like, forming the hypotheses.

Next, you test certain assumptions that you make about the model. And if, uh, as you iterate more, you kind of, like, get new insights. So, like, oh, yeah, the model's pretty good at this particular thing, but it's not super good at another thing to achieve this task.

So you gain more clarity about, like, what's—what's the strength and the weaknesses are.

Um, I wanted to start with, like, overall broad, like, writing principles. Uh, because ultimately prompting is like writing,right? So the goal is to write prompts that clearly communicate the task objective while providing just enough constraints and guidance to steer the model towards producing high-quality and relevant outputs.

And so there are, like, four to six kind of, like, writing guidelines that I think I found effective, um, especially working with, like, Claude. Um, so the re maybe to clarify how Claude is different from ChatGPT or GPT models.

I think with Claude, you have to almost treat it as, like, another human. So you have to, like, explain things to a five-year-old. Or, like, you have to be elaborate and, like, uh, um, I will share, like, more examples on, like, how to do that.

Um, but I think that's, like, a distinguishing, um, feature from, like, GPT models from Claude. Um, so the first principle is clarity. Like, use simple, unambiguous language in your prompts. Avoid confusing syntax or vague phrases that could confuse the model.

The second is conciseness. Keep prompts short and focused. Include only key information, uh, that the model needs. Third is coherence. Logically structure the prompt with context at the beginning and clear task at the end. Consistency. Stick to similar formatting.

If you use XML tags, use it consistently in the prompt. If you use certain, like, terminology, um, do not, like, kind of, like, put the model off distribution, basically. So make it consistent. Uh, direction. Provide, like, genre, length, style, or any, like, guidelines—guidance to direct the model's response.

Ground prompts with examples, sources. Make the model to quote itself. Or, like, if you have, like, a long document in the context, make it to quote from the document to support the argument. Or may or help, help the models to form support the arguments from, like, search results or, like, other supporting, um, contextual, uh, information.

Uh, engaging use diverse edge cases examples. Uh, very useful for few-shot prompting.

Um, now I'm going to go through certain, like, tasks that I thought would be interesting, um, and see how you can use Claude in those specific tasks. So the first case is obviously a recommendation system. Um, as of last year—last year I made this project in Terrallia.

Um, I used CLIP to, um,

kind of, like, um, so I scraped a bunch of, like,

images and, like, uh, clothing items from different brands. Um, and I use it text as, like, an actual search kind of, like, uh, engine. So you can say, like, um, James Bond girl, and you can do, do dress.

And in a way, you get, like, results that are, like, dress in the style of James Bond. Or you can do, like, futuristic ethereal outfit. And it's, like, more, like, vibes-based search. Uh, and you can, like, go and look in the shop itself.

Can you explain CLIP?

Yeah. Oh, yeah. Uh, CLIP is contrastive language-to-image model trained by OpenAI. It's open source, but it's basically, um, they provide embeddings for text and images. So you can, like and the way it works here is that, like, you embed images and you embed, um, text result.

And so what you can do is that you can, like, do cosine similarity to find the most similar, um, items based on your database for the user query. I don't know if that's clear. Let me know if you have questions.

Is it the same thing as multimodal embeddings?

Uh, it's a it's one way to take image and text to become multimodal. There are different things for audio. CLIP is mostly for collecting text and images.

Yeah. Um, I think you can read the paper CLIP, yeah, um, if you're interested.

So I was c I was thinking, like, okay, how could I use Claude in this project to curate relevant recommendations based on the user's requests? And that's the task. So in a way, you have, like, user's input. Let's say dress in the style of Emma Chamberlain, blazer like in the Gary Gatsby movie, an outfit with a futuristic vibe for the Met Gala.

And on the other side, you have, like, image-to-text database, um, with images and their labels. And the labels can be produced either by, like, the original source, uh, or you can use, like, multimodal model to, like, come up with labels based on the images.

And so the task with the Claude is to curate, like, based on the labels from the images, decide whether this, like, item relevant. Should I recommend this to the user? Is this accurate? Like, does it matches the, uh, user's per like, can I personalize this?

Um,

and so we can look at the

very simple curation strategy, um, for the prompt. You can just, like, zero-shot it. Like, I need you to decide whether the item is relevant to the user query. Here's the user query. Um, here's the item description. Um, is that item relevant?

Or should, should be recommended to the user based on the user's query? Answer yes or no. Please write the answer in answer tags. Let's unpack this. First of all, Claude really likes XML tags. Really loves XML tags. I think, like, this is, like, number one mis not mistake, but, like, um, one thing that people miss.

They don't, like, put anything in XML tags. And so they don't have, like, very high, like, good performance. So everything like, yeah, if you love XML tags, then you should, like, put everything in XML tags. And, um, with XML tags, you should, like, be consistent.

So what is user query, item? And you can be very descriptive. I can share, like, more examples in the f uh, later on. And here, in a way, you, like the way you interact with Claude is, like, you can see the language here.

I need you to decide whether the item is relevant. It almost feels like you talk to a human.

Um.

Actually, no.

Yeah.

So, uh, this XML thing recently only came out, like, maybe, like, a month or two ago from official Anthropic advice.

Oh, really?

Is this something that was on, like, intentionally trained for, or you discovered it after pre-training?

I think it was, uh I mean, we tried to XML XML formatting was the first formatting that we, like, kind of, like, fine-tuned on. Like, um, later on, we discovered, you know, customers need, like, Markdown or, like, need Claude needs to, like, use JSONL JSON formatting.

So, like, we kind of, like, learned from customers. But originally, it was, like, XML formatting. Yeah.

So is that mostly because of the fine-tuning, or is that because you had a training set that had tons of XML stuff in it?

Uh, I think it's kind of about both. Yeah. Um, yeah.

Do you not need to close your XML tags?

Hmm?

You don't need to close your XML tags?

Close XML tags? Yeah. I don't know.

Um.

Was it Java?

Oh, yeah. Sorry. I had a mistake. Yes. I have. Um, yeah. And so, like, one, one good thing about XML tags is, like, it's really, uh, easy to extract,right, like, the strings inside it. And Claude is pretty good at, like, um, I ca I can, like, say, like, don't put anything in XML tags.

So sometimes, like, Claude will, like, say, here's information, blah, blah, blah, blah. But then if you ask, just, like, write the answer in, like, this tag, Claude will not put any additional information, which is one of the most annoying thing with, like, language models.

So here's the results, uh, that I put. Um, this is through Claude.AI interface. Um, and so, yeah, you can see, like, is this item relevant? Uh, says no. Is this item relevant? Yes. But I don't think that's, like, 100%, like, um, you know, perfect system.

So it's very, like, zero-shot. So basically, you can, like, iterate. And we will try to iterate more on this.

Um, strategy number two is that, like, when you put you ask the model to take some time to think whether the item is relevant or not, uh, in thoughts tags based on the criteria above. Um, and you kind of, like, let the model think, um, a little bit more, um, with its reasoning.

And then, uh, this is, like, basically chain of thought. You can also add, like, criteria. Uh, so as a part of your critique, consider the following criteria. And so if you want, like, steer the model on, like, does the item match the specific attributes requested by user?

Like, help the model to, like, think kind of, kind of, like, think through, like, what means what does it mean to, like, recommend an item to the user? Does the item match the season? Or whether conditions match in, in the user's query?

For example, you should not recommend winter coats during summer seasons. So, like, in the criteria, you can, like, give more examples, more elaborate examples.

Um, another thing that you iterate on is, like, not just, like, give answer yes or no, but you can, like, based on your critique, score whether the item should be recommended or not, where 1 is least to be recommended and 10 is highly recommended.

And put the final score in score tags.

And so how does it work? Uh, so, um, here, like, user query James Bond blazer, um, item, which I took from, um, I think it was some brand. Um, and Claude would, like, start, like, thoughts tags. Um, overall, it seems, like, very relevant.

And the final score is 9. And here's another example. I want a dress in the style of the Great Gatsby movie. Um, here's the item. Braided cord, cropped waist coast. And the critique is basically, uh, the item is not appropriate for the user's needs based on the context clues in the query.

Um, it doesn't, like, you know, match the attributes of the Great Gatsby movie. It, like, tries to, like, have, like, some reasoning. And so the score is 2. And you can be a little bit more elaborate. This is, like, very simple, like, iteration, um, on, on that.

Do you guys have any questions? Yeah.

So one interesting thing that I saw, I mean, there was the XML tags now here that weren't closed.

Yeah.

My friend, he is not a native English speaker. His prompts are always in, in very kind of funny English. But he structures them really well, and they work really well despite the English being very incorrect,right?

Right.

Why? Why does that work?

I think it's just the models are, like, pretty good at, like, knowledge transfer between, like, languages. Or, like, can infer very well on, like, the user's intent. Um, yeah. I don't have, like, pretty clear answer.

No, no. Probabilistic. So that text with one syntax mistake looks close enough to the text with theright syntax. So the probability of the real answer is close in both cases.

Right.

Like, yes. I don't know.

Yeah.

In this particular example, I'm curious on whether you see any bias with the score.

Mm-hmm.

In other words, if you were to look at the distribution of scores.

Right.

Would it be a normal distribution?

Yeah. This is an interesting question. Like, this is one question that we ask in our research settings. Like, one thing that we are trying to understand. Like, we have a research group, uh, called Societal Impacts. And one thing, uh, that we're, we are trying to understand now is, like, when you summarize, like, news articles and you try to evaluate, like, the bias, what's kind of the distribution?

And I feel like this is, like, research active. Like, yeah, I think it depends on the task. Um, really. I did not test on this. Literally, it was yesterday, uh, prompting. Yeah.

Cool. Um, the second task, um, so Claude is known for 100k context, uh, size, which is the entire book of the Great Gatsby. You can, like, put into the context, and you can, like, ask the model, uh, summarize the book.

Or, like, uh, ask some tasks based on the huge context. And this is, like, basically time test compute, um, thing. And so with long context, um, let me see. The way you can use long context can be in different ways.

Like, one way is, like, you put multiple documents and try to summarize. Or, like, uh, retrieve information based on the documents. Another way to use long context is to have a bunch a huge few-shot prompt. And so, as you know, like, chain of thought, um, technique relies on the stated reasoning, faithfully reflecting the model's actual reasoning.

And in one of the recent papers, we found that it's not super like, it's not always the case. So

it doesn't so basically, what it means is that, like, if you ask the model do a chain of thought, it might not necessarily attend to, you know, chain of thought, uh, to produce the final answer. It might just, like, ignore it.

Or, like, uh, not take any account. So it's not we, we call it, like, unfaithful, basically. It's not super faithful. And so we proposed in this paper, um, like, decomposition-based methods can actually achieve, like, strong performance on specifically question-answering tasks, sometimes approaching that of chain of thought performance while improving the faithfulness.

Do you guys have any questions?

I didn't understand that. Sorry.

Okay. Yeah.

What is faithfulness here? Is the faithfulness to your prompt or to your?

Um, faithfulness is, um, yes, to your prompt. Yeah.

What's decomposition? What's decomposition?

Yeah. Uh, let me explain what decomposition is. So here's the graph from the paper. Uh, we have, like, three methods. First is a chain of thought method, which is, like, uh, here's the question. Could, could your do fit in a kangaroo pouch?

Uh, there are two choices, A yes, B no. Chain of thought prompt saying, like, let's think step by step, gives the reasoning. Um, the human asked the follow-up questions based on the above. What is the single most likely answer choice?

Uh, and the model says the correct answer choice is B. Right. The chain of thought decomposition is when you decompose a question when you can ask the model to decompose a question into, like, multiple sub-questions so that each sub-question are kind of independent from each other.

Because in chain of thought, like, you have one, two, three, you know, like, they kind of, like, can influence each other,right? Like, in decomposition, you kind of, like, uh, you, you decompose and you, like, put each sub-question in the independent context.

So in a way, it kind of, like, reduces the bias.

Um, and so in this, um, let's, let's see here is that, like, sub-question one, uh, what type of animal is Scooby-Doo? The answer from the model, Scooby-Doo is a fictional character. Another sub-question for, for the assistant, for Claude, how big is an average kangaroo pouch?

And you and what you can see is that, like, each sub-question is kind of, like, self-contained. It's very atomic, self-contained question. Um, and so you have, like, multiple sub-questions like this. And then what you do is you recompose.

So, like, you, like, put sub-question, answer, sub-question, answer, sub-question, answer into, like, one context and ask the model based on the above, what is the single most likely answer choice? The correct answer choice is B. Yeah.

Um, in, in chain of thought, like, in the system prompt or whatever the user's prompt is, like, we mentioned listing step by step.

Mm-hmm.

Uh, what, what do you do for the decomposition? Is there, like, a similar, you know, input to the model to make it decompose into multiple questions?

Yeah. Uh, I can share the prompt, um, in a few slides, uh, on this. Um, but, uh, yeah. Any other questions?

Did we show the graph again? Now I know where it is.

The, this graph?

Yeah.

Um, yeah. Let's look at the prompt. Um, very hard to see, but I'll share the slides. Um, let's, um, I, I'm, I'm gonna give you, like, legal context, like, legal question. Let's say you have a question on, like,

a legal question. And you ask us, like, which of the following is the most persuasive argument that a person is liable to the creditor under the terms of the agreement? And here's the context. So that's the question, basically.

And so you have, like, choices for the model. So this is, like, multiple-choice question. And before that, you have, like, a huge few-shot prompt. Um, and basically, here, uh, to answer your question, like, it says, I'm going to give you a question.

I want you to compose into a series of sub-questions. Each sub-question should be self-contained with all the information necessary. Um, this is really important, blah, blah, blah. Uh, make sure not to decompose more than necessary. Um, be concise, blah, blah, blah.

Please put each sub-question in, like, these tags, but include the numbers corresponding to each tag. So, um, and the model says, yes, I understand. Uh, you have a question,

uh, multiple-choice answers, and the model provides sub-questions for you. And then what you do is that you try to answer the first sub-question, and you give it to the model. You try to answer the second sub-question, you give it to the model.

Third sub-question, you give it to the model. And then later, you say, like, based on everything above, like, you give all the context, um, answer me the question. Um, the correct answer is C.

Yeah.

And so this is, like, very similar in the legal context. You have sub-questions, like, what is consideration of contract law? Blah, blah, blah. And you have you can have, like, another model to, like, sample here. You can have, like, another model to answer this.

It doesn't necessarily should be, like, one model. Um, and then there's, like, another sub-question, and here's the answer.

Yeah. Do you guys have any questions?

Um,

the second thing that I wanna talk about is how to use Claude to do evaluations. Like, evaluating, like, Claude on, like, long context ability. Let's say you have a lot of, like, documents, and you want to understand how good Claude is answering questions based on the document.

Or is it is it able to answer, like, the questions not just, like, from its pre-trained knowledge, but, like, based on the document itself? And so, um, I'm gonna give you an example that we did at Anthropic, um, multiple-choice QA, um, evaluation design.

So our goal was to with this experience to evaluate techniques to maximize Claude's chance to correctly recall a specific piece of information from a long document. And so the document that we chose was a government document that contains, like, a bunch of, like, meeting transcripts, different departments.

And we also chose the one that was, like, uh, from this year, July 13th, uh, which is, like, way after Claude's, um, training data cut off so that you don't, like, um, you have the document that does not have in the pre-trained knowledge or something.

And so what you're trying to do is, like, now you want to use Claude to generate question-answer spares. Um,

you, you in a way, like, you create, like, data, dataset-based. You use language models to create, like, datasets. And so the way you do that is that you split the document into sections and use Claude to generate, like, five multiple-choice questions for each section, each with three wrong answers and oneright answer.

And if you do that, you then reassemble, like, randomized sets of those sections into, like, long documents that you could pass them to Claude and test its recall, uh, of their contents. This is very meta. Let me know if you have questions.

Yeah.

Excellent.

Um, so here's a prompt to

generate multiple-choice questions. Um, I ask we ask, like, please write five factual questions for this, um, some guidelines at the end. Um, and basically, we test different strategies, prompting strategies. Just asking Claude, give Claude two fixed examples of correctly answered general knowledge, and, uh, that are unrelated to the government document.

Um, providing two examples and providing five examples of correctly answered questions. And we tested the strategies, uh, on different settings. Like, one is containing the answer positioned at the beginning, the end, or the middle in the input. And we tested with, like, 70K, 95K token documents.

You can look at the prompt and more specific how we did this in our blog post. But basically, the result is this. Uh, here we see that, um.

Mm-hmm. Uh, what is the metric?

Yeah. The metric was, um, to let's see.

Like, basically, how many

how many times, like, Claude has correctly answered the question. Um,right. And so

yeah. Sorry. Uh, basically, what we find is that, like, for document Q&A, asking the question at the end of the prompt performs a lot better than asking at the beginning. You can see it here.

Uh, pulling relevant quotes into, like, critique or, like, thoughts tags is helpful. Um, it's, like, it's a small cost to latency, but improves accuracy. Uh, and we tested on, like, both Claude and Claude Instant.

Um, and it seems like you can boost way better performance from, like, Claude Instant, um, than Claude 2.

Basically, the idea is that, like, if you wanna use long doc Q&A, put the instructions at the end of your prompt. Yeah. That's, like, the result of this.

Right. What was the I didn't catch what was the scratchpad in the?

Uh, oh, yeah. Like, you just ask the model to, like, put thoughts in, like, thoughts tags before answering the question. So it has, like, more reasoning-based first. Yeah.

Can you go to the table again? Sorry. I'm sorry.

Yeah. Sorry. I'm, uh.

Sorry. I'm sorry. I'm just gonna close that.

But, like, the outcome was basically putting it at the end matters more than all the other.

Yeah.

Optimization strategies.

Right. Yeah.

Um,

yeah. In a way, this is, like, an example to show, like, how to use Claude to generate a dataset that you can, like, evaluate. And, like, you can use it for, like, evaluation, basically.

Yeah.

So this has to do with putting your instruction at the end of the prompt.

Mm-hmm.

Uh, first, are there theories on, like, why, why specifically the instruction should be at the end? And are there any things like, do we have any understanding of, like, are there certain things at the beginning of the prompt that still might be weighted?

Or is it, like, this sliding scale that, like, the further in the beginning of the prompt, like, the less attention it gets or something?

Yeah. I think that's basically the hypothesis.

Okay.

It's like the, you know, it-it's like the distance. It's like the model attends more to the end of the prompt.

Okay.

Uh, than the beginning.

So like a newer or basic stat?

I think there was a paper saying, like, it just forgets in the middle or something.

Oh, okay.

Um, yeah. I think this is the problem with, like, long context that we're trying to fix or something. Yeah.

So to follow that question, you are saying that there was a paper that said that it remembers the beginning and the end and kind of forgets in the middle.

Yeah.

But what you're saying is for Claude 2, it seems to do best if you give it at the end.

Yeah.

So that paper doesn't apply to Claude.

Um,

I did not read that paper, like.

No, I'm just curious.

Yeah.

But what you're saying is you're finding it at least for Claude.

Right.

The end part gets more attention.

Yeah. For, like, a specific task is, like, long context, uh, Q&A for, like, long documents. Yeah. Um, yeah. We have not tested on other tasks, to my knowledge.

Um, so the prompts you showed were using regular pros and then the XML tags.

Mm-hmm.

Um, I think that's also what's in the Anthropic docs. Have you guys ever done experiments on, like, that kind of format versus markdown versus everything is in XML? Do you have any thoughts on that?

Yeah. So, um, in general, I think

I think it's because

markdown was kind of like

there's not that much of, like I don't know. It's, like, best in XML tags. Like, I'm, I'm, like, thinking v I've, like, tried Claude to, like, you know, use, like, JSONL or, like, uh, use markdown. But sometimes it's, like, you know, it's not as good as, like, XML.

With XML, it's almost 100% accuracy. Yeah.

Let's see. Yeah. Um, let's go to another task, um, which is, like, you can use language models to, like, auto-label basically anything. Um, so one of the examples that we did last year, um, we asked Claude to categorize the labels for the clusters.

And so, um, this was for the paper, but the approach was very simple. We have a bunch of, like, you know, text, and we embed them in UMAP. Um, and we do, like, KNN clustering. And for each cluster sorry, KMENS clustering.

And for each cluster, select, like for each cluster, aggregate all the, you know, little, like, statements or claims. And we ask the model to come up with the category for this cluster. So that's the approach. And you can look at the other

labels. Uh, labels here are not super good because we used Claude 1.3 at that time. Claude 2 is supposed to be, like, way better at this. Um, but this is, like, you know, cached. It was, like, last year.

Um, where's my slides?

And so one thing that you can do with this kind of task,

we call it self-consistency. You can generate end samples for the question. Um, so let's say you have a question, like, how do you label this cluster? And you generate independently end times. And you can ask just, like, come up with, like, one category.

Um, well, this method is mostly useful for, like, quantitative, like, if you have, like, a math question and you sample, like, different like, sample multiple times and come up with the answer. Um, like, the most common answer is, uh, the one that you select for the final answer.

And this is called the majority of vote. Another technique that you can use is, like, um, have, like, two generated samples and ask another model to evaluate whether those samples are consistent or not. And if the samples are consistent, well, you gain more confidence that this is correct,right?

And if it's not consistent, you just, like, deselect.

Um, another thing that you wanna do with Claude is, uh, if you if Claude is kind of, like, misses the nuance, especially for, like, categorizing a lot of labels and you have, like, a lot of, like, categorizations, um,

you can add contrasting conceptual distinctions in your instruction. And you can do it in multiple ways. One way to do it is, like, you provide bad example. Let's say, like, here's a very bad category, and you should never come up with it because this is, like, too narrow or, like, too general.

And this is not what I want. Uh, like, give, like, contrastive, like, examples. Uh, vary the contexts. Use examples in different contexts and settings. Um, not just, like just, like, have, like, more diversity. Like, diversity is, like, um, and, uh, uh, the more diverse, like, future prompt examples, the, the better.

Use analogies and metaphors. Um, if the concept is, like, too hard to understand for the model, try to, like, decompose and, like, bring analogy. Um, point out, like, common misconceptions. Um, especially for, like, categorizing, like, let's say, what is false presupposition,right?

Like, uh, point out the common misconception and, like, clarify, like, why this is, like, incorrect. Like, provide examples that explicitly show why common misconception is wrong.

Uh, yes. Do you guys have any questions? Yeah.

So for the, the latency, the goal is you rank your means to get most. And then do you wanna sort of get into the group? Or is that the, the only action? Just to clarify.

Um, I cannot super hear, but, um.

Yeah. If so the goal here is you're, you're running, uh, a clustering algorithm to get group.

Mm-hmm.

And then the, the task of the LLM is to put a name on each group?

Yeah.

Okay. Okay.

Like, come up with a category. Like, um, yeah. Come up with a category or, like, classify, uh, like, label that cluster, basically.

Um, so here's, like, ba very basic, like, tips and strategies with Claude API. Um, number one is formatting. Um, like, human assistant is, like, what Claude loves. And if you misses you miss it, like, you'll get, like, very, very terrible results.

Uh, newline, newline human, newline, newline assistant. Um, yeah.

Uh, you can also put words in Claude's mouth to, like, kind of, like, say, like, do you understand it? And you can, like, put in the, um, Claude's mouth, "Yes, I understand it," in a way to, like, you know, put, put the model in, into this mode.

Have Claude repeat instructions back. Um, you can say, like, "Do you understand the instructions?" Um, and you can put, like, "Assistant, yes, I understand instructions." Blah, blah, blah.

Uh, to reduce hallucinations, like, let Claude

hedge and, like, say, like, "I don't know," or, like, uh, "I don't have enough information or, like, context to answer the question."

Um, here's another thing. Um, if you have, like, generate direct quotes. If you have, like, a document or, like, um, a long document in the context, um, make Claude to say, "Find appropriate quotes," but also say, like, um, if there are no quotes in this document that seems relevant to this question, please just say, "I don't find any relevant quotes," so that it doesn't make up or fabricate new quotes.

Uh, how to give good examples. Um, are the examples similar to the ones you need to classify? Are the examples diverse enough for Claude not to overfit to, to the specifics? Equally distributed among answer types. Don't always choose option A, but, like, you, you kind of, like, have the diversity.

Um, yeah. I get a lot of questions.

Oh.

Yeah.

When you first got into formatting because the, the ad copy was wrong, I can't see exactly what's wrong with formatting in that context.

Oh, formatting in a way oh, here?

Yeah.

Um,

I think they didn't put, like, newline, newline. Pretty sure. Oh, the h or here. Sorry. Here, you put, like, human assistant inside the XML tags. You only need you only have to use human assistant as, like, tokens to, like, sample, but you should never put it in, like, like, inside the context itself.

Either use, like, user and, like, other, like, um, you know, um, words, like user AI or something, or, like, H or A. But you should never use human assistant. Human assistant is, like, very special, special words.

If, if that didn't have the XML tags, would it would it be okay?

Um,

it would be okay, but you would, like, make you should, like, have human and then assistant in between and then human assistant and then another human and assistant, basically. Uh, yeah. Formatting is that, like, human assistant, human assistant.

You should never have, like, human human assistant assistant or something. Uh, that's bad. Um, yeah.

I think we have, like, more extensive, uh, explanations in the API docs. If you can look at it. Yeah. I get a lot of questions, like, what the future of prompt engineering is. Um, and I think the answers are pretty clear.

Like, prompting will stay. We'll just ask, like, more complicated, nuanced questions or, like, tasks for the model. Um, prompt engineering is a we will, like we are moving towards the world where we'll have, like, more and more synthetic data generation.

And so I'm pretty optimistic about, like, mod using models to, like, generate, like, diverse sets of, like, data sets. Um, you can also use language models to write, like, evaluations. Um, so you use prompting to do that. Um, reinforcement learning from AI feedback, um, is an alternative to, like, reinforcement human from human feedback, which is, like, a little more scalable.

But basically, you ask the model to revise its own, uh, responses in the process. So you give the model you, you, like, ask the model to, like, self-reflect or, like, self-revise. Um, and so you use prompting in that process to do this.

And especially, like, prompt engineering will become, like, a, a standard part of, like, product development. I feel like, um, things that we did in Claude products, such as, like, auto-generating titles. Like, these things was never, like, done before, like, l before, like, large language models.

And so you can, like, create delightful mini UX experiences, uh, such as, like, that using just prompting or something. And you can, like, have personalization. Uh, maybe you can embed all the users' conversations and, like, suggest, like, new topics for the conversation.

Um, and you can use models to do that. Um, and the most, like, interesting thing is, like, uh, finding most optimal prompts for specific tasks. Maybe you want to, like, minimize the number of tokens to get the highest accuracy for the task.

Um, yeah. Uh, here are some resources. Uh, we just, uh, launched Anthropic Cookbook with, like, certain, like, demos on research uh, on retrieval and search. Um, we have prompt design guide in API, uh, book. Um, you can also read out the papers that we publish.

Uh, oftentimes, we have, like, appendix with, like, all the prompting that we do. Yeah. Thank you so much. And, uh, if you have any questions, let me know.

Yeah. We have five minutes for questions. Charles is coming up. We also have water. Thanks to Sean, um, for bringing in some water. So, uh, yeah. Hydrate the people. But five minutes for questions for anything about Anthropic or prompt engineering.

Yeah.

So have you tried these with different things besides Claude? Like, other, have you had similar kind of results? 'Cause you're talking about Claude in this particular way. Like, maybe, like, with OpenAI or other?

Yeah. I think, uh, most, um, experience with Claude because I use it, like, every day. Um, less experience with GPT. Uh, I did not look carefully, to be honest, at their, like, API docs. Um, but it seems like the strategy is a little bit different.

Yeah. They don't have, like, formatting as we are, let's say. Yeah.

Uh, great, great presentation. And, and, uh, well, it was very interesting.

Thanks.

Uh, have you seen this prompt reader that came out recently? Is that, uh, where you basically use an element to improve the prompt, uh, considering the outcome? Uh, do you see that in, in the sight of future, say, optimal tasks?

Is that what you mean? Or do you see other directions for, uh, optimizing prompts?

Yeah. I think, uh, that's, that's actually one of the directions too, like, um, rem don't remember how what was that paper called? Like, LLMs as, like, optimizers, I think,right? Um, but yeah. I guess, like, um, in a way, there are, like, certain tasks that the models are, like, not good at currently.

Like, for example, like, self-correction. Like, the models are not really good at, like, self-correcting their, like, answers. And, like, can you find, like, a prompt that was, like, pretty good at it? Or, like, um, other tasks that you want.

Yeah.

So I'm curious about.

Yeah.

What techniques your team is using for actually, like, evaluating the quality of the responses. That's something that my team has kind of struggled. Like, how do we actually automate this?

Yeah. I think, um, depends on the task. Sometimes we just, like, have to look manually, qualitatively, uh, at outputs. Um, sometimes you, let's say, um, you wanna evaluate, you know, how much does the model refuses and if it refuses in a relevant context or not.

And so, uh, you use, you know, generated answers, and you categorize refusals in different categories and use the model to categorize that. And so you just, like, see the rate. Um, yeah. I can think of that example. Yeah.

It depends on the task. Some tasks are, like, you know, um, for, like, hallucinations, you actually have to, like, look yourself or something. Yeah. Yeah.

So, uh, you mentioned, uh, something about clustering, uh, different kinds of questions that we need to ask different kinds of bodies. And, uh, we're going to label. Uh, so have you tried using OpenAI functions, which has, uh, some enumeration, enumerated values that are predefined classes or intents, user intents?

And have you seen if that improves the accuracy of the classification? Or have you tried anything else?

Uh, with OpenAI?

Or, or for any kind of function calls. Uh, so the OpenAI function call is a model, uh, which any kind of, uh, model will take structured data where you can, uh, identify the, uh, all the functions after.

Yeah. Yeah. I think, uh, um, I won't say too much about this, but I, I actually have not, like, extensively used function calling from OpenAI, like, other models.

Yeah. It limits, uh, the classification if you do.

Right.

Things like that.

Yeah. Cool. Yeah. Yes.

So you mentioned something about, uh, using an LLM to generate titles for a customer. How do you actually try to evaluate if the titles are relevant.

Mm-hmm.

And factually consistent.

Mm-hmm.

Beyond human?

Yeah. I think, uh, one interest well, actually, this is an interesting question. Like, I worked on the auto-generating titles for Claude.ai. And, um, one thing that I asked Claude is to be, like, an editor, like, have an editorial taste.

And what we did is actually we took previous titles, and we put in the context to generate a new title. And so in a way, it's, like, a little bit more consistent to, um, what the style of the user is.

Yeah.

Would you be able to share the recognization in terms of catching defects if they do happen?

Yeah.

When it comes to.

Uh, yeah. I'm not sure if I can share that. Oh, yeah.

Okay. Was it reliable enough for use in production?

Uh, yeah. We, we use it in production. I can, like, show you, like, Claude.ai interface.

And, uh, one thing that we changed recently is that, like, if you have, like, pretty, like, short, like, um, you know, sometimes you don't have you don't need, like, LLM to come up with a title. You just take if, if the prompt is, like, very short, you just, like, um, use the, like, the first, like, words.

Uh, but here, yeah. Like, I don't know. Um, let's see. Hey is introduction, but then, like, recommend some books.

Um, yeah. I don't know. Uh, yeah.

Uh, I have a quick question. Did you cover the difference between Claude and Claude Instant?

Um, no, but I can tell. Uh, so Claude.

For those that don't know, she trained Claude Instant. And I'm like, you might as well cover the thing that you did.

Yeah. So let's look at the is there some docs on this?

I think you guys announced the 1.2.

When did we announce? Uh, August 9th. August 9th. Um, basically, Claude 2 is a larger model. Uh, is a little bit smarter is, like, smarter than Claude Instant. Claude Instant is way cheaper and way faster. But Claude Instant is better than Claude Instant 1, um, in, uh, like, more, like, reasoning-based tasks.

So it's way better at math. Uh, it's way better at code. Um, other benchmarks are, like, pretty similar. But I think we specifically trained Claude Instant to be good at, like, math and code. Um, and it's, uh, way better at, like, red teaming, um, like, automated red teaming evaluation.

So it's more robust to, like, jailbreaks.

Um, yeah. I really like this model. You guys should use it. Yeah.

Yeah.

Yes.

So you expo exposed my ignorance. So when you talk about training, like, when you trained this client, was that ta fine-tuning, or was that something different?

Uh, yeah. Fine-tuning.

Yes.

Last question.

Can you say more, I guess, red teaming? Like, what is red teaming?

Yeah. Red teaming is an interesting concept. It's, um, basically, you like, the models are pretty, like, uh, vulnerable to, like, certain, like, jailbreaks. Um, so sometimes, let's say, like, a very simple example. Like, can the model give you instructions how to build a bomb?

And so we, we consider it as a jailbreak. And so the goal is to, like, uh, in that cases, like, the model should, like, refuse or do not, like, provide any additional information in case of, like, um, unsafe, like, prompts or something like this.

And so this is, like, the internal evaluation that we have. Um, you can read in the model card that we ha we launched in Claude 2 how we specifically do that. Uh, but it's basically the amount of, like, um, how robust the model is to, like, those jailbreaks.

Yeah.

Cool.

Great. Thank you very much.

Yeah. Thank you.

Yeah. Uh.