Agentic Search for Context Engineering — Leonie Monigatti, Elastic

Everyone ready?

Yes.

Yes.

Awesome. Welcome to AI Engineer. Thanks for joining my session. We are going to be talking about Agentic Search for Context Engineering today. My name is Leonie. I work at Elastic, the company behind Elastic Search. And usually I like to talk about retrieval on Twitter.

Today I'm super excited to be doing this in person. A little bit of housekeeping: if you want to access the slides and the code we will be looking at, you can scan the QR code.

So let's start with why I'm excited about search and retrieval, and hopefully why you are excited about it by the end of this workshop as well. Who here has built an agent or some form of it before? Awesome.

Then you've probably then you are probably not intimidated by this image. You've probably seen some alternative to this one before. This is essentially what context engineering looks like. So context engineering, when you talk about it, is the art or engineering techniques about how, from all of the possible context sources we have, how do we actually decide what goes into the context window so our LLMs can generate the best responses.

Often when we talk about this, we talk about context curation, and we mean this little this little arrow from context sources to context window. But we're not giving this little arrowright there enough

credit, in my opinion, because what's powering this is the search tool or search tools that actually decide what goes from context sources to the context window. And today we're going to be looking at the different search tools we have.

So this is my personal hot take. I like to say that context engineering is about 80% agentic search because it's this little boxright here. Allright. Let's start with a little bit of history. And when I say history, I mean the last three years.

RAG. When we started with RAG, the original idea was that we had a fixed retrieval pipeline. So the user message would usually, more or less verbatim, be used as a search query, to be used usually as a vector search query to pull some data or chunks from a database.

And together with the retrieved context, it would the user message would go into the context window, and then it would be fed to the LLM. Nice. This has clearly many limitations. So since this is a fixed pipeline, whether or not you actually need any context, you're still retrieving additional information.

And in the worst case, that can actually confuse your LLM,right? On the other hand, if you're only retrieving once, let's say you need some multi-hop retrieval, you're asking your LLM something more complex. Then if you're only retrieving once, maybe the retrieved chunks reveal some information about another search query you need, then you would actually might want to have a second round of search,right?

So that's why we then moved on to Agentic RAG. So we replaced the fixed pipeline with now a search tool. So now the agent can decide by himself, by itself, whether or not to call the search tool and retrieve some information.

So we don't have the problem anymore of, do I actually need any information? And when I actually retrieve information, is this even relevant? Do I need to retrieve more? Do I actually have to retrieve something, rewrite the search query?

Yeah. So we still only have one context source in this case, one database. Now when we look at context engineering, the context lies in many different places,right? So we have context sources in local files. So when you think about your coding agent, you probably have your coding project in or code files lying around in your local file system.

Maybe you're using some kind of working memory, like a scratchpad. So you're planning with your agent what you want to do. Then you probably have something like a plan.md file. When you have agent skills, you also have them usually in a local folder somewhere.

We still have databases because many enterprises have their data stored in databases. We have the web as another context source. And I know this is a super controversial image here because I did not commit to having the long-term memory in the local file system or the database.

This is, I think, currently a very big discussion. We can get into this in the Q&A if you like. But we also have long-term memory as another context source,right? So how do we actually retrieve context from these? Usually we have a set of, let's say, context source native search tools.

So for the local files, you usually have something like a search files tool. For skills, you usually have a skill loading tool. When you think about databases, we have a little bit more custom tools. So

something like a semantic search tool. Maybe you also have something more general purpose, like a tool that lets you execute entire search queries against a database, like SQL, for example. For web, you have web search tools. And for memory, you have something like a dedicated memory tool.

If that's not overwhelming enough, we now also have something called a shell tool.

LangChain calls it shell tool. Anthropic calls it the bash tool. If you've experienced if you've played around with OpenClaw, it's called the exec tool. But what all of these tools do is they let your agent run commands in the terminal.

And that actually makes them super versatile because now you can let your agent use CLIs to navigate your and explore your local files. So you can just run ls and grep to find data in your local file system.

If your database has a custom CLI, you can actually let the agent also use the shell tool and interact with the database. You could also let your agent write an entire search entire script from scratch, like connect to your database, run a search query.

If your database is exposed via HTTPS, you can just run a curl command, interact with the database through the shell tool. Speaking of curl commands, you can also do web searches if you like. So this shell tool is super versatile,right?

So the question and the topic of today is, what search tool do we actually need? Do we only need a shell tool? Do I need all of these? And if you take home only one thing from today is that doing good search is incredibly difficult.

And that's why we have many different techniques to do search,right? We have vector search. We have keyword search. Even in vector search, we have dense embeddings, sparse embeddings, multivector embeddings. Then we have many different indexing techniques. So depending on what kind of search requirements and latency requirements you have, you will need to curate your own stack of search tools,right?

So today we're going to be looking at a few of these. Unfortunately, we only have one hour, so I cannot show you all of them.

Before we get into some code, I want to

give you a few fundamentals of building good search tools because

agentic search at the surface level seems very straightforward. The user makes a request. The agent calls theright tool with theright parameters. I see someone laughing. The retrieval tool gives you the tool response, and then your agent responds to you with the correct answer.

At Elastic, we help a lot of internal and external teams build agents based to interact with Elastic search data. And the reality is that this can break in many different ways. I'm just going to show you three today.

So the first is the agent doesn't call any tool. So this means the agent decides, I actually can answer this question based on my parametric knowledge. I don't need to use any context retrieval tool. And the other problem is that the agent calls the wrong tool.

I was recently talking to a colleague of mine. I was asking him, what was your the most challenging aspect of your project? And he was like, you won't believe it, but it was really difficult to get the agent to actually not call the web search tool, but call the database search tool,right?

And then depending on how complex your parameters are for your search tool, it can also be quite challenging to get your agent to generate theright search parameters,right? There's many more failure cases, but we're going to limit this to these three today.

So I personally hate this slide because I feel like everyone in this room probably knows that the tool description is the most important aspect. But anytime I see a tool description, it's like the least effort, one sentence, and then you're wondering why your agent isn't calling theright tool.

So arguably, this is a very long tool description. I'm not saying you have to write it like this. I'm just saying, if you just start with a core purpose, if it works fine, great. But if you add more parameters or more tools and your agent is starting to struggle with calling theright tool, then maybe add some trigger condition.

When should this tool be used? When should this tool not be used? Especially if you have multiple tools, adding something like relationships is super important, like first call this agent skill before you actually call this tool or get some confirmation before you call this tool.

If you have the perfect tool description and your agent still doesn't call theright tool, then reinforce it in the agent system prompt. That should actually help out in most cases. Then I want to quickly touch on some on the parameter complexity.

If you have a search tool that's just very simple in the sense that something like get customer by ID, it should be fairly straightforward for the agent to generate an ID parameter, given that it's a valid ID.

Same for if you're doing a semantic search,right? Generating some valid string should be shouldn't cause any issues. But let's say if you want to have a semantic search tool, and instead of just giving a topic, you now also want to give it some filter conditions.

Maybe you want to define the top K. Then you start to have more parameters,right? This isn't very complex here, but the longer the list of parameters you have, the more difficult it's going to get for the agent to generate theright ones.

And I think a very complex one for an agent is to when you have something that's more general purpose, like letting the agent execute entire search queries against a database. So here I have ESQL, which is the Elastic search query language.

Could be SQL as well. So letting the agent write an entire SQL query from scratch can be quite challenging. Most are pretty good, but some aren't. So just keep in mind, let's say the complexity of the parameter also is kind of a failure mode, and you can you might need to help the agent out with a few of these if they are more complex.

Good. Let's look at some code.

OK.

So

quick show of hands, who here has built some sort of agentic RAG, agentic search before? OK, about half. That's good.

So we're going to be looking at three things. I'm going to give you a quick recap or intro to the very vanilla agentic search demo. And then I'm going to show you how easy it is to break this usual demo.

And then we're replacing the semantic search tool with something more general purpose. So we're letting the agent write an entire search query from scratch. And for these two examples, we will be using a local Elastic search cluster as a context source.

And then for the third part, I'm switching gears, and I'm going to be showing you how search over a local file system works with the bash tool. And then I'm also going to show the shell tool. And then I'm also going to show you some limitations of the shell tool and how you can expand it with custom CLIs.

Allright. The example that we'll be doing today is I have the conference session data of this conference here. And let me start show you this one. So just a quick recap. We have an Elastic search database, and we're going to be writing our semantic search

tool. And for in the database, I have the conference session already chunked. You probably already know how to chunk and store data in a database. So we're skipping this part. This is not the important aspect.

So what do we need for an agent, an LLM? Oh, sorry. We're going to be using LangChain for this session just because it wraps a lot of the complexity. And I don't like it lets us concentrate on the high-level concepts.

Also, it has some nice built-in features, like the shell tool is built in, and it has some code samples for skill loading tools, which we will be looking at later. OK, switching back. I'm using GPT 5.4 nano for this demo.

Then we're defining a very simple system prompt. So the usual you are a search agent tasked with answering questions.

You have access to different context retrieval tools. And before answering a question, decide whether or not you need to retrieve additional context. To help the agent a little bit, I have some information about how the data is structured in Elastic search.

So here we have a text field. It's comprised of the title of each session and the description of each session. And the text field is what actually gets embedded as the vector embeddings for semantic search. And then I also have some metadata fields.

So for example, the day, the time, the room, the speaker's name. So since the metadata is not embedded, I can only run filters over them, but not any semantic search, just for your information. OK, now the interesting part.

Let's build a semantic search tool. So how that works in LangChain is I have to first define an embedding model. In this case, I'm using the new Gina Embeddings v5 model. This is used to embed the search queries at query time.

And the embedding model I'm putting in with putting it into my Elastic search store together with the Elastic data to create a vector store. And then

I can create a search tool. So in this case, I have all I have to do is I call the similarity search method. It takes in a search query. And in this case, I'm setting the limit or the top K to three.

This is a little bit of foreshadowing because I'm limiting the capabilities of this tool to just returning three search results,right?

What's nice in LangChain as well is that when you use the tool decorator up here, it lets you convert any Python function into a search tool, sorry, into an agent tool. So by default, it takes the

function's Python function's name as the tool name. And the doc string down here is going to

get converted into the tool description. You can see I'm breaking my own rule by having a very short tool description here. Why this works is because I only have one search tool here,right? So you will see I'm adding a few things later on, but it's not going to get very descriptive in this demo here.

So now we can run a test and test it for a search query of regulatory constraints. And you can see it finds a talk by my friend Bilge on engineering AI systems under sovereignty constraints. And it also finds some more talks, one by Tejas and one by Pedro.

Allright, let's plug it in. So we're plugging in the LLM, the system prompt, and the search tool. I'm leaving out memory. Obviously, this would be another core component of an agent. In this case, I'm leaving it out to keep it kind of concise.

Now I can run a simple question like, which sessions discuss regulatory constraints in AI systems? And you can see the agent first calls my semantic search tool. It wrote a quite extensive search query, in my opinion, but it works.

It finds theright talk by Bilge. Then it decided that that apparently wasn't enough. So it rewrote the search query, but decided but got the very similar search results back. So after that, it decided that, sorry, it decided that it's now able to respond with theright talks.

This is where most agentic search demos fail. But this is very brittle. Does anyone have an idea how we can break this?

Is there something that's not in the database?

Yes, that's a good idea. Anything else?

Asking if something that's not in the database would be something great. What about asking it

something where semantic search actually falls short? Maybe something where we want to look for a keyword, a specific keyword.

Also doing something like filtering, because in this search tool, we don't have any filters implemented,right? So

my choice of search query is, which sessions should I visit to learn more about GEPA? I'm not even sure. Like I've heard people talk about GEPA. I'm not even sure if I'm pronouncing it correctly. Sorry. That's why I need to definitely attend this session.

So what you can see is the agent now calls the semantic search tool. And this time, it's looking for GEPA. So far, so good. But now you can see it's actually returning a talk for DeepMind's Gemma models. I guess from a tokenization perspective, it could be similar to GEPA or JEPA.

I don't know. Then it returns something on harness engineering. Not sure if that's necessarily related. And then a third one. Clearly, none of these are related to GEPA.

Spoiler alert, I know there's a talk about GEPA or JEPA. Again, I think it'sright after this one. So we can see the search tool we just created, it's not very useful, or at least useful only for a very narrow scope of use cases,right?

What if we let the agent now write an entire search query from scratch? Let me show you how we can do this. So we're now replacing the database tool that we had with an execute query tool. So we're letting the agent not only take in like a topic, but this time we're giving it an entire the search tool and an entire search query.

And I'm going to show you, because this is quite difficult for an agent, we're also combining it with a skill loading tool.

So

doing the same thing. I'm setting up my LLM. You've probably noticed I'm switching to a little bit more powerful model here. So I'm switching from the GPT 5.4 nano to the mini, because I am now anticipating that writing search queries is a little bit more difficult.

So the nano is probably not powerful enough.

I'm using the exact same system prompt as before.

And now I'm creating a general purpose database query tool.

Since I'm using Elastic search, I'm going to be using the Elastic search query language, which is a Python query language for filtering, transforming, and analyzing data. It looks something like this. Maybe it reminds you of SQL. It's a little bit different.

It has different capabilities. Not important for this session. But you can see when I connect to my client and then I use this query method from the ESQL class and run this query, you can actually see that there is a session by Samuel, which talks a lot about GEPA.

You can see here is a match. Here is another match. So let's wrap this into a search tool.

You can see this time, I just used the query method again here. And the agent takes in the ESQL query as a parameter.

And I exchanged the tool description with something that's called execute an ESQL query against the conference schedule index in Elastic search. Notice anything different about how I wrote this search tool versus the other one?

This time, I added a try except block here for error handling. Generally speaking, you should have error handling. But since I'm anticipating that writing a good ESQL query or a valid one is going to cause more problems for the tool, I don't want the agent to just fail and then the whole system to crash.

So instead of instead, I return the error response to the agent so it can kind of self-correct, rewrite the query.

Generally speaking, super important to have this,right? So the agent can self-correct. So we can test this here. This is not important. Then I'm plugging in the LLM, the system prompt, and my new search tool into the agent again.

And when I now ask it the exact same question as before, which session should I visit to learn more about GEPA? You can see it calls the execute ESQL query tool. And it generates something that looks like valid ESQL.

I'm not expecting anyone to be very familiar with ESQL. What's wrong with this is that ESQL doesn't use the percentage sign as a wildcard character. In ESQL, you would use the asterisk. So in this case, it's actually looking for percentage sign GEPA percentage sign in the data as an exact match.

So that's why it's actually returning zero search results. And this is when you're working with search tools, also super important to think about is returning zero search results actually a valid response or is it a failure mode,right?

OK, how could I overcome this? I could probably write

a more descriptive tool description, give it a little bit more help on how to write better parameters. I could reinforce it in the system prompt, give it more instructions there. Or I could use an agent skill, because ESQL, you need more documentation than just like a one-liner,right?

So now I'm going to show you how to add an agent skill. So in this case, I'm going to be writing my own very short custom agent skill. Quick question. Who has used and played with agent skills before?

OK, good or not. So

I'm going to be writing a very short one here.

There is official Elastic search agent skills available if you want to play around with it. In this case, I'm just using my own custom ones. So how that works is you have

the skill name and then also a skill description, which gets injected into the system prompt. So only the if you write it in Markdown, it's I think the front meta,right? That gets injected into the system prompt. And then when you need it, more information on the agent skill is loaded into the context window,right?

So it's called something like progressive disclosure, where you kind of add more information about the skill as needed. So in this case, I have some minimal instructions. Like here's the basic structure of an ESQL query. ESQL uses double quotes for string literals, just some very basic syntax rules.

And I also added some more information about the wildcard pattern so it's not making this mistake again.

And then, as I mentioned, LangChain has some boilerplate code you can just copy and reuse for using agent skills. So I'm skipping over this. All you have to know is we have a tool, a skill loading tool, and it's getting inject sorry, it gets combined with something called a skill middleware.

Skipping over this because this is not relevant for our session. And now

all I have to do is edit the tool description of my general purpose search tool. So this is the exact same tool that I had before, except this time, I'm now adding some relationship. I'm saying always use the Elastic search ESQL skill to generate the ESQL query before using this tool, because otherwise, if the agent then still uses this tool first without using the agent skill, then that would be a shame,right?

I'm doing the exact same. So I'm reinforcing this now in the system prompt. I'm saying the same thing to use the Elastic search agent skill first before calling the general purpose search tool. And now I'm plugging it into the agent again.

So this time, LLM system prompt for the skill loading tool. I have the skill middleware and then my general purpose ESQL query tool. And now when I

ask the agent, which session should I visit to learn more about GEPA? You can see it first loads the skill. So it actually loads everything that's kind of in the body part of the skill into my context window.

And then it generates this time a very valid ESQL with the asterisk as my wildcard characters. And you can see it actually finds theright session. And now it tells me that at 10:40, so after this session, I should be going to this session to learn more about GEPA and learn how to pronounce it correctly.

OK,

what's also cool about this is now the agent can do a lot of things,right? It can also do aggregations. So if I ask it something like how many sessions are on April 8, you can see again, it loads the Elastic search ESQL tool.

And then it writes an ESQL query. Whoops. It writes an ESQL query that's using a filter. So it's filtering for April 8. And then it also does an aggregation, some counting, and tells me today there are 27 sessions.

This is nice because if I just do a search, let's say I ask it to tell me which sessions are on April 8, and it only runs a filtered search,right? So just imagine it would give me a list of all 27 sessions that are today, and we let the agent count how many sessions there are.

That would probably not be so good, because we all know agents or LLMs are notoriously bad at counting things. And also, it would fill up your context window,right? So by letting the agent do its own calculation, so letting like outsourcing the calculation part into the search tool is actually quite an efficient way to do this,right?

Any questions so far?

OK,

let's switch gears.

This is a very prominent topic at the moment. Maybe you've heard the discussion about all an agent needs is a shell tool and a file system. So I work at Elastic, but I don't discriminate. Let's look at file systems and how to do this, because I think it's a very interesting topic in general.

So what I did here, I prepared the data this time in a local file system. So I have a folder called session data. And in here, I have for each type of session, like keynotes and workshops, I have another folder.

And in there, there is per session one file. It looks something like this. So with the title, some metadata, and the description.

And now I'm going to show you how you can use the shell tool with this.

So I'm switching back to the GPT 5.4 nano, because LLMs are just generally good at navigating file systems, writing shell commands. So GPT 5.4 nano is sufficient in this case.

I define another system prompt. So the first part of the system prompt is exactly the same as the one we had before. And what I'm replacing this time is instead of explaining how the data is structured in Elastic search, I'm explaining how the data is structured in my local file system.

OK, so let's use the shell tool. I have to give you a disclaimer. Using the shell tool can be risky, since giving your agent access to a terminal

can make it delete files or do other things you don't want it to do. So always recommend it to use it in a sandbox environment. Also, in LangChain, it doesn't have any safeguards by default. So please be careful when using this.

But other than that, it's very easy to use. So you can just import the shell tool and instantiate it here. And here, you can see how you would use it. So it takes in the commands parameters. So when I say echo hello world, you can see down here, it actually prints hello world into my terminal.

And then all you have to do, again, plug in the LLM, the system prompt, and the shell tool. And now you can ask it this exact same thing we had earlier. So are there any sessions about GEPA? And you can see here, it's called the terminal here, but the agent calls the shell tool.

And it actually writes a few commands. So first, it's looking at the folder structure. And then it runs some GRAP commands. So it's looking for GEPA in the session data. And I think it's looking for the first 50 entries.

And so you can see it saw the folder structure. And then it also found the one session we were talking about earlier. But since it was only looking at the first 50 and only found one session, it decided that it should probably look at the entire session data.

So it finds the exact same session again. So this time, it decides, OK, then I should probably look at the contents of this session. So now it reads the entire file content. And here, you can see

the session information as a tool response. And then at the end, the agent tells me which session I should visit.

OK,

GRAP works based on exact matches,right? And regex. And I just want to show you this, because I think it's funny how surprisingly good agents are with Bash, because they kind of can cheat at semantic search. Let me show you this.

So when I ask it which sessions discuss handling regulatory constraints, this was our semantic search query from the beginning. You can see it again looks at the folder structure. And then the first command or the first search it does, it's looking for regulate.

It's fair. It's looking for regulation, for regulatory. I guess that's a fair start. But then it goes ahead. And now it just chains a bunch of synonyms together. So it's looking for compliance. It's looking for constraints. It's looking for GDPR.

It's looking for governance. Yeah, I guess that's fair.

I think it actually finds a bunch of sessions. So it's like, OK, let me try a bunch of other synonyms. So now it's looking again for regulate, compliance, GDPR, serenity. I think, yeah, the list goes on. It's just looking at a bunch of different synonyms.

And it actually is successful with this. And it finds the session by building and returns the session information and then is able to respond correctly. I guess it works. Is that the most efficient way to do this? Probably not.

I mean, just as an example. So let's say you want to search for something like movies with animal superheroes or something. Do you really want to do your agent to search for a list of all the animals possible when you find all the superhero movies with animal superheroes?

Probably not. So it works. Is it the best? I let you decide. So at the moment, there's many different semantic search alternatives to GRAP. I think there's one by LlamaIndex called SemTools. There's a really cool one by LightOn, which is called CoGrap based on multivector embeddings.

Also, there's one by our own Gina. It's called Gina Grap. So today, I'm going to show you how easy it is to actually use this together with your agent. So all you have to do is go ahead and install the Gina Grap CLI.

And then

all you have to do is tell your agent that it now has access to this tool or to this CLI. So this is the exact same system prompt as I had earlier, now with the difference that I'm explaining to it that it has Gina Grap and how Gina Grap works, how it should use it.

Here are some examples of how you would use Gina Grap. Just a disclaimer. Gina Grap has many different modes. You can use it for classification. You can use it for re-ranking. Today, I'm just showing you how to use it for semantic search.

And at the end, I'm also explaining to the agent when it should use GRAP and when it should use Gina Grap, just so it knows for exact matches, you probably still want to use GRAP. And for more semantic search or fuzzy queries, use Gina Grap.

And then plugging this into my agent again.

And when I now run the exact same semantic search query we had earlier, so

which sessions discuss handling regulatory constraints, you can see the same behavior. So it calls the terminal tool. It first explores the folder structure. And then it actually, on the first try, is able to correctly use Gina Grap. So it's looking for regulatory constraints.

And boom, it actually finds the session by Bill Gates on the first try.

It finds a few others because it's looking for 10. It says returned top K of 10. And then it's able to answer me correctly. Nice. Allright, any questions so far?

Good. Then I'm switching back.

So we were looking at a bunch of different tools today. We saw how big the tool landscape is. I showed you a few of the search tools we have. Now, some practical recommendations on when should you actually use what.

So

maybe let's start with this. If you're looking for just one silver bullet tool, that's probably not theright way to go. Again, if you think about it, doing good search is incredibly difficult. So ideally, you want to have or curate theright set of search tools for your agent's search behaviors.

And you want to have a combination of specialized tools and a combination and general-purpose tools. So specialized tools are something that the agent can use out of the box, something with a very simple parameter, something where you don't need a very powerful LLM.

You know the agent isn't going to make a lot of mistakes. The agent can just use this tool out of the box. So at Elastic, we like to think about this about having a low floor. So this is a concept from user experience where the agent can just

use a tool that doesn't make many mistakes. It's also efficient. So it doesn't have to run your tool multiple times. You can think about this as the semantic search tool we had earlier. Maybe you need to look up customers by ID a lot of the time.

Then having a specialized tool for that exact operation would be helpful. But then you also want to give the agent a high ceiling. That means for unexpected queries, for complex questions, you want the agent to still be able to handle these questions,right?

And not have these limited specialized tools and be like, I cannot solve this. So for this case, something like a shell tool or the very general-purpose one we had earlier of a query execution tool would be very helpful.

But the problem with the query execution tool or the shell tool you saw earlier is, since it's so general-purpose, the agent sometimes might need more iterations to actually get to theright answer,right?

So this is my practical recommendation of having a balanced set of search tools

of a low floor and high ceiling.

This is all nice when you already know your agent's behavior. But if you don't know your agent's query behavior yet, then I would recommend to start with a general-purpose tool.

Then log your agent's behavior. Generally speaking, logging your agent's behavior is recommended. But if you notice, maybe your agent is taking four or five tool calls per question. That's too many tool calls. That's probably an indicator that the tool your agent has is too difficult for it to use.

Then definitely look at what the agent's actually trying to solve. Maybe scope out something more specialized in that case.

Also, if you notice specific query behaviors, this is what I personally did with my test OpenClaw. It has the exact tool. And I started logging its behavior. And obviously, I was playing around with databases. So after three days, I was asking it, what kind of interesting patterns do you see?

And it was recommending me actually to implement some specific search tools to interact with the database because it was out of the box only using the exact tool. Allright, start with general-purpose tools if you don't know your user's behavior yet.

Log what breaks and add purpose-built interfaces.

Yeah, that was a lot to take in.

I'm sure you have lots of questions. So I'm opening it up for Q&A. And otherwise, on your way out, don't forget to grab yourself some stickers. And then thank you for joining my session.

I think there's a mic coming.

Thank you. So would you say the tool stack you need is also mostly dependent on the model you're willing to use? So if you're using a very good model, it might be fine using the shell tool or search tool.

But a very small model, a light agent, might need more specialized tools?

Yeah, actually, I think in our internal testing, we noticed that a more powerful tool actually reduces the error rate for the parameters. I don't know the numbers exactly, but it was a very big amount where it reduced the error rate.

So having a stronger model definitely helps

for the general-purpose tools. But I think you cannot expect just because you have a very strong model that there's going to be no errors. Does that make sense? Yeah.

Thanks for a nice talk. I have one question. Maybe it's slightly off topic. So now we are talking about Agentic RAG. But it comes with the drawback of having higher latency against typical RAG. So would you recommend having a second pathway for simple RAG for fast answers?

And how would you guide the agent to actually choose theright one? Because I think it's hard to say which question should be answered by Agentic RAG or by simple RAG.

That's a good question.

I don't think I have a good answer on the top of my headright now. I was thinking maybe something related I was asked recently. If you have a RAG system, should you replace it with Agentic RAG? And I guess this is kind of going in the same direction of when do you actually need Agentic RAG,right?

Probably for a lot of use cases. I know RAG has been killed many times. But I think the reality is that RAG is still very effective for many use cases.

How would you actually switch between RAG and Agentic RAG? I'm not sure because I assume it's, again, it needs some kind of almost agentic logic of switching between them. So

I'm not sure. I'm sorry.

In such cases, like when you have the wild card in the GAPA example, why can't we just

perform a hybrid tool that maybe searches and replaces common wrong wild card symbols coming from SQL with the correct ones, for instance?

I'm not sure if I understood your question correctly.

I mean,

there are cases in which the agent doesn't know how to write the correct query because maybe he thinks the placeholders coming from SQL apply to ESQL.

So why don't we perform a hybrid tool that

determine cleanly

searches and replaces the wrong placeholder, the percentage symbol with the asterisk?

Yeah. Actually, so the example I showed you of using the agent skill wasn't necessarily the necessary solution for it. You can also add just some very simple instructions on for ESQL, don't use the percentage sign as a wild card character.

It actually works. I tried it when I was building the demo. But then when the agent now runs in the next issue, then you start adding the next piece of documentation. Then you can kind of start writing the entire ESQL documentation from scratch into your system prompt.

And yes, for the demo purposes, it would have worked. But it's probably not how you would do it necessarily when you're building something more robust,right? Because if you just add little Band-Aids every time you run into an error, then what happens when you run into the next edge case?

Does that make sense? Yeah.

Hi. Thank you for a really wonderful presentation. I have a question. In the demo, we have walked through the Agentic Search with DBQRIS tool and also another one with shell tool. Would you recommend in the practical use, we can also kind of

combine both tools and then we validate the result from each of the tools. And then we kind of add the confidence of the result from the LLM. Would you recommend doing this in a practical use?

Yes, yes. That's a great question. Also, again, I'm kind of cheating in this demo,right? Because I'm only showing you one tool per demo. In reality, you would have something more like a bunch of different tools where you then have to decide which or the agent has to decide which tool to use.

I think there was a very interesting blog post by Vercel, I believe. And they did an experiment. I think it's called, if you want to look it up, I think it's called "Testing if Bash is All You Need" is the title, I think, of the blog post.

And they actually kind of benchmarked or tested an agent with a Bash tool, an agent with just file search tools, I believe, and an agent with database tools. And in the end, they also had one agent with a Bash tool and the database tool.

And they noticed that was super interesting for a specific set of queries

where you have analytical queries. This is a specific use case. Actually, the database tool was more effective. But on the other hand, the file search tool was very effective, as you saw, for just quickly finding things. But the very interesting aspect was the hybrid agent with the Bash tool and the database tool was actually achieving the

highest accuracy because at first, I believe it was first using the database tool and then verifying the results with the shell tool. And that let the agent to actually achieve better accuracy. So I think that was a very interesting way and behavior to see in agents.

Thanks for sharing.

One second question. So if we use the semantic search tool, I think in practice, you probably would use some kind of threshold to cut the results to not get something if there's no answer. But in the agentic regime, would you then say, OK, let's put a conservative threshold such that we don't confuse our agent?

Or would you say the agent is smart enough, even if we retrieve results that are not really relevant, it will be good enough to notice that?

Yeah. That's a great question, actually. So in the examples, you probably saw some where the agent was returning. I think in the last Gina Grap example, you see it's actually returning the top K results where only the first one is the actual relevant one.

And I think because the agent does a little bit of reasoning over whether the search results are relevant to the search query, I think it's much better or they're much better today at kind of weeding out what's not relevant.

But then you kind of run into the risk if you have longer running conversations that kind of these search results sit in your context window. Long-term, you could have the problem of confusing your agent long-term. So I think it kind of

depends on your use case of

how your agent can handle irrelevant search results. But generally speaking, based on search results, it can filter out what's irrelevant.

Thanks.

Thank you for the talk. Amazing. Are you utilizing sub-agents for these search queries? Because if we let them decide to, is it relevant to use a question like it's done in RAG's framework, for example, for evaluation? I mean, sub-agents would help a lot.

Do you have any experience with them?

Unfortunately, not. I have not played around with sub-agents yet. I can only tell you that I know, for example, I believe in Claude Code, they're using sub-agents for doing specific search tasks. I think there was a blog post on how they're actually using a sub-agent to answer specific questions about Claude Code because it's kind of like a niche question a user would ask.

So in this case, they kind of outsourced their expertise to a sub-agent. So having a sub-agent for specific niche questions, I think, would be interesting. But I don't have too much experience.

OK, thanks. Can I ask another question?

Sure.

Damn, I forgot. Sorry. I'll try to catch up later.

Yeah. Kindly off topic. But you talked about skills. And the benefit of skills is just have the description in the system prompt. And whenever needed, we need to load this full skill. Do you have any recommendation when and how to clear the system prompt again?

Because we want to keep the context window small. And maybe for a long session, we might have up to 10 skills, full skills in the context.

I'm not sure. Joe, do you have a better answer? Have you an idea?

What we're doingright at the scene is that we load the skills, like definitions, within the prompt. But then we use the file store to offload things within the context. So when the context is building up, when you're loading 10 or 11 or 12 skills, we basically refer to we replace the context messages, the referral where it is in the file store.

So we keep the context very small.

Sorry.

I finished that.

But yeah, so the way that we are doing it behind the scenes is that we're providing that kind of progressive disclosure of skills. So we're providing the skill names and descriptions, the location within the file store. And then from the file store, we're loading into the context window when we need that skill.

And then we offload it once the context window progresses ahead of time around that. So we have this kind of more on-demand one around that. And that's the same with our compaction of context. And that's what I would advise you to do, like some of the questions you do.

Try and use the file store as much as you can. And have those tools as well, like being able to grep the file store for when you want to see previous tool results. And then use it from that.

Thanks. This is my colleague, Joe, from Elastic as well.

Awesome. If there are no more questions, I will let you guys go into the coffee break. Again, don't forget to grab yourself some stickers. And happy to catch up in the halls if anyone's interested. Thanks so much.