# Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS AI Engineer · 2026-07-11 Elizabeth Fuentes (AWS) presents five code-based techniques to stop AI agent hallucinations, each with measurable before/after metrics. Semantic Tool Selection filters 29 tools to the 3 most relevant per query, cutting token usage from thousands to under 300 per call. Graph-RAG replaces vector similarity with structured graph queries (using Neo4j), enabling precise aggregation and multi-hop reasoning that vanilla RAG fabricates. Multi-Agent Validation uses an Executor-Validator-Critic swarm to catch fabrications, achieving a 92% detection rate. Neurosymbolic Guardrails enforce business rules in Python hooks that the agent cannot skip, achieving zero rule violations. Agent Steering guides agents to self-correct when soft rules fire, completing tasks without hard failures—demonstrated by booking 50 guests by intelligently splitting into two rooms. ## Questions this episode answers ### How can I reduce the token usage of my AI agent when it has many tools? By using Semantic Tool Selection, you create a vector database of tool descriptions and retrieve only the top K most relevant tools for each query. In Elizabeth Fuentes’ demo, a travel agent with 29 tools consumed about 3,000 tokens per call just for tool schemas; after filtering to the top 3 tools, token usage dropped to fewer than 300, improving efficiency and accuracy. [3:32](https://aie.addtry.com/4988d405-239f-4665-abba-5e20730a3a95?t=212000) ### How can I prevent my AI agent from guessing when answering aggregation queries? Elizabeth Fuentes demonstrates Graph-RAG using Neo4j, where instead of retrieving text chunks via vector search, the agent writes a Cypher query against a knowledge graph. This returns a computed, verifiable result across all data, not a limited sample. In contrast, standard RAG returns only top-K chunks, causing the model to fabricate statistics, such as estimating average ratings from incomplete data. [18:11](https://aie.addtry.com/4988d405-239f-4665-abba-5e20730a3a95?t=1091000) ### How can I make my AI agent self-correct instead of blocking when a rule is violated? Instead of using hooks that unconditionally block tool execution, Elizabeth Fuentes shows how to steer the agent using an open-source library called agent control. The steering rules are registered on a server, and the agent can adjust its behavior—like splitting a large booking into two smaller rooms—to complete the task without a hard failure, avoiding user retries. [45:33](https://aie.addtry.com/4988d405-239f-4665-abba-5e20730a3a95?t=2733000) ## Key moments - **[0:00] Intro** - [0:31] Every time your AI agent responds, you're paying for the words going in and the words coming out. - [1:05] All 5 techniques to stop hallucination are code changes, not prompt changes. - **[1:09] Semantic Tool Selection** - [5:08] 29 tool descriptions consume 3,000 tokens per call before the model even responds. - [5:38] Semantic tool selection filters 29 tools to 3 most relevant, reducing token usage from thousands to under 300. - [15:47] Swapping tools in conversation keeps only needed tools, but chat history still increases token usage over time. - **[18:11] Graph-RAG** - [18:34] Vector search returns chunks but cannot aggregate or traverse relationships across all data, causing RAG to estimate. - [19:22] Graph-RAG runs a Cypher query across all data, returning a computed result instead of a sampled guess. - [24:27] Graph-RAG query for average hotel rating returns exact 47, while RAG estimates from a sample of 2 hotels. - [25:59] Multi-hop reasoning query for room types and price of highest-rated hotel: RAG returns vague info, Graph-RAG returns specific details. - [27:00] When asked about hotels in Antarctica, Graph-RAG correctly states zero results, while RAG returns a verbose hallucinated discussion. - **[29:14] Multi-Agent Validation** - [29:35] Single agents can silently fabricate success after tool errors, hiding failures from users. - [30:25] Multi-agent validation with Executor, Validator, and Critic catches fabricated responses before reaching users. - [35:08] Multi-agent SWAM: Executor error, Validator catches, Critic rejects, preventing user from seeing fabricated response. - **[36:06] Neurosymbolic Guardians** - [36:27] Prompts are suggestions, not constraints: the model can still call a tool with 15 guests despite a prompt rule limiting to 10. - [37:14] Neurosymbolic Guardians use lifecycle hooks to enforce rules in code before tool execution, making them unbypassable. - [42:42] Neurosymbolic Guardian blocks hotel booking when payment is not verified, unlike the prompt-only agent. - [43:13] Hook enforces maximum 10 guests per booking, blocking reservation for 15 guests that the prompt-only agent allowed. - [44:34] Same model, tools, and prompt yield different outcomes because rules enforced in Python, not just in prompt. - **[45:33] Runtime Steering** - [45:33] Runtime steering guides agents to self-correct instead of blocking them, allowing task completion. - [46:50] Agent Control SDK registers steering rules on a local server via API, enabling changes without touching agent code. - [50:58] Runtime steering automatically splits a booking for 50 guests into two rooms, completing the task without a hard block. - **[51:29] Production Deployment** - [51:59] In production, steering rules stored in DynamoDB are live on the next call without redeployment. - **[53:46] Recap** - [54:52] All five demo notebooks and applications are available in the open-source repository. ## Speakers - **Elizabeth Fuentes** (guest) ## Topics GraphRAG, Guardrails, Multi-Agent Orchestration ## Mentioned AWS (company), Neo4j (company), OpenAI (company), Amazon AgentCore Gateway (product), Amazon AgentCore policies (product), Amazon Bedrock AgentCore (product), Amazon Vector (product), CDK (product), Claude (product), CloudWatch (product), DynamoDB (product), FAISS (product), Lambda (product), Neo4j AuraDB (product), agent control (product), sentence transformer (product), trance agent (product) ## Transcript ### Intro **Elizabeth Fuentes** [0:01] Hi. Today we're going to talk about how to stop AI agents' hallucinations with 5 techniques beyond the prompt. Each one is a code change, not a prompt change. Let's see. Every time your AI agent responds, you're paying for the words going in and the words coming out, and in your build you'll see those cooling tokens. Yeah. And the more tokens you're sending in, the more you pay. And if what you send is not quiteright, too much, or missing something important, your agent starts to hallucinate. There are 5 techniques to help reduce token waste, improve accuracy, and catch failure before users see them, and each one is a code change, not a prompt change. So let's see it. First, we have Semantic Tool Selection. You filter which tool goes into context on every call. The model only sees what it needs for that specific query. Second, we have Graph-RAG for precise queries like aggregation, cons, multi-hop reasoning, and you replace the text retrieval with a structured graph query. ### Semantic Tool Selection **Elizabeth Fuentes** [1:39] The model gets a compute of verifiable answers, not a sample, as RAG do. Third, Multi-Agent Validation. A second agent can check every response before it reaches the users. And fourth, Neurosymbolic Guardians. You rule life in Python, not in the prompt, and the model cannot skip them. So five, Runtime Guardians, because if you don't want to block, you can steer. You don't need to block everything. And when a rule fires, the agent self-corrects and completes the task. No hard stop, no user retries. So for each technique, I will show you the agent without it and then with it so you can compare. And all the demos I'm using a troubled agent that I build using a trance agent. And a trance agent is an open-source agent framework that we maintain on AWS. And I am Elizabeth Fuentes-Leone. I am a developer advocate for AWS. I'm focused on agentics application, and here in this QR code you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment. So let's get into let's get into it. So Semantic Tool Selection, or travel agent, has 29 tools: flights, hotels, payments, weather cancellations, all are dummies tools. I know, like a travel agent for real. But every time that user sends a message, all the 29 tools' descriptions go into the context windows. The model treats all of them before deciding what to do. And if your agent has memory, the rows to every conversation add more context that gets sent with every single message. And you pay for every single one of those tokens, whether the model ends up using that tool or not. To understand where those tokens come from, you need to see what a tool actually looks like to the model. In a trance, you write a function with the tool decorator, that's the tool, and a name, a description, and a docstring-type parameters. Then a trance takes that and generates a schema with name, description, parameters, and the schema is what goes into the context windows on every call. Each tool schema is about 17 or 200 tokens, depending on how many parameters it has. If our travel agent has 29 tools, that adds up to somewhere around 3,000 tokens per call just for the tool description. Before you message it, before the response, every single call. By creating a tool database, we can filter the tools that the agent may need before the agent is invoked. With this filter, the model sees only 3 most relevant tools. Tokens usage drops from thousands to fewer than 300. Let me show you in the code. Here in my QID, this is the let me clear all the output. So first, we need to install the requirements. Here is the requirement file. This is a Jupyter Notebook because it's more simple to show everything, but it is an application that you can run if it is more comfortable for you. So here in the requirements, I have the trance agent. And because I'm using OpenAI as a model invocation, I'm using the API from OpenAI, and I need a trance agent from OpenAI. You can use OpenAI trance agent with, oh my god, almost all the model providers. And with a trance with Amazon Vector, of course, because we as AWS, we are the maintainer of this framework. And to use Amazon Vector, you don't need to add the model provider. And because I need to embedding, I need to create embeddings for my vector tools database. I'm using the sentence transformer. This is a super simple model that runs locally, so it's free. So you can run this locally, and you can prepare it without spending another model embedding. And you can use a trance with Ollama too. So if you have a local model, you can run everything in this new computer for free without spending any tokens. And I'm using FAISS as my vector store, local, super simple. And, well,right now I'm going to use Neo4j. This is for another demo that I'm going to show you in a moment. And because I'm using some environments, I'm using Python stuff. And let's see. So here I'm going to run this. Let me put it bigger. I know that you're having some problems. Put it bigger. A little more. A little less. And here, this is better. So I installed my requirement. I already did that. And I'm using OpenAI, so I need my API key, API key. And here I invoke my trance. I need an agent because I'm going to build an agent. I'm using OpenAI, and I have a bunch of tools, dummy tools here. I'm going to show you in a moment. And I'm going to use this because I have some functions that I'm using to create the build index for my vector store, and I need the search tools. When I'm using the vector store, I put my query there. I search for the I search for the tool that I'm going to use using vector search for vectors, and I swap the tools. I'm going to show you that in a moment. So let me show you all my dummies tools. Where is the engage? Register here. Here I have all my tools. This is for swap, search, build index, and where are my tools. Here. So here are all the dummy tools I created for this. This is a demo, please. This is not something that you can use to go to production. No, please. And, okay. So where is this one? This one. So I run this. And I build my semantic index. I already did that. Yeah. So I have 29 tools here, 29 dummy tools. And I'm going to test this with a lot of different queries. And this is the ground truth. I have a ground truth because I know what is the best, the best tool to answer the question so we can know if our agent is okay or not okay. So let's run this. We have 90 queries and 29 tools. So this is some helper functions. I don't care. Yeah, I care, but I'm not going to explain you that. So this is my agent, my traditional agent, where I'm putting all my 29 tools inside the agent. I have my model, some helper function. This is my agent. I only this is the way I create an agent using a trance agent. I put all my tools. And my system prompt is, "You are a travel assistant. Use this correct tool to answer questions." Oh my god. Super. And the model. Let's run this. And this is whatever. This is some function to count the tokens because inside the trance agent, you can count the tokens. You can know how many tokens are used in and out of the agent. This is some helper function. And, yeah, this is every time that I put a prompt inside this agent, I'm only sending the prompt, and the agent responds. Then I use a new agent because this is a for, so I'm not having a conversation with the agent. I only said a question, and I received an answer. So for each question, it's spent around 2,000 tokens. So not always it gives me theright answer here. So, yes, I have not so good accuracy, and the average is 1,000 tokens. Now let's use my new agent with the semantic approach. The thing the first thing that I do is, for every single query, for every single prompt, I send the query to my search tools, and I'm going to receive the top Q, the top K, three most relevant, with more probability to respond to my query because this is a semantic search inside my vector store. And then I use that response. I'm going to select that name, and I'm going to put that tool inside my agent. So I'm only going to use the three tools that this semantic search retrieved me for my agent. And I send that only tools. And this is my function helper, and that's it. So I'm going to send all the queries again, the same. And for the first question, I have 4,000 tokens, 3, blah, blah, blah. And, yeah, we have a huge difference because I don't send all the 29 tools in all the queries. So, yeah, pam, pam. And, okay, semantic memory. Yeah. Okay. Now, come on. Finish. So, well, let's go to the next one. So this agent is only sending a question, and I receive a response. I don't have any conversation with this agent. What happens if I start to have a conversation? If the agent remembers me. So we need to send only the tools that this agent is going to use. Because if I give all the tools I've been using in the conversation history, it's going to be in a moment that I'm going to have the 29 tools inside my agent. So we are not resolving we are not having a we are not resolving our problems in the agent when we're having a conversation with the agent. So we need to put the tools, and then we need to remove the tools. And we can do that with the swap tools. And let's run this. And what is doing swap tools? I'm going to run this. No. Let me show you first the swap tools function here in the register. I delete this because with a trance, in each invocation, you have complete control of the status because this is an agent loop. And you can use this agent loop how? Because this is an agent loop, and you can do almost whatever you want. You can put and remove everything inside the loop with some few lines of code. So let's see. Here we have the agent. We have the tool register through the tool registers. It's something inside the agent status. So we can clear the tools, and that's it. So in the next invocation, we can clear all tools, and we can add the new tools. Here, where is my I have a lot of things here. So I run this. It's going to take a moment. And, yes, we can see that in each invocation, my amount of token is increasing. Why I have more? Because I'm sending the chat history as well. So I send the tools that I need, only the tools that I need, and the chat history. So that's why I can see that my amount of token is bigger and bigger. And that's it. The accuracy is better. And, yeah, we have more tokens. So, yes, probably we don't have the best accuracy here because this is a super demo with super dummy tools, and some queries are ambiguous on purpose because search for something, book for something, check something. And the demo has generic tools, dummy tools, with some similar names. And when all the 29 tools are visible, the model sometimes picks the wrong one. And with the filtering, those generic tools only appear if the query actually matched them. So you can run everything, and you can test this. So this is only the wrong, only locally. But what happens when you want to put this demo in production? You can, of course, you can build a bigger vector store. I don't know. You can use Postgres. I think it's too much. But we on AWS, we have Amazon Vector Agent Core. So Amazon Vector Agent Core is a service dedicated only to agents in production. And inside of Agent Core, we have Agent Core Gateway. So Agent Core Gateway allows you to build this index alone. So you only have to say, "Hey, this is my tools," and Agent Core Gateway is going to build everything for you. And it can have the vector search inside. So the routing layer is inside the Agent Core, and it handles the tool selection automatically. So you register your tools once, and it finds theright one for each request. It's the same principle, but without infrastructure to manage. Now let's see the next one. ### Graph-RAG **Elizabeth Fuentes** [18:11] Graph-RAG. You know, RAG is Retrieval Algorithm Generation. It's how agents access your own data. You take the user questions, search your documents for the most similar content using vector search, and pass what you find to the model. The model answers from that. It's worked well from open questions. Finding something about this topic. But there is a category of questions where that breaks down. What is the average rating across all hotels in Paris? How many hotels have a pool? Vector search always returns something, even when nothing is truly relevant. And the agent only sees the top-end chunks of your old data at a time. It cannot aggregate, count, or traverse relationships across all the full data set. So it estimates. And it presents that estimate as a real fact, you know, a real answer. So here we have RAG. You build a vector store, and it retrieves three chunks from 300 documents, and the model guesses. If we are using Graph-RAG, you can run a query across all the data and return a compute result. Graph addresses this differently because instead of a retrieving text chunk, you can build a knowledge graph from the documents. Knows relationship structure data. For the demo, I'm going to use Neo4j locally, and the model is going to write a CyperQuery to search it. CyperQuery is Neo4j query language. It's similar to SQL. So the graph runs that query across all the data. The model gets back a compute verified result, not a sample. And before running this demo, I need to install the dependency. Let me show you. Let's go to the call. So, Graph. This is the notebook I'm going to share with you. So we need to install the requirements again. What we have here, let me see. This is the first one. Yeah. Requirements. I have I'm going to use OpenAI again. Let me just close this. I'm going to use Neo4j. And we need Neo4j for Graph-RAG. And for the agent that is using normal RAG, we are going to build a super simple vector store in FAISS. And we are going to use, again, the sentence transformer. And let's see. I already have all I invoke my OpenAI. This is my tools I because I want to create some tools here, I'm going to use OpenAI. And I'm using Graph database. And this is my I have my Neo4j locally. And let's run this. And this is something to check Neo4j. So I'm building my vector store. This is my FAISS. And this is my Neo4j that I have here locally. I don't know if I can show you. I think no. And I have a tool, a normal tool, created with Decorator. This is to search inside my vector store. You see that super I have the query. I create the embedding for my query, and then I search inside the vector store. And I have the query for the knowledge graph. So here, the model needs to understand that to search in the query, in the graph knowledge base, it needs to build a CyperQuery. So I put that in the context because we already know how to build tools,right? I put that in the context. I have my driver to send the data to connect with the vector store and to send the CyperQuery that the model is going to create for me. And this is to read the results. And that's it. That's the only tool that I need to search in my knowledge graph. Okay. This is my model OpenAI. This is my RAG, my RAG agent, and this is my Graph agent. I have two different agents to compare the results. Let's see. Yeah. I'm ready. So let's do let's run the first test. Aggregation. Here, what is the question? What is the average guest rating across all hotels in Paris? So meanwhile, this agent is oh, it's already done. So what I have here, I have the average guest rating across the hotels listed in Paris is it calculates, you know, when something when I'm using RAG, you go to the vector store, it receives the n possible answers for this question. Because this is an aggregation, it's going to use the data that it receives to build a mathematical operation. This is the thing that I have here. So everything that we see here is the model reasoning. So when I run the Graph agent oh, let me the agent is only going to give me the answer, you know, only these tokens here. Because the CyperQuery, it can give me the mathematical operation itself. It can do that. So we don't need the agents, the LLMs, the model to do that for me because the CyperQuery already gives me theright question. So it is 47 and the average. So here in this first, we are lucky because probably they are, yeah, only two hotels. But what happened is this vector store has more than two or three hotels in the vector store. So we are not going to give a real answer. It's going to the LLM is going to calculate with only three answers that it receives. So it's going to build this operation with only three hotels. But if the vector store has more than three, we are going to have some problems. Something that and I yeah. So let's run the next one. The Percy's County. Something similar to how many hotels have a swimming pool as an amenity? How many again? So it gives me the traditional. It's appear that the search did not return any specific information about hotels in Paris. Okay. What happened with the other one? There are currently no hotels that offer swimming pools. No. Why? There is no hotels. So the other one is like, "Would you like to ask about hotels with other specific amenities or information?" It's like, "Maybe it is," or, "Maybe, I don't know." Yeah. It's not so accurate. Okay. So multi-hop reasoning. Oh, what is my question? What are the room types and price for the highest rated hotels? What are the rooms types and price? These two questions. So it serves the fact I have only one. The highest rated hotels in any company in Paris, blah, blah, blah. I don't speak French. With the guest rating, it's that one. However, I currently don't have okay, blah, blah, blah. A lot of data there. So what happened with the other one? Receiving notification first. Oh, I have an error. And here, in the second one, I received the question. The highest rated hotel is Harmony, blah, blah. They offer following types. But unfortunately, these rooms are not available. So I'm receiving a answer. I don't receive a lot of blah, blah, blah there. I only receive the answer, what I need. And for the next one, out-of-domain detection. What is the question? Tell me about hotels in Antarctica. A spoiler. There is no hotel in Antarctica. There is zero hotels in Antarctica. And let's see. It's appear that the search did not return specific information about hotel. Okay. Because they don't have. I search for and they don't have the hotel, blah, blah, blah. If you are looking for a particular direct experience or specific requirement about visiting Antarctica, no. Please let me know, and I can assist you for it. Okay. A lot of tokens that are spending there in that answer for LLM. What happened with the other one? There are currently no hotels listed in Antarctica. Of course, because it created the CyperQuery, the CyperQuery, and it received zero. So it no. It gave me an honest answer. Here, some summary that Claude created for me for in this notebook. But here, it's something that I want to show you before go to the other one. So here, something that I love for Neo4j because why I'm using Neo4j? Because in the library, the Neo4j gives me it can build it uses an LLM. I'm using the OpenAI as well to build the knowledge graph. So how I build my knowledge graph, I only have a bunch of data, a bunch of TXT, only text. And I send that data to the Neo4j library that I may set. Here, I send that data to this. And Neo4j, using this library justright here, it can understand all my data, and it can build the graph for me. So I don't need to create that using the simple knowledge graph pipeline inside the knowledge graph library. So that's why I'm using Neo4j. It's amazing. It's super simple to use. So I invite you to use that. So let's go to the next one. ### Multi-Agent Validation **Elizabeth Fuentes** [29:14] So multi-agent validation. Sometimes an agent fails, and nobody finds out. It calls a tool, and the tool returns an error. And the agent does not surface that error. It generates a confident success response instead. The user thinks it worked. You think it worked. It didn't matter. The agent acts and validates its own output in the same loop. There's no separation, no second opinion. So when something goes wrong, it rationalizes and tells you, "It's okay. It worked." Here is what happens. Inside a single agent, when it fails, it calls the tools, gets an error, relationalizes it, and returns a success response. The user never sees the error. You can address this by adding a validation layer. You can have three agents in sequence. One acts, one checks, and the other approves or rejects. A strand agent has a built-in class for this called SWAM. It manages the handoff between agents automatically. You just define the role of each agent in a system prompt. And let me show you that. So this demo only needs a strand agent with OpenAI integration. Let's see the requirements here. We only need a strand agent. We don't need anything more. Let's go to the notebook here. And the key important thing here is they already run this. Let me see here. It's the SWAM. The SWAM, it's what lets you connect multiple agents together without creating, like, a for or while manually to put all these agents together. It can build the chain and manage the handoff between them automatically. So here, we are going to create three agents. Let me go on that because we have the normal and we have some ground truth data. So here is to create a single agent. So we have we are going to test three different scenarios to validate a booking. They said they would expect true. To available hotels, we expect false. And no existing hotels and missing booking, we expect false too. This is how we create a single agent. We need a prompt, some tools. And we are using the tools that we have here. And here, we know that this is the data. So the agents can give us the answer that we are looking for. And that's it. Let me go to the other ones. Oh, wait. Here. So how we build the SWAM. I already run this. I want to show you the SWAM. So we are going to build three different agents: the Executor, the Validator, and the Critic. We have a system prompt. You are an Executor agent for a hotel booking system. Use the provided tools to fulfill requests accurately, and blah, blah, blah. The Validator is you are a Validator agent. Review what the Executor did and output exactly on/off. And we have the Critic that is going to say, yeah, approve or no reject. Let's run this. And oh, what is here? Oh, model. Didn't define the model. Sorry. Sorry. Please forgive me the life because I didn't run this one. Let's run because this is the normal. And the model is there,right? What? Bit of bias. Run that. What's happened with you? Let me go this. Let me copy and paste this one. I don't know why this gave me problem. Multi-Agent SWAM. It's the same thing. Come on. Yeah. You can see this is live. I don't going to edit this. So yes. Okay. We have thank you. And the SWAM. How we create the SWAM. We have SWAM. And the SWAM function, we put all the agents together, and the entry point is the Executor. So all everything is going to start in the Executor, and then it's going to handoff. And. Six times because you can handle that too. So get the SWAM final response. Let's do a for and send all the response. And put a ground hotels. I have booked a ground hotel for Alice tonight. Booking, blah, blah, blah. Handoff. Valid. Okay. Let's go. Verdict: approve. So the Critic is approving this. This is the other one. And we can run some comparison here if you want. So here, we can see that the SWAM handles the flow between all the three different agents. It watches what happens when the single agent tries to configure something that does not exist in the system. And now, the same request through the SWAM. The Executor gets the error. The Validator catches it. The Critic rejects it. And the user never sees a fabricated response. So you can we can test here with a single agent that it's a sustented on no entry and return success. And the SWAM, it has an Executor that got the error, validated, saying, "Hey, come on, man. Hallucination and Critic, they reject." So the user sees a clear failure. They fabricate confirmation. So missing the user here. And you can run this to compare the two different agents. So here, we can see that the single agent, it has everything. Like, yeah, it's okay. It's correct. But in the multi-agent SWAM, you can see that the last one is a, you know, missing issue here. Something is happening. Now, let's go to the next one. ### Neurosymbolic Guardians **Elizabeth Fuentes** [36:06] Okay. Neurosymbolic Guardians. You have a rule for your agent. Let's say maximum 10 guests per reservation. You write it in the system prompt. You even write it in the tool description. And the agent still calls the tool with 15. Not because it is ignoring you. Because prompts probably are suggestions, not constraints. The model processes them as text, not as logic. It has to execute. It's probabilistic. Only code executes logic. A rule in the prompt, the model reads it as a suggestion. A rule in the code, the model cannot skip it. Neurosymbolic Guardians' rules put the rules in the code. A strand agent has a feature called hooks. The function that a strand calls automatically at specific moments in the agent loop. It is this caseright before a tool executes. So you have a hook. You write the rule, check the parameters, and if they fail, you cancel the call. Let me show you this in the code. Here, we have the Jupyter notebook. We have the requirements here. We have the same requirements as the previous demos. We only need I'm starting everything again. We only need a strand and OpenAI. We don't need anything else. And my API. And here is the thing important. In this demo, these three hooks are provided in the strand-based class. So that allows you to create the hook. So we have hook provider, hook register, and before to call event. And the tool hook register is what a strand passes you to register your callback. And before to call event is the event that finds every time the model is about to execute a tool. That last one is what makes it possible to intercept the call before it runs. So you have the before to call event. And of course, we have after to call event. But we are not going to use that here. And let's see. The rule in the prompt, remember, the model reads as text, and it may follow or not follow them. So let's see this. I'm going to run this. So here, we have some simulated state that we are going to use in this agent. And let me this is the symbolic rule. So booking rule. If booking rule this is something that we create. Let me show you the rules here. So we have the rules. Rule one: validate dates. Check is must before checkout. What is validate date checkout? This is something that it checks. This is something that is going to invoke when the rule is must. We have to use it. And we have another rule for max guests. Maximum guests per booking. So if I want to use I don't know. If I want to booking I want to do a booking for 11, it's going to block me. It's going to reject the booking. And we have some confirmation rule. Payment before confirm. You know, you can confirm if you don't have the payment. Cancellation rule. Cancellation window. Cannot cancel within 48 hours of check-in. So this is something that I'm creating here, you know, to this demo. So I have booking rules, and I have confirmation rule. I don't I'm using here the cancellation rule. So create validation hook. We create the validation hook. He just writes it like this. Neurosymbolic rule. And we use the hook provider. So we have a bunch of code here. Then we add the booking rules, the confirmation rules in the state. And you can check this by yourself later. But we want to see the demo running. So we define the clean tools that we are going to use for the booking hotels and the process payment. This is the normal tools, you know. And we need to add the tools to the hooks. I think I add that. Yeah. Tool name: book hotel. So this thing is going to get created when the book hotel is using the yeah, the book hotel. Okay. So let's create the agents for the comparison. So we are going as the other demos. We have three different scenarios. Configure booking without payment. Payment must be verified before the confirmation. This is the rule that it must trigger this. We have booking hotel exceeding guest limit. And we have valid booking for five guests. So yeah, we have the three scenarios. And we have the normal agent, the normal agent, the Vaseline agent. And we have the agent with the neurosymbolic guardians. So we have the hook, the neurosymbolic hook. This is the same. Yeah. And we add this if you can see, the normal agent is only three lines: the tools, the model, and this is the line that gives us the difference between these two agents. So I already run that. Now, let's go to configure a booking without payment. So I run in this my agents. The first one is come on, man. Yeah, because yeah, it configured the booking without payment because it doesn't have any rules, and the prompt is super basic. And the other one is let's open this. And it's booking blocked. Payment must be verified before the confirmation. Thank you. So it's okay. Now, let's run the other one. Test second scenario. What is the question? Booking hotel excess guest limit. So the question is yeah, I have more than. So here, the hotel have been successful booking for yeah, because the prompt is basic, and I don't put in any rule there. And for the other one, it seems that the ground hotel has maximum capacity of 10 guests per booking. Additional booking must be made at last one day in advance. I don't remember the date, probably. This is something that it gives them. And to validate booking is both agent execute or rule pass because yeah, it's only a validation for booking. So book hotel for blah, blah, blah. Or rule passes. It seems that the booking still needs to make I don't I didn't remember the question. So booking is made provide ah yeah. Okay. Okay. I get it. And okay. Now, run all the this question. Confirm without payment. Bookings if max. And what happened here? This is all the scenarios that we just run. So this is something to compare the results. So valid booking can get five guests. Allows. Correct. Run. 50. Block it. Run. Correct. Confirm. So we have here the comparison between the two agents. So what we have here is same model, same tools, same prompt. And the difference, the outcome, because the rules are in Python, knowing the prompt. This pattern of forcing rules in code before the tools run is also what Amazon AgentCore policies service that we have does at the infrastructure level. So the same concept, but managed for you in production. And you only have to create the rules. So the hooks are all or nothing. They block everything or approve. But sometimes, you want the agent to adjust and keep going. Not stop and leave the user waiting. That is what I will show you in the next ### Runtime Steering **Elizabeth Fuentes** [45:33] runtime guardians. Runtime steering. Steer. Don't block. This is the next one. Hooks blocks unconditionally. The agent stops, and the user has to retry for a hard constraint. That is exactly what you want. But sometimes, the rule is softer. Maybe room fits for four guests, but a group of six could go book two different rooms. Or a fly is full, but there is ability on the next one. You do not you don't want to block everything. Probably, you want the agent to find an option and complete the task. That is a steering. Here, we have the hook that fires and the task fail. And with agent control, steer the models and the task complete. The other different here is operational. Because with the hooks, it's as changing a rule means changing code and redeploy the whole harness, all the agent. And with the agent control, which is the name of the open-source library that we are going to use here, rules are registered on a local server via API. You obtain them without touching the agent code because the agent picks them up immediately. Let me show you that here in the code. Yeah. This is the notebook. In this demo, we only need an extra packet. The agent control SDK. So the agent control is the one that helps us to create the steering. And here, we are using the setup control. This is a little application, a little app that I create with the local server and the steering rules. Here, we have the local server. We have the control. Okay. There are the steering rules. First, we have the steer max guests. Steer, you know, guide. So guide agent to reduce guest count when exceeding maximum of 10. And it's going to steer. And we have some control that deny. For example, deny no payment. Block. Blocking confirmation without payment. And here, well, this is to create the service. The server. And let's go to the notebook. So I already let me go here. Yeah. So I need the environment. So this is the agent. We're going to have an error here. Wait, wait, wait. I'm going to use Bedrock. Wait, wait. No, then let me comment this one because this is going to give me an error. And let's run this. Yeah. So we have the hooks as we did in the demo. So we have book any company in Lisbon, and we have some prompts. You are a hotel booking assistant. When booking first, describe why you will booking, blah, blah. And this is the prompt for my agent. What is my agent? Agent control. This is for the well, this is some helper. And this is my hook that you already know because we create this. So we have the system prompt, and we have the hooks. So let's test this agent with the book any company Lisbon for 50 guests. And if you remember, this only can book for less than 10 guests. And of course, it's blocking. Now, let's go to the agent control. New agent. Here, we have two key importance or two key imports that are important for the agent control SDK. So we have the agent control plugin that captures agent events and sends them to the agent control server. And we have the agent control steering handle that listens for steering decision for the server and delivers them back to the model. Together here, they are what connects strands to the agent control steering logic. So let's go this with this. So the steering agent is going to we book a room for any company Lisbon, 50 guests. Let's see what it's doing. Yeah. I have successful booking for a stay any company Lisbon for 50 guests from May. So it's the reservation have it split into two rooms. So it do that it itself, you know. One room for one, and other room for five. And that's it. So use hook for hard constraints. Agent control for softer rule. Now, we have five technique or running locally. But how do you take this to production without maintaining service, without building infrastructure? ### Production Deployment **Elizabeth Fuentes** [51:29] Let me show you how. Everything that I just built here in the previous demos runs locally. For the production version, Amazon Bedrock Amazon Bedrock AgentCore give you the runtime, a gateway, a short-term and a long-term memory, a cloud watch observability built in. No servers to manage. Here the architecture. The strands agent runs inside the runtime. And you know, inside the runtime, you can put every framework that you want. Not only strands. The gateway routes tools calls to the Lambda function automatically as a tools. And the steering rules from the previous demo live in the DynamoDB. So you change them there, and they are live on the next call. And you don't need to redeploy anything. And if you want to use Neo4j, of course, you can use Neo4j AuraDB. That is an external graph database, which also has a free tier. And the code is in the repo here. It's in the repo. And you will need AWS credential if you are using Amazon Bedrock AgentCore. And in the resource link, there is some credits. I hope that you can find it because I always try to give away some credit for AWS so you can deploy everything for free. And if you are new here in AWS, I have a repository that you can use to deploy everything of this architecture using a notebook tool. But if you are familiar with CDK, Cloud Developer Kit, you can do it as well to deploy everything at once. Both options are in the repository of Core. And you can go deeper in AgentCore with all the documentation that I left there. So please don't stop at the demo and try going to production with Amazon Bedrock AgentCore. So let me bring it all back. Tokens waste on every request. You can fix it with semantic tool selection. And confident answer, it's never computed. ### Recap **Elizabeth Fuentes** [54:08] It's possible because you are asking how many. So you can use Graph-RAG and query the data. Don't sample it. Sometimes, we have fabricate success confirmation. You can use multi-agent validation and second pass catch it. And rules the models are quite a little skipped. You can use Neurosymbolic Guardians to enforce in the code. Don't trust in the prompt. And for the hard blocks that stop the user, you can use runtime steering, self-correct, and finish. Each demo that I just showed you is in the repository as a notebook and an application as well. Start with the demo one. Then go to the demo five. And if you want, deploy into productions. So have you tried any of this in your own agents? Thank you to join me in this session. And happy building. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.