# Your Agent Is Wasting Tokens and You Don't Know It - Erik Hanchett, AWS AI Engineer · 2026-06-28 Erik Hanchett, a Senior Developer Advocate at AWS, presents five specific techniques to reduce token costs in agent workflows without changing prompts or models. He demonstrates caching system prompts with 'cachePrompt' to avoid resending full context, and routing tasks by difficulty using cheaper models like Claude Haiku for simple requests. He advises offloading large tool results by storing them externally and sending only summaries, capping tool loops with a max iterations setting to prevent infinite loops, and trimming conversation history using a sliding window that keeps only the last 10 messages while optionally summarizing older context. These methods directly address common cost pitfalls, such as using expensive models for trivial tasks and letting context windows grow unchecked. ## Questions this episode answers ### How can I cache system prompts in my AI agent to reduce token costs? Erik Hanchett demonstrates using `cachePrompt = default` in AWS Strands Agents, which sends the full system prompt only on the first call and a much smaller version on subsequent calls, cutting costs. You can also cache tool prompts and messages. This simple change doesn't require altering prompts or switching models. [0:19](https://aie.addtry.com/785441bd-52ba-489e-9f27-9498c99c8213?t=19000) ### Is it cost-effective to use the same expensive AI model for every task in my agent? No, Erik Hanchett advises routing tasks by difficulty. Use cheaper models like Claude Haiku for simple tasks and reserve Claude Sonnet for harder ones. You can even employ a cheap model to decide which model to use. This avoids wasting money on overqualified models for simple inference calls. [0:56](https://aie.addtry.com/785441bd-52ba-489e-9f27-9498c99c8213?t=56000) ### What should I do when my agent's tool results are too large and filling up the context? Erik Hanchett recommends offloading large tool results and using summarization. Store the result locally or in the cloud, then send a summary back to the LLM instead of the full result each time. This prevents the agent from bloating the context window during tool loops, saving tokens significantly. [1:43](https://aie.addtry.com/785441bd-52ba-489e-9f27-9498c99c8213?t=103000) ### How can I stop my AI agent from looping tool calls endlessly and wasting tokens? Set a maximum iterations for tool loops. Erik Hanchett warns that without a cap, an agent might run 10–20 times or get stuck in an infinite loop, disastrous for token usage. He also suggests using observability tools pre-deployment to check tool call duration and loop counts to optimize efficiency. [2:42](https://aie.addtry.com/785441bd-52ba-489e-9f27-9498c99c8213?t=162000) ## Key moments - **[0:00] Intro** - **[0:19] Prompt caching** - [0:19] Cache the system prompt with cachePrompt=default so subsequent agent calls send a reduced prompt, saving tokens. - **[0:56] Model routing** - [1:40] "Don't use the most expensive model for everything you're doing," Erik Hanchett. - **[1:43] Offload results** - **[2:42] Cap loops** - **[3:38] Trim history** - [3:38] Use a sliding window conversation manager to keep only the last 10 messages in context and summarize older history to save tokens in multi-turn agents. - **[4:51] Conclusion** - [4:51] Erik Hanchett's five token-saving techniques: cache prompts, route by difficulty, offload tool results, cap tool loops, and trim history. ## Speakers - **Erik Hanchett** (guest) ## Topics Agent Engineering ## Mentioned AWS (company), Claude Haiku (product), Claude Sonnet (product), Strands Agents (product) ## Transcript ### Intro **Erik Hanchett** [0:00] Hey everyone. My name is Erik Hanchett, I'm a Senior Developer Advocate at AWS, and I'm going to talk to you about how you can save on token costs. Now I'm going to show you 5 ways that you can reduce your token costs while using and creating agents. So the first way you can do that is to cache your system prompt. Let me show you some code. Now, I'm using AWS's Strands Agents. This works with all different providers. This is a little bit of pseudocode, but the idea is that you can add cachePrompt equals default. ### Prompt caching **Erik Hanchett** [0:39] And what that'll do is, on the first call of your agent, it will send the full system prompt over, and then on every subsequent call it will have a much reduced system prompt being sent over. So it'll be cached. You can also cache the tool prompts and messages as well. This may sound obvious, but you want to look into routing your different messages based on the difficulty. So here's a code example. Let's imagine that you have a task that's very difficult. ### Model routing **Erik Hanchett** [1:15] You may want to use one of the newer frontier models. However, if it's something simpler, you want to use a cheaper model. In this case, maybe we use Claude Haiku for a cheap something cheap, and then use Claude Sonnet for something a little bit more difficult. And then you can use an if statement. You can even have another model that's very cheap decide which model to use. So you can play around with this, but I highly recommend don't use the most expensive model for everything you're doing. ### Offload results **Erik Hanchett** [1:43] You want to use multiple different models based on the use case, and then try to route to it inside your agent. Another good tip is to offload the tool results. Let me show you some code on here. Once again, I'm using Strands Agents. This is a manual way to do it. There are some additional APIs that the Strands Agents offers to do this. If you have a large tool result that's coming back, you can store it locally or in the cloud, and then use some kind of summarization that saves on tokens. So that way, when it's being called over and over again, the tool result isn't added into the context every time every time the tool loops or every time the agent loops. So if you can find any way that where you have this tool result that you don't necessarily send it on every single call back to the large language model, that will save a lot of tokens for you. And like I said, there's a few APIs to do this, but essentially you can do this summarization technique. You can also cap your tool loops. So when you're dealing with the agent loop and it decides to do a tool call, I've had this happen often where it calls the tool over and over and over again. ### Cap loops **Erik Hanchett** [3:00] And if you don't cap that tool call, then it might run 10, 20 times. It might get into an infinite loop, which would be very bad for your token usage. So always set a max iterations of how many times it will loop. A good thing you can do before you deploy your agent is to run some observability tools and take a look at the tool call use for every single tool, and then see how long each one of them is running and how many times they're looping. And that way you can get an idea of how efficient the tool call is. Last but not least, we can trim the history. So if we're using a multi-turn agent and we are talking back and forth, you will find at times that the conversation history will get very large. On every single call, that whole conversation history will be sent back to the large language model. ### Trim history **Erik Hanchett** [3:59] And this can eat through hundreds, if not thousands, of tokens. In Strands Agents, we have something called Sliding Window Conversation Manager. Which this does is it looks back at the last 10 messages and only sends those back. And you can set this to whatever you want. And that way you're not sending these huge message histories back to the agent every single time a new message comes in. The downfall of this, or the trade-off of this I should say, is that you will lose the message history from the beginning. The way you want to deal with that is you can use some sort of summarization of the history and then put that into the context window. So rather than sending all of it, you may send a small amount once you hit this sliding window. This will save you a lot of tokens. So in conclusion, we have 5 things: cache the system prompt, and if you can, maybe the tool prompt messages. Route by difficulty. Don't use the same expensive model for everything you're doing, for every single task. Offload these big tool results. ### Conclusion **Erik Hanchett** [5:08] So if you have a large tool result, you can summarize it and not have it being sent in the agent loop and overloading your context. You can cap those tool loops. Make sure you use observability tools to see how long tool calls are taking and how many loops they're doing, and then try to iterate over that. And then, of course, trim the history if you're using a multi-turn conversation with your agent. So that way the whole conversation isn't being sent over and over again for very long conversations. Thank you for listening to my very quick, almost lightning talk today. If you'd like to go deeper, find me on LinkedIn at Erik Hanchett. I also blog at programwitherik.com. And then you can find me on social media at erikch. Love to talk to you guys more. Thanks. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.