# Architecting Agent Memory: Principles, Patterns, and Best Practices — Richmond Alake, MongoDB AI Engineer · 2025-06-27 Richmond Alake from MongoDB presents memory management as the key pillar for building believable, capable, and reliable AI agents, arguing that agentic systems require structured memory types—persona, toolbox, conversational, workflow, episodic, and entity—to achieve statefulness and reduce reliance on prompt engineering. He introduces MemoRiz, an open-source library implementing these memory design patterns, and positions MongoDB as a flexible memory provider with its document model and hybrid retrieval capabilities (vector, text, graph). Alake details practical patterns: storing tool schemas for scalable tool use, persisting conversation history with timestamps and recall signals, leveraging workflow memory to learn from failures, and using MongoDB's upcoming integration of Voyage AI embedding models to simplify chunking and retrieval. He connects these advances to neuroscience, citing how feline visual cortex research inspired CNNs and noting recent collaborations with neuroscientists to further agent memory research. ## Questions this episode answers ### What is MongoDB's plan to simplify memory management for AI agents, and how does the acquisition of Voyage AI fit in? Richmond Alake says MongoDB is positioning itself as the memory provider for agentic systems. They acquired Voyage AI for its leading embedding models and rerankers, and plan to integrate them directly into MongoDB Atlas. This will let developers avoid writing custom chunking and retrieval strategies, making the database itself handle these concerns to boost developer productivity. [14:04](https://aie.addtry.com/f294d3aa-a76f-4e1c-b4bb-0eafb30f4a01?t=844000) ### What memory types does Richmond Alake's open-source library MemoRiz implement for AI agents? MemoRiz is an experimental, educational library that encapsulates design patterns for multiple memory types. Alake lists persona memory (for believable, human-like interactions), toolbox memory (scaling tool schemas in the database), conversational memory, workflow memory (storing failed steps as learning experiences), episodic memory, long-term memory, an agent registry, and entity memory. [9:14](https://aie.addtry.com/f294d3aa-a76f-4e1c-b4bb-0eafb30f4a01?t=554000) ## Key moments - **[0:00] Intro** - [0:15] Memory management will be the most relevant skill for building believable, capable agents within 6 months, predicts Richmond Alake. - [0:43] Audience member interrupts: 'You're not going to get there' — Richmond Alake responds 'We're going to get there.' - **[1:36] AI Evolution** - [2:16] Agent 'agenticity' is a spectrum, not a binary definition, similar to self-driving levels, says Richmond Alake. - [3:09] Memory is essential for agents to be reflective, interactive, proactive, reactive, and autonomous, says Richmond Alake. - **[4:17] Human Intelligence** - [5:24] Procedural memory in agents mirrors the human cerebellum, which stores skills like backflips, says Richmond Alake. - **[5:48] Agent Memory** - **[7:21] MongoDB RAG** - [7:55] RAG requires more than vector search; MongoDB provides graph, text, and geo-spatial queries in one database. - **[8:56] Memory Types** - [9:25] Richmond Alake unveils MemoRiz, an open-source library implementing design patterns for AI agent memory. - [9:44] Richmond Alake on OpenAI's ChatGPT personality update: 'They didn't do a good job, but they are going in the right direction.' - [10:45] Database as toolbox for tool schemas scales beyond OpenAI's 10–21 tool limit in the context window. - [12:12] Storing failed workflow steps as learning experiences prevents agents from repeating mistakes, says Richmond Alake. - **[13:20] MongoDB Memory** - [14:04] MongoDB acquired Voyage AI to embed top embedding models and rerankers into Atlas, reducing hallucination. - **[15:37] Neuro Inspiration** - [16:17] Neuroscience research on cats' visual cortex inspired convolutional neural networks for face and object detection. - [17:00] MongoDB's AI scientists met with neuroscientists and MemGPT creator Charles Parker to advance memory for AGI. - **[17:22] Outro** ## Speakers - **Richmond Alake** (guest) ## Topics Agent Engineering ## Mentioned MongoDB (company), Voyage AI (company), ChatGPT (product), Claude (product), Mem0 (product), MemGPT (product), MemoRiz (product), Zep (product) ## Transcript ### Intro **Richmond Alake** [0:15] In the next 10 to 15 minutes—here's, I guess, my promise to you—I'm going to give you some information that will be high-level. There will be some practical component to it, but this information I'll give you within the next 6 months will be very relevant, and it will put you in the best position to build the best AI applications, to build the best agents that are believable, capable, and reliable. **Guest** [0:43] You're not going to get there. **Richmond Alake** [0:44] I know. We—we're going to get there. You know what? Just for you. There you go. You're welcome. So we're going to be talking about memory. We're going to be talking about the stateless applications that we're building today, and how we can make them stateful. We're going to be talking about the prompt engineering that we're doing today, and how we can reduce that by focusing on persistence. We're going to be turning the responses in our AI application and making our agents build relationships with our customers. And all of it is going to be centered around memory. So I'm going to do a very quick evolution of what we've been seeing for the past 2 to 3 years. We started off with chatbots, LLM-powered chatbots. They were great. ### AI Evolution **Richmond Alake** [1:36] ChatGPT came out in November 2022, and yeah, exploded. Then we went into RAG. We gave these chatbots more domain-specific, relevant knowledge, and they gave us more personalized responses. Then we began to scale the compute, the data we were giving to the LLMs, and it gave us emergent capabilities,right? Reasoning, tool use. Now we're in the world of AI agents and agentic systems. And the big debate is: what is an agent,right? What is an AI agent? I don't like to go into that debate because that's, like, asking what is consciousness. It's a spectrum. The agenticity—and that's a word now, agenticity—of an agent is a spectrum. So there are different levels. I came here and I saw Waymo, and to me it was pure sorcery. We don't have that in the UK. And there are different levels of self-driving, so you can look at the agentic spectrum in that respect. We have a minimal agent where there's an LLM running in a loop. Great. Then you have level 4, is autonomous agent, a bunch of agents that have access to tools. They can do whatever they want. They're not prompted in any way or in minimal way. But this is how I see things: it's a spectrum. So what is an AI agent? It's a computational entity with awareness of its environment through perception, cognitive abilities through an LLM, and also can take action through tool use. But the most important bit is there is some form of memory, short-term or long-term. Memory is important. It's important because we're trying to make our agents reflective, interactive, proactive, and reactive, and autonomous. And every—most of this, if not all, can be solved with memory. I work at MongoDB, and we're going to make—we're going to connect the dots, don't worry. So this is all nice and good. This is what you look at if you double-click into what an AI agent is. But the most important bit to me is—I'll go slightly. People are taking pictures, sorry. Allright, let's go. The most important bit is memory. And when we talk about memory, the easy way you can think about it is short-term, long-term. But there are other distinct forms,right? Conversational, entity memory, knowledge, data store, cache, working memory. We're going to be talking about all of that today. So these are the high-level concepts. But let me go a little bit meta. Why we're all here today in this conference is because of AI,right? We're all architects of intelligence. The whole point of AI is to build some form of computational entity that surpasses human intelligence or mimics it. Then AGI, we're focused on making that intelligence surpass humans in all tasks we can think of. And if you think about the most intelligent humans you know, what determines their intelligence is their ability to recall. ### Human Intelligence **Richmond Alake** [4:39] It's their memory. So if we—if AI or AGI is meant to mimic human intelligence, it's a no-brainer, no pun intended, that we need memory within the agents that we're building today. Does anyone disagree? **Guest** [4:53] Yes. **Richmond Alake** [4:54] Good. I would have kicked you out. Okay, let's go. So humans, you. In your brainright now, you have these, you have this. This is not what it looks like, but it's close enough. You have different forms of memory, and that's what makes you intelligent. That's what makes you retain some of the information I'm going to be giving you today. There is short-term, long-term, working memory, semantic, episodic, procedural memory. In your brainright now, there is something called the cerebellum. I always get the word wrong, but that's where you store most of the routines and skills you can do. Can anyone hear your back flip? **Guest** [5:27] Yes. **Richmond Alake** [5:28] Really? Wow. You can see my excitement. Your—the information or the knowledge of that back flip is actually stored in that part of your brain. So I heard it's 90% confidence, by the way. That is actually—it is,right? I'm not going to do one, but—but it's stored in that part of your brain. Now, you can actually mimic this in agents, and we're going to—I'm going to show you how. But now we're talking about agent memory. Agent memory is the mechanisms that we are implementing to actually make sure that states persist in our AI application. ### Agent Memory **Richmond Alake** [6:08] Our agents are able to accumulate information, turn data into memory, and have it inform the next execution step. But the goal is to make them more reliable, believable, and capable. Those are the key things. And the core topic that we are going to be working on as AI memory engineers is on memory management. We are going to be building memory management systems. And memory management is a systematic process of organizing all the information that you're putting into the context window. Yes, we have, like, large context windows, but that's not for you to stuff all your data in. That's for you to pull in the relevant memory and structure them in a way that is effective, that allows for the response to be relevant. So these are the core components of memory management: generation, storage, retrieval, integration, updating, deletion. There's a lie here because you don't delete memories. Humans don't delete their memories. Except it's a traumatic one and you want to forget. But we really should be looking at implementing forgetting mechanisms within the memory management systems that we're building. ### MongoDB RAG **Richmond Alake** [7:21] You don't want to delete memories. And there are different research papers that are looking at how to implement some form of forgetting within agents. But the most important bit is retrieval. And I'm getting to the MongoDB part. This moving around, this is RAG. It's very simple,right? Because we've been doing it as AI engineers. MongoDB is that one database that is called to RAG pipelines because it gives you all the retrieval mechanisms. RAG is not just vector. Vector search is not all you need. You need other types of search. And we have that with MongoDB. Anything you can think of. You're going to be hearing a lot about MongoDB in this conference today. But this is what RAG is. And you level up, you go into the world of agentic RAG,right? You give the retrieval capability to the agent as a tool, and now it can choose when to call on information. There's a lot going on. I'll send this somehow to you guys. Or you can come to me and I'll LinkedIn it to you. Add me on LinkedIn and just ask for the slides and I'll send it to you. Richmond Alake on LinkedIn. This is memory. MongoDB is the memory provider for agentic systems. And when you understand that, we provide the developer, the AI memory engineer, the AI engineer, all the features that they need to turn data into memory to make their agents believable, capable, and reliable. You begin to understand the importance of having a technology partner like MongoDB on your AI stack. So these are—this is the same image, but just a bit more focused on all the different memories. I'm going to skip through this slide because I go into a bit of detail. ### Memory Types **Richmond Alake** [9:14] I'm also going to give you a library. I'm working on an open-source library. I'm ashamed of the name. I was trying to be cool when I came up with it. It's called MemoRiz. You can type that on Google, you'll find it. But it has all the design patterns of all of this memory that I'm showing you, all these memory types, and that I will show you as well. But there are different forms of memory in AI agents, and I will make them work. So let's start with persona. Is anyone here from OpenAI? **Guest** [9:42] No. **Richmond Alake** [9:44] Leave. I'm joking. Well, a couple—a couple months ago,right? So they gave ChatGPT a bit of personality,right? And they didn't do a good job, but they are going in theright direction, which is we are trying to make our systems more believable,right? We're trying to make them more human. We're trying to make them create relationships with the consumer, with the users of our systems. Persona memory helps with that. And you can model that in MongoDB,right? This is MemoRiz. You—if you spin up the library, it helps you spin up all of these different types of memory types. So this is persona. I have a little demo if we have time. But this is persona memory. This is what it would look like in MongoDB. Then there's toolbox. The guidance from OpenAI is you should only put the schema of maybe 10 to 21 tools in the context window. But when you use your database as a toolbox where you're storing the JSON schema of your tools in MongoDB, you can scale. Because just before you hit the LLM, you can just get the relevant tool using any form of search. So that's toolbox. That's mem—that's toolbox memory. And that's what it would look like,right? You would store all—this is how you model it in MongoDB. You store all the information of your JSON schema. Now, you'll begin to understand that MongoDB gives you that flexible data model. The document data model is very flexible. It can adapt to wherever data—wherever model you want your data to take, wherever structure. And you have all of the retrieval capabilities: graph, vector, text, geo-special query in one database. Conversation memory is a bit obvious,right? Back-and-forth conversation with ChatGPT, with Claude. You can store that in your database as well, in MongoDB, as conversational memory. And this is what that would look like: timestamp, timestamp, and you have a conversation ID. And you can see something there called recall recency and associate conversation ID. And that's my attempt at implementing some memory signals. But—and that goes into the forgetting mechanism that I'm trying to implement in my very famous library, MemoRiz. I'm going to go through the next slides a bit quicker because I want to get to the end of this. Workflow memory is very important. You build your agentic system. They execute a certain step. Step one, step two, step three. It fails. But one thing you could do is the failure is experience. It's learning experience. You can store that in your database. I see you nodding. You're like, yeah. You can store that in your database. And you can then pull that in, in the next execution, to inform the LLM to not take this step or explore other paths. You can store that in MongoDB as well. You can model that. Because what you have with MongoDB is that memory provider for your agentic system. And that's what—this is what that looks like when you model it. An example of it, anyway. So we have episodic memory. We have long-term memory. We have an agent registry. You can store the information of your agent as well. And this is how I do it. You can see the agent as tools, persona, all the good stuff. There's entity memory as well. So there's different forms of memory. And the MemoRiz library is very experimental and educational, but it encapsulates some of the memory and implementation and design patterns that I'm thinking of on an everyday basis, that we're thinking of in MongoDB. So MongoDB, you probably get the point now. The memory provider for agentic systems. There are tools out there that focus on memory management: MemGPT, Mem0, Zep. They're great tools. But after speaking to some of you folks and some of our partners and customers here, there is not—there is not one way to solve memory. ### MongoDB Memory **Richmond Alake** [13:44] And you need a memory provider to build your custom solution to make sure the memory management systems that you're able to implement are effective. So we really understand the importance of managing data and managing memory. And that's why earlier this year we acquired Voyage AI. Now, they create the best—no offense, OpenAI—embedding models in the market today. Voyage AI embedding models are—we have text multimodal, we have rerankers. And this allows you to really solve the problem, or at least reduce AI hallucination within your RAG and agentic systems. And what we're doing and what we're focused on, the mission for MongoDB, is to make the developer more productive by taking away the considerations and all the concerns around managing different data and all the process of chunking, retrieval strategies. We pull that into the database. We are redefining the database. And that's why in a few months we're going to be pulling in Voyage AI, the embedding models, and the rerankers into MongoDB Atlas. And you will not have to be writing chunking strategies for your data. I see a lot of people nodding. Yeah. That's good. So MongoDB is a household name, to be honest. We've—I watched MongoDB IPO back when I was in university. I bought the stocks when I was in university free, just free. I only had about£100. I was broke. But we are very focused and we take it very seriously, making sure that you guys can build the best AI products, AI features, very quickly in a secure way. So MongoDB is built for the change that we are going to experience now, tomorrow, in the next couple of years. ### Neuro Inspiration **Richmond Alake** [15:37] I want to end with this. You know who these two guys are? **Guest** [15:40] Noble Prize. **Richmond Alake** [15:42] Damn. Okay. This is Hobo Lunwizo. They won a Nobel Prize in the late '90s, but they did some research on the visual cortex of cats. They experimented with cats. This probably wouldn't fly now, but back in the '50s and '60s, things were a bit more relaxed. But they found out that the visual cortex of the brains between cats and humans actually worked by learning different hierarchies of representation, so edges, contours, and abstract shapes. Now, people that are in deep learning will know that this is how convolutional neural network works. And the research that these guys did inspired and informed convolutional neural networks. That's face detection, object detection. It all comes from neuroscience. So we are architects of intelligence, but there is a better architect of intelligence. It's nature. Nature has created our brains. It's the most effective form of intelligence. And, well, some humans that I meet, but it's the most effective form of intelligence that we have today. And we could look inwards to build this agentic system. So last week Saturday, myself and Tengyu, who is the chief AI scientist at MongoDB, also the founder of Voyage AI, we sat with these three guys in the middle, our neuroscientists. Kenneth has been exploring human brain and memory for over 20 years. And over here is Charles Parker. He's the creator of MemGPT, your letter. And we are having these conversations. And once again, we're mirroring how we're bringing neuroscientists and application developers together to solve and push us on the path of AGI. ### Outro **Richmond Alake** [17:22] So that's my talk done. Check out MemoRiz. You can come talk to me about memory. Add me on LinkedIn if you want this presentation. Thank you for your time. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.