Context Engineering: Connecting the Dots with Graphs — Stephen Chin, Neo4j

Hello everybody, and welcome to my session at AI Engineer Code Summit, and I'm going to talk a bit about how you can connect the dots with graph technology and solve problems like context engineering, um, improving retrieval patterns, and also agentic memory.

So we're going to have a lot of fun. My name is Stephen Chin, I'm VP of Developer Relations at Neo4j, and you can find me at all the different social media outlets with my handle, steveonjava. So, excited you're all here to join for the session today, and I think this is how a lot of us have felt the past couple of years, as AI technology has basically taken our jobs away.

We've become slaves to, um, AI programming, to prompt development, to building things off AI models. Now, it's not all bad, I mean, we have a lot more time to play games, to hang out in The Matrix, but what we really want to be doing is we want to be doing things which are higher value.

So this is where context engineering comes into the picture and transforms what we've traditionally been doing with prompt engineering to get different results out of the AI, and we're evolving that to have a more dynamic and wider scope of things, which we're feeding the AI as context, which gives us much better results.

So, um, this allows us to feed the desire of agents to get even more context and information to do things together, um, to have more dynamic models and applications, to make our applications goal-driven, um, selectively curate the information for the relevancy of the particular domain which we're working in.

So if you're working in a, um, an enterprise domain, if you're working with a lot of business context, this is particularly important. And then we can structure the input and get a lot more signal over all the noise of what's being entered into the model, which is one of the biggest problems with models today.

Um, huge context windows, but very little attention focus, and simply not looking at theright parts of the context to give us good results. And this allows us to think not like prompt engineers, but like information architects, where we're building the model context which actually gives us superior results coming out of the AI.

And this evolves us from being, um, you know, your traditional trapped matrix worker, to being superheroes. So this is, this is where we want to be. We want to be in control of our destiny. We want to be able to give the agents all of the information, all of the context which they need to perform the task and to do exactly what we want to get for results out of it.

And there's a lot of tools at our disposal now which allow us to manipulate, control the context, and really, um, feed the AIs all the information which they need to be successful. So in the, kind of the scope of context engineering, there's a whole bunch of things which are clearly, um, part of the domain that this now encompasses.

So one is prompt engineering. Of course, we need to design, engineer good, good prompts, um, make sure that the AI actually has theright instructions, theright information, and theright grounding which it needs to do its job well. But we also need to pull in from different data sources by using things like retrieval augmented generation.

So, um, RAG is still very relevant for the ability to pull in data from enterprise context, from different business contexts, and then supply that as additional information to the AI that it can use to make decisions. Um, pulling in state and history as well.

So now we actually want our models to want memory, um, both short-term memory so they can collaborate with each other, and also long-term memory so they remember the conversation state, the history, and, um, they can do more effective long-term operations.

And we also want to be able to structure the output in a meaningful way so we can actually feed into not only other applications, but other tools and things which we need to collaborate with and integrate our, um, context with.

And when you put this all together, this is kind of the, the scope and domain of context engineering. Now, one of the big focuses of this is all about memory. So it's all about how we capture the AI memory and what we're able to do with it.

So, um, there's kind of two main categorizations of memory. One is short-term memory, so this is what the AI is currently working with on the current tasks. Um, we want to compress as much information as possible into the short-term memory and give it relevant results which are high up in the search window.

Um, be able to integrate tool results into this as well, although, you know, not give it too much information from tools, especially from previous exchanges where the tools might have dumped a lot of output or information which will fill our context window.

Um, and in addition to this, we also need to mix in long-term memory. So things which you've learned over a long set of conversations, which might be episodic, um, we need to figure out the semantic and the structural meaning of past conversations.

Um, kind of pull this out into things which can either be used as instructions for the AI and also for procedures and operations which we can use to, to guide and plan the artificial intelligence. And, um, when we put this together, this helps us pull the more relevant context higher up into the context window, fill in the gaps, and then avoid a lot of the noise which gives us bad results or, um, hallucinations or other problems coming from our AI applications.

And, um, memory is really the core of what we need to accomplish. Um, you know, if you're plugging yourself up to the matrix, this is where you synergize all of your memories, all the things which you want to get into the AI, together with, um, your neuronal brain, your own neural network that you want to, um, express.

And it's extremely importantright now because LLMs are only as good as the quality of the response that they're getting from the data. So if you give them bad data, if you give them garbage, then you are going to get garbage back out again.

So we need to give them the, theright information in the context window and kind of limit and give it, um, move it up as high as possible. Be able to do more dynamic prompting with things like DSPY and BAML, um, ability to do more reasoning, um, so we can do internal context engineering on top of our data.

This will turn us from human developers into agents where we're actually using more agent technology to fuel our applications, to build things. Um, and then this allows us to focus more on the time which we're doing our tests rather than just focusing on the time which we're training our models.

Um, so together when we have more context now, it allows us to do better things, but then it's still important that we really have structured information, relevant inputs, and this improves the reliability and the explainability of our models that come out of it.

Um, so one of the ways which we can do this is by leveraging knowledge graphs. Knowledge graphs are a technology which has been around for a while, but they're very applicable for AI because they fill in that gap between the AI, which is very good at, um, creating things, building things, kind of pulling from different sources it has, but, um, structured information, knowledge graphs are a structured representation to understand a bit about how a knowledge graph is constructed.

It's typically built with facts, which are, are nodes about people, places, events, or things. Those are linked together by relationships or, um, um, lines between them which reference how those things are related. It's very easy for both humans and LLMs to read knowledge graphs.

So both acts as a, um, organizing concept, but also a way which you can understand what your AI is doing and actually look at some of the data behind it. And, um, it can be also very useful as a digital twin of your organization, of your supply chain, of, um, a whole bunch of different processes in your organization.

And the basic construct of a knowledge graph is, um, nodes which represent different people in the situation, relationships, and then you can attach properties to these nodes. So this is an example of a knowledge graph where you have two people, they know each other, they live with each other.

Well, one person lives with the other, so I guess technically it's Anne's house. And they both drive a car. Now, the car is owned by Dan or, or by Anne, but it's driven by Dan as well. So they both have a relationship with the car and they have a relationship with living with each other.

And you can see the, the, there's attributes on this, how long has Dan lived or driven the car, the type of car. So it's a Volvo. Um, it's a, um, model V70, some information about it, and also some embeddings.

So we can also encapsulate embeddings on the graph as well. So we can do vector lookups. And this allows us to do fairly complex things as we build larger knowledge graphs to capture all this information. And what knowledge graphs give us the benefit of is all of that knowledge, context, and enrichment that we can build into a representation of knowledge.

In addition to LLMs, which have kind of that language, reasoning, and creativity, and when we put them together, we can do really powerful things. Um, so we talked a bit about RAG being an essential part of context engineering, and an even better way of doing RAG is GraphRAG.

Now, what is GraphRAG? So GraphRAG is any retrieval pipeline which also uses graphs as part of the retrieval process. And so, um, an example of this is a user asks a question. Um, it goes to the LLM and it does a search and it asks for if there's any relevant information, which will go as a query out to a knowledge graph.

This then gets passed in as additional context to the LLM when it's answering the question, and then the LLM gives an enriched answer, which is more relevant. So it's, it's a, will give you more relevant results than just a vector similarity search because you also have information about relationships, about nodes, about community grouping, more context.

So you can now get domain information, factual information, structured knowledge on your subject. You can explain what the LLM is actually doing because you can see the part of the knowledge graph which got passed to the LLM. And you can also evolve the knowledge graph over time.

And you can now start to implement overlays like role-based access. So you can say only these people get access, for example, in a, um, patient information system. Only the doctor would have access to the diagnosis, but only the person who, um, handles the administrative information would have access to phone numbers or addresses or other personal information about the patient.

So it allows you to kind of overlay that role-based access directly on the knowledge graph and then instruct the LLM on what information it's allowed to respond with. And, um, knowledge graphs allow this sort of explainable AI. So in a, in a large graph with a lot of nodes and a lot of information, now you can store the learnings from the user and agents into the interactions of the graph context.

You can start to visualize conversation flows with the addition of reasoning. You can analyze the context data of agent systems about performance, um, identify opportunities for improvement over time of the, um, either the, um, the quality of the results which you're passing in, the relationships, um, removing duplicate nodes so that you get better quality results coming out of it.

So it gives you a lot of control over the application and the ability to modify and control what the AI is answering, kind of like you're, you're training in a, in a dojo. So I think, uh, you know, in the, in the film, Neo spends a lot of time doing virtual training, improving his skills with different programs he's loaded up.

And, um, this is how we're able to do a lot of amazing things like this demo which I'm going to show you. So the first demo we're going to show is a, um, GraphRAG demo using the LLM knowledge graph builder.

So I've already set up a Neo4j ORA instance. This is the, um, um, online free version of Neo4j. You can see I have a, a running instance with a bunch of relationships loaded up. And to load up those relationships, I use the knowledge graph builder.

The knowledge graph builder is a very simple web application. It's open source, and it lets you do a couple of things. So it lets you upload files so you can drag and drop different files into the user interface.

Before the presentation, I loaded up a couple representative files of a, um, supply chain use case. So one is a supply chain document. And as you can see here, it has a whole bunch of information about different artifacts, um, and the digital signatures of them and the relationship of them.

And the second one is the more interesting one. This is a, a VEX document which is a security standard, and it talks about some vulnerabilities, um, in this case inside the Jackson library and talks a bit about, um, how to remediate with it, which versions are affected, um, which commits fix it, and all that good stuff.

So, um, we have quite a bit of information which we loaded up. And then what I've done is I've already dropped those into the knowledge graph, and we can take a look at what got generated by the LLM.

So it takes this through an ingest phase, um, where the LLM actually builds out a knowledge graph. And then we can see that some of these nodes represent different parts of the, um, VEX document. Um, here we can see some information about, um, um, who found the vulnerability, information about the, um, vulnerability database URL.

And, um, all this stuff is connected with different relationships, and this allows us to query, navigate, and traverse this information to build better responses for the LLM. So what we're going to do in this demo is we're going to take this knowledge graph which we built, and then we're going to run an LLM which does a two-pass process.

The first pass, it's going to do a vector lookup and find a similarity search to find related nodes in the knowledge graph. And the second pass, it's going to take those nodes which are related to the result, find related nodes, and then pass those in as context to the LLM.

And ideally, what we'd, what we'd like to get from the LLM is that, um, it will answer questions with information it has from the knowledge graph, and then it won't be able to answer questions or refuse to answer questions with things which are outside of that knowledge, um, pool.

So let's ask it, um, about vulnerabilities in the, in the Jasper library. So Jasper is another, um, Java library that's very commonly used. It wasn't actually referenced in the VEX document. So we're, in this case, we're hoping to get a no response.

Okay. So that's amazing. I, I made a typo. I should have said Jackson. Let's see what we get when we, um, ask about the Jackson library, even with a typo, because LLMs know that humans aren't perfect and they're very good at fixing our mistakes.

And, um, here we can see that it actually pulled that information about the Jackson data bind library with an XML injection attack. It knows a bit about the vulnerability, what version it's in, um, whether it's fixed and in which version it's fixed.

And, um, all this information is, is pulled from and aggregated off the knowledge graph. So, um, it gives us quite a lot of information, um, very detailed and, and very focused because it's grounded in a, um, data which is, um, very, um, uh, complete.

Um, it's finite and it's something we can edit, modify, and change the response over time. So knowledge graphs are a very powerful tool. It allows us to do things like this where we can, um, get better responses and better answers.

And now with knowledge graphs at our disposal, now we can, we can go and we can fight the, um, the evil agents. Actually, it's kind of ironic that the, um, the agents of the Matrix film were, um, the bad guys, but actually they operated and acted a lot like modern agents where we're having LLMs collaborate, pull together, and, um, cooperate on different algorithms.

And even the agents in the film had different personalities and, um, different types, um, kind of like individual agents. So, um, we need to power up and, and get our, um, matrix and graph skills up to level where we can go tackle the agents with new tools like memory retrieval.

So we, we talked a bit about graph retrieval, and, um, graphs are also a great tool and mechanism which you can use to do, um, memory retrieval as well. So we can do search, um, in, in memory, use graph memory retrieval tools.

We have an open source MCP server which does a lot of this for graph retrievals. And now you can query the graph not only for knowledge graphs, but also vectors as we did in the last, um, example. And we can also use graph data science algorithms like, um, community groupings or k-nearest neighbors or different graph algorithms which allow us to get, um, pull some insights out of the relationship and the structure of the graph, pull back relevant information, and then pass this as additional context, um, either for short-term or long-term memory into the agent loop.

Um, we're now, we're feeding the agent with additional information and context from either, um, a short-term memory source about the current conversation or knowledge pulled in, like kind of what we showed in the previous example from a graph or from a long-term structure of memory where we memorize, um, previous conversations, give them temporal information, and then structure those into a memory that can be retrieved from the graph.

And, um, now we're able to use graph memory to capture knowledge in the form of entities and relationships between them where some nodes have the relevant properties such as text details, embeddings, time, and location on top of them.

So this is kind of a, a visual representation of our, of our graph, our memory graph. Some of these properties get vector embedded, and this enables us to do vector-based semantic search. So now we can do semantic search on the graphs via projections into vector space.

But then we can also use algorithms like k-approximate nearest neighbors, community groupings, um, page rank algorithms on top of the graph to answer different types of questions and to kind of bubble up the most relevant results into the context.

This gives us quite a lot of power because we can do both the vector embeddings, but we can also do additional graph algorithms on top of it.

Okay. So now we have our, our superpower with our, our graph. We're, we're able to do amazing things which aren't possible just with, um, vector embeddings and similarity searches, kind of like, like the bullet time in the Matrix films.

Um, this will allow us to do amazing stunts and to evade the, um, um, the agents. But let's give a quick example of how this would actually work in practice. So let's say my question to the LLM was, "Let's update this presentation."

From the last time, I presented with Sid, who's my colleague in India. Um, so we can now search this information in the graph, and there's two relevant people for this,right? So it's, it's me, um, VP of DevRel. It's Sid, who's a community manager.

And the event and the last time we presented it was at an event called GIDS, um, which is an event in Bangalore, an awesome developer conference. So, um, now we have kind of that temporal relationship with the two people and then an event.

And we can add to this the, the memory record at a particular time of the presentation. So now we're pulling back information about this presentation at a specific point in time, and we can pass this in as context to the LLM.

So when we ask it to update the presentation, we both have the context of who presented, where they presented, and the time period in which they presented for the LLM to build additional information on top of it. And this is only possible because we can do this, um, multi-stage query with graphs.

Graphs excel in use cases where you're able to pull in multiple facts which are related, um, but don't get pulled back in a single query. If you can do it in a, in a one-shot or you can get a, a single similarity search, um, standard vector RAG is, is fine for those sorts of use cases.

It's where the relationships are two or more where you get the real value from doing, um, GraphRAG and graph memory. And, um, this allows us to do a whole bunch of different types of graph retrievers. So, um, we can do explicit retrieval queries where we have pre-canned retrieval queries with different entry points and retrieving some context.

So this gives us some great information from the graph, but we can do better by doing text decipher. So fine-tuning the LLM with schema, generating a query for the question. And then we can kind of take this to the next level with agentic traversal where we iteratively navigating over the graph, collecting information until we answer it, and using an appropriate set of tools.

And to show an example of this, um, we're going to use the same knowledge graph which I loaded up again in our demo number two, but this time we're going to query that knowledge graph using Claude Code. So what I've done is I've hooked up Claude Code using the, um, Neo4j MCP cipher, um, MCP server, which is an open source extension.

You can see Neo4j cipher MCP server, which I've already configured with the database settings. And now when we talk to Claude and we ask it a question, it can answer with that additional graph context that it can tell us things.

So I put a few keywords into the MCP server like, um, graph and database, and we can ask it, um, "What do you know about the Jackson vulnerability, uh, based on your graph database?" And so now, in addition to, you know, pulling in information from its standard knowledge sources, it's going to use the Neo4j MCP server and then query it to get additional information.

And you can see that it's doing this multiple-step plans, um, search of the graph. So first it gets back the schema of the graph so it can understand the relationships and how the graph is structured. Now that it understands the schema of the graph, it can go back and it can make a bunch of queries to get information about the particular vulnerability.

So it's firing off a bunch of different cipher queries. Cipher is the, um, graph query language for Neo4j, and most graph databases support it. It's also now a standard. The GQL standard that's ratified by ISO is, um, basically a subset of cipher.

And now that it got back information about the vulnerabilities, it's pulling back some of the text chunks to get additional context which are hanging off of those nodes. And this way you can give us a, a very complete answer with as much information and context as possible from the graph.

And the, the main difference between this approach when compared to the previous one is the previous approach was relatively fast, but you, the level of detail it gave us on the CVE was limited. When we give a, an agent, in this case we're giving the Claude agent the ability to kind of have at it for the graph, do traversals, get information, do more traversals, you can see that it gives us back very detailed information about the vulnerability.

So it figured out the CVE number, the affected vulnerability, the type of attack, the severity, and a technical description of the attack. So it's a lot more detailed than what we got before. And it tells us specifically what versions we need to upgrade to remediate the attack, um, and gives us some advisory information as well about this.

So, um, if we were trying to develop a, um, vulnerability report or something to kind of explain how we should, as an organization, um, address this vulnerability, um, using this sort of agentic multi-step, um, MCP retrieval approach is quite powerful because you can see that it gives us, um, the best possible response since it's able to go back and keep pulling additional information from the knowledge graph that it needs.

Okay. So we've seen a few different ways which we can apply knowledge graphs to solve and improve the context of our AI applications. So now that we know that we need graphs, we need, we need a lot of graphs.

And the best place to find information about graph technology and getting a lot of different use cases for graphs is Graph Academy. It's an entirely free resource to learn about, um, both cipher queries. It has courses on cipher queries.

It has courses on GraphRAG with examples in both Python and TypeScript. Um, we have some more advanced, um, GraphRAG courses coming up as well. So, um, a lot of information which is all free and very hands-on for you to get started and actually build your first application, kind of like the ones which I showed here in the presentation.

Um, now, if we want even more knowledge, kind of a wider span, we can then go to Nodes AI 2026, which is our free online virtual conference. Um, this is following up the amazing Nodes conference we had last week with, um, over 13,000 registrants.

So it was a huge event. And Nodes AI is all about AI for, um, an entire day with AI-focused sessions. The CFP is openright now if you'd like to submit, and it's free to attend and watch all the sessions.

And if we want to really get down and, you know, beat the architect at his own game, then we need a lot of deep research and information. And the best place for that is graphrag.com, which is a community site which we support, um, where it has all the latest research on different GraphRAG approaches, um, how-to guides, and conceptual information about how to implement GraphRAG, and just a general resource which will help you to up-level your ability to apply graphs to different problem domains, um, with a whole bunch of the cutting-edge latest research which is coming out.

So, um, exciting stuff. Thank you very much for joining me for the session today, and I hope you learned a little bit more about how you can use graphs to connect the dots and improve your context engineering for your AI applications.