Connecting the Dots with Context Graphs — Stephen Chin, Neo4j

Hello, and welcome everybody to "Connecting the Dots" with Context Graphs. My name is Stephen Chin, I run the Developer Relations team at Neo4j, and you are in store for the power hour of context and graphs and all of this technology.

So, um, I'm the first speaker; we have some other amazing talks after me. So I hope you enjoy all the great content, which you're going to see over the next, um, hour or so. So, what I'm going to talk about is a bit about how we've all been feeling with the AI revolution, where we are trapped as engineers.

We are using AI coding tools, or maybe they're using us, where our work is being reviewed. Who here has their work reviewed by an agent when they check in their PRs? Yes. All of you. So we are, we're stuck in this limbo where we have amazing tools, we have amazing capabilities, but rather than us controlling them, they are controlling us.

And we would like to get to a state where we're in control of this. So we have to decide, is it going to be the, the blue pill where we're stuck inside of this mire of disparate knowledge, stuck in, in Slack discussions and little customer threads and different enterprise systems, which are all segregated and siloed?

And when we ask the agents to make critical business decisions, or our applications to make critical business decisions with, with all this spread, it can't possibly give good answers because it doesn't have the context. Or do we want to dive in and, and embrace the red pill, escape from the matrix, and have a system of reasoning where we actually have all these systems connected, all of our different enterprise data sources, previous decision traces, the reasoning tool calls of the tools to give us a more consolidated view of our enterprise stack and escape from the matrix?

So who, who's, who's going to, who's in the escape club? Who, who wants to break out? Okay, hopefully if you're in the room, you're, you're with me. Um, and guess who else is with us?

Gartner has now officially made Context Graphs as part of the AI hype cycle. So we have been officially recognized by the, um, the analysts of the world. They, they also realize that we're all stuck in this, in this mire.

Um, Foundation Capital actually started this thread with their $3 trillion startup opportunity post about how Context Graphs are going to move forward the industry and dramatically change how we build applications. And, um, what I'll do is I'll, I'll show some demos and I'll talk about how we can move from being stuck in this matrix, stuck in this world, and then become the superheroes of our organization and actually build the capability of the systems using technologies like Knowledge Graphs.

So Knowledge Graphs are a very powerful tool for us to aggregate all this information, create the connections, create the relationships. And at a fundamental level, they, they hold nodes, which are, are people or, or things or companies or relationships.

Um, you have relationships between nodes where, um, in this case, um, you know, Dan, those are properties, um, lives with Ann. They, he drives her car apparently, so we know who, who wears the pants in this relationship. And we have some embeddings on top of the car, so we're embedding vector information in it so we can also do similarity searches and kind of combine the best of both worlds with building information, but then also combining it with LLMs.

So when we take what LLMs are really good at, this language, this reasoning, this creativity, when we combine that with what Knowledge Graphs are really effective at, so knowledge, context, and enrichments, then we can start doing things with our data like, um, storing all these relationships together, visualizing them so we can get to the data which matters, finding hidden patterns, and then analyzing this and getting more insights, which will help power the Context Graph demonstrations, which I'm going to show you all.

So here's a, a simple example of how graphs power retrieval, because I think it's, it's good to understand what the difference is between a baseline LLM. So this is a, um, healthcare case. What was the care plan associated with Andrea Jenkins' emphysema?

And when you ask the LLM, it has broad knowledge. It, it understands a lot of information. It knows what emphysema is. It knows what standard practice is. So it gives a, a very generic answer: preventing damage to the lungs, yada, yada.

Now, when we give it a RAG system, so we give it a vector database, now it has more context. It knows a bit about the, the patient and their information, and it tells you, maybe recommend some activities like respiratory therapy, deep breathing, coughing exercises.

So this is pretty generic medical advice. Now, where we want to get to is grounded complete information where we're pulling in who's the patient, what was the previous diagnosis, what operations have they had. And you can see here that it, it's specifically recommending medication management, smoking cessation counseling, pulmonary rehabilitation exercises.

So that clearly the parent here has a, the patient here has a history of, of smoking, has had an operation. So like there is, there is certain things which are background information that was lost in the similarity search.

And if our agents have this information, then like the matrix, now we've loaded all up this information and we can, you know, we're like Neo. We can do kung fu, we can shoot bullets, we can do all this amazing stuff with theright memory structure in place.

So this is kind of the second layer. So now we have the, the grounding with, with Graph RAG and retrieval, and we can pull things out of Knowledge Graphs. But we need to now store the, the memories, the short-term memory, the long-term memory, and the reasoning memory so that we have our complete history of what's happened that we can build on top of this.

Um, so I think you all know that short-term memory is things which are happening in the current pipeline with agents, the conversation, the, the current state of, um, activities which your, your agent architecture is working on. So this can all be persisted in the Knowledge Graph, and it gives important information in the execution pipeline.

Long-term memory is really important and needs to be organized well because there's so much of it. So you have to figure out how to aggregate and pull this information in, have a good domain model for representing the, the different business processes, the entities, the, um, folks who are part of your application or part of your domain.

And then you can actually store the information which your agents are working on over longer, um, tasks and procedures and across multiple different, um, user or customer interactions to give that history and context for which your application is done.

And then finally, reasoning traces give us the ability to understand why decisions were made and how they're done. So typically what we get from LLMs is we, we get the result,right? They'll, they'll tell us, well, this is what I recommend.

This is, you know, advise this. But to get to that result, there's, there's thinking, there's reasoning which happens behind the scenes. And we'd like to make it repeatable where that information, it's learning from the experience of the previous traces.

It has that decision provenance for if there were previous decisions, you're going to draw on that knowledge and we'll use that to come up with better future decisions. And this also gives us a great hook in for compliance and debugging.

One of the great things about Knowledge Graphs is they're great for tooling. They're great for LLMs. LLMs can build Cipher, which is the query language for Knowledge Graphs. They can create Knowledge Graphs and create structure out of, um, unstructured documents.

But it's also how we would represent things as humans. It's how we would draw things on a whiteboard. It's how we would show things. And with these, now we are loading up the memory into a structure where we can actually start to do interesting things with our memory.

So graphs are a, a great use case for memory because relationships are first class within Knowledge Graphs. They're part of the structure. It's not like you have to join a bunch of tables together. It's a natural progression. They're highly performant for multi-hop traversal.

Um, a lot of the Graph RAG research papers talk about this specifically as a major advantage to Graph RAG and Graph AI architectures where they can navigate more complex structures at a very performant speed. Um, using graph embeddings like FastRP, we can also do vector lookups, which is a great way to get a starting point or hook into the graph where we navigate using algorithms like the Lovaine algorithm for community grouping.

And then we get explainable decisions. We have more cross-knowledge and we're building ACID-compliant, um, solutions with things like the Neo4j agent memory package. So this is an open-source package which we built on top of Neo4j. We have an open GitHub repo.

We encourage other folks to contribute for it. And it brings these three concepts together: short-term memory, long-term memory, reasoning into a Context Graph structure. And the first demonstration I'm going to show is an example of how you can build a Knowledge Graph which ties your short-term, your long-term memory, and your reasoning memory together to answer questions from Lenny's podcast.

Who's, who's a fan of Lenny's podcast in the room?

Me.

Oh, okay. A bunch of folks. Um, it's a great resource and, um, but it's hard, like podcasts are hard. They're very dense. There's a lot of connected information and topics. And we'd like to be able to extract that and then understand more of the context and things which are happening with the help of AI.

So, um, we built a little demo, Lenny Memory Podcast, again, an open-source project. Um, what it lets you do is it has all the podcasts loaded up. Um, for those of you who don't know, Lenny talks a lot about different AI topics, about product management.

And, um, one of the things we can do is we provide the AI with different tools for accessing the memory. This is all written on top of the Neo4j agent memory APIs. And then, for example, we're pulling back locations in the episode, and it's using that to design and build a graph and, or in this case, a, a map and show us all the different locations of things mentioned in the film by aggregating all that context.

And because we have it in a graph format, it's not just pulling out some similar locations and getting like part of the data. We get a holistic view of the entire data set which can be navigated and queried dynamically.

So now we've shown what we can do with graphs, what we can do with memory. But what we're all here for is, is Context Graphs,right? How can we take this and actually apply this to solve those cross-domain business problems where it's very hard to get the information, it's very hard to quantify why decisions are made.

And, um, Context Graphs are really powerful for this because unlike a traditional audit log, they're capturing the, the why, the decision traces that happened while you're evaluating your models. It organizes these by entities and relationships, and then it's pulling all the knowledge from, from different sources.

So rather than having conversations hidden in Slack or emails or other informal, um, conversations, now your app becomes a central point where they can look up previous decisions, they can get that advice, and then they can add that recommendation back to the reasoning traces for, um, future lookups.

Broadly, the architecture is you're, you're searching, you're using your Context Graph retrieval tools from your agentic architecture. It's using a combination of Knowledge Graphs, vector search, and, um, data science algorithms. Then the, when you go through the agent loop, it's then pushing that back into the context memory, which gets added back into the graph.

And subsequent queries are then pulling this back as part of your reasoning traces and your output to solve specific domain problems. So what I'm going to show here as an example is a financial services application. Um, for this, we're going to have entities of different people and organizations, um, different events for decisions, transactions, and approvals which happened during the workflow of the application.

And then the context of why, what policies were applied, what risk factors are there, what was the employee reasoning behind giving a certain recommendation. And the architecture, again, it's an open-source project. You can try this out on, we have a hosted version of this, and you can try it out with the GitHub project and run it locally.

But it's pulling in from a variety of different data sources. So we've hooked it up to a support ticket system, a CRM, and an internal business data system with 10 different MCP tools that it has access to. And then we've used a Claude agent to create OpenAI embeddings and then populated Neo4j Context Graph with a lot of this information so that it has a, a domain, domain graph and a reasoning graph which it can look into.

And then finally, it's exposed with a user interface, which is a simple Next.js application that gives us a front end like what we'd want a, an end user, a consumer to, to use for this particular use case.

And for this application, what it does is it presents to you a prompt where you can ask it a bunch of questions. We're going to ask it about Jessica Norris and see whether she should get an approval. And it's going back to the graph, and it's querying both information about her history.

So it, it knows what her bank account is. It knows that she has some related margin trades. You can see some of the Cipher queries there that were queried through the model. And you can also see the Knowledge Graph that we're traversing and populating.

So this is why Knowledge Graphs make things explainable and auditable because now we see exactly the information which is being populated and used. You can see there was a previous rejection. And these are the sort of things which get, which get lost in, in disparate systems and tooling where we don't bring all this information together in a, in a queryable, in a understandable form where we can build and pull out that knowledge.

Now, unfortunately for Jessica, the, the AI model recommends not giving her the loan, but it gives us the reasons, the risk factors. It gives us previous decisions which should influence this. And, um, fraud detection patterns about why this may be a big risk for our organization.

And as a, an agent who's, or a, a user, a human who's using the system to make decisions, this is the sort of information you need to actually make a decision you can stand behind and you can justify to your organization.

And then us as developers, now we can justify why our agentic applications are actually solving real business problems, providing grounded information that our users can rely on, and are taking advantage of the latest techniques with Context Graphs, which, um, as we know, Gartner approves of.

Allright. So let me leave you with some resources that you can use to learn more. Um, I run the DevRel team. One of our big pushes is free education. So we just want to help people to understand how to, how to use graphs, how to use Context Graphs, how to use, um, AI.

We have a new Context Graph course that we just released on Graph Academy. Um, also, it makes it really easy to get started because we, in the background, we spin up a free Aura instance. So you just have a graph database to play with for free.

You can try a bunch of these techniques out before you even try it in, you know, your own production instance, your own enterprise instance.

So I hope you guys enjoyed the talk and learned a little bit about what the possibility with Context Graphs is. The next set of presenters, my, my colleagues, um, Zaid and ABK, are going to dig a little bit more into agentic use cases of Context Graphs.

And, you know, please come chat with us either after the talk or at the Neo4j booth. We're, we're happy to have conversations, kind of dig more into demos, dig more into your use cases, and help all of us to escape the matrix.

So thank you very much.