Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications: Zach Blumenthal

So thank you, everyone. We have a very full room today, so we'll get started with the repo here in a second. There is a QR code here, and then in a couple other slides that all link to the same place.

We'll be using a Python notebook, and we'll be doing GraphRAG with some real data using Neo4j. So it'll be myself, I'm Zach, and then we have Andreas, or ABK in the back, who will be helping. And then we have Michael Hunger somewhere else in the room who can help answer questions.

And Tomasz as well, hello Tomasz, back there in the beige, who can help answer questions as well as we go through. So this is an interactive session. We have only 2 hours, so we're asking to save the big questions to the end.

But basically, we'll be popping back and forth between the slides, and then between hands-on work inside of a notebook. So if you do get stuck with anything, raise your hand. We'll come over and try to help you through, and try to stick to our schedule as well with around 10 to 15 minutes per each section.

And there's about 5 sections for us to walk through. So before we start, there's two things we want to do, if you haven't done them already. The first thing is, we want to create a blank Neo4j graph data science sandbox.

This is going to be at sandbox.neoforgj.com. So basically what you're going to do, if I go ahead and click this link, you might have to create an account really quickly on this page. And then you're going to click for data science here, and then it's going to be this, if you can see,

this blank graph data science thing here. So you're going to click on that, blank sandbox graph data science, and then that will create an instance for you. And it should come up with a username and password that you're going to use inside of the notebook.

So this will be the database that we'll use throughout the course. The next thing that you're going to do is go to the notebook. So the notebook is going to be in this link, and also, by the way, this QR code.

So I'll go back to that in just a second if you missed it. But before that, I just want to show you this GitHub repository, and it's GenAI Workshop is the notebook. So what you're going to want to do is you're going to want to go to this notebook, and then you're going to want to click open in Colab.

This will require a Google account. If you don't have a Google account, you can use VS Code, or you can run it locally, that's fine. But then when you go in, if you are using Google Colab, the internet's a little bit slow, so it might take a while for it to load here.

Hopefully, once everyone gets settled, it will go a little bit faster. But basically, what will happen is you just want to go to the top menu, and you're basically going to go like File, and you're going to save a copy and drive, just so that you have the ability to edit it.

So you go File, and then you say Save a copy and drive, and then it will create another copy for you.

So I'm going to go back to the slide, and I'll give everyone a little while to get that all set up. Actually, too, while we're waiting on that.

15 people can still sit in the front if you don't want to stand in the back. So feel free.

Yes.

It might be a good idea for you to start running the notebook cells. So especially the first cell, the pip install, just because that might take a little while. When you just start, that might it's probably the longest running cell in the whole notebook, it's just loading all the requirements for today.

So to say again, create your sandbox, blank sandbox at blank sandbox GDS, graph data science, and then you're going to open that notebook, create a copy of it, or have it locally, and then start running that pip install cellright here.

So what happens? I did the blank sandbox instead of the blank data science sandbox.

I did the blank sandbox instead of the blank data science sandbox.

That one will maybe work, but what I'd recommend doing is just terminating that instance. So if you go to the if you go to the screen once it's there, just press Terminate. There's like a garbage can icon. And then go and create a graph data science sandbox.

Perfect, thank you.

Yeah, blank. It's blank sandbox dash graph data science is the one we want.

Thank you.

So the main difference is actually that the graph data science sandbox has a little bit more memory. So as you're doing some more involved operations, it's just faster to complete. So the regular blank one should still work, but it will take longer.

So the graph data science one will just be faster. So it runs with a little bit more memory.

Yes, it says blank, so it's quick.

Yeah, I know. We need to work on that. It's a little bit tricky.

Allrighty, so I'll leave this here. Is there can I get a show of hands of how many people have the notebook set up? OK, so we're pretty good.

Is there anyone who still needs to stay on this PR code?

No.

No? OK. We'll go ahead and move on. So I'm going to do like my general spiel that I normally do. I think for a lot of us, this is going to be review. We're going to be talking a lot about retrieval augmented generation today.

And for those who are unfamiliar, retrieval augmented generation is just when you have this step in the middle when you're communicating with a large language model, where instead of sort of like in a ChatGPT-like way, where you're just sending a question and getting a response back, you're going to have that LLM application go and pull relevant data through some query that it can then use to reduce hallucinations, provide more domain-specific context, enable better traceability, et cetera.

And when we talk about graph-powered retrieval augmented generation, and we'll use the word GraphRAG kind of throughout this course, is we're talking about the combination of three things that we're going to look at today. So that's going to be vector search, which I'm sure a lot of you are already fairly familiar with.

Knowledge graph traversals, which is going to be traversing over structured data in addition to the vector search. And then we're going to do some cool things with graph data science. So we're actually going to create a new type of embedding, a graph embedding, and we're going to use that vector to do some very interesting things for recommendations with structured data.

And today, so everyone does chatbots, which chatbots are wonderful. I wanted to do something a little bit different today. So we're actually going to do an example of using a large language model to construct an email, sort of like a recommendation type of email.

And this is going to be for clothing. So imagine someone's looking for something like a halter necktop,right? How do you choose what to buy? Well, you need a list of things that are available in your online store. You need to match that specifically for that specific customer,right?

Which different customers might have different preferences. And then you might want to have other recommendations to pair with that item, like maybe a pair of pants, or a hat, or other accessories, and things of that nature.

And so what we're going to do today is sort of build an AI fashion assistant, so to speak, to help scale this. So we're going to use real-world data from a Kaggle competition from H&M, and we're going to do some search and retrieval.

We're going to add some context for personalization, a small recommendation engine, and we're going to wrap that into sort of an LLM chain inside of LangChain to show you how to generate that response. And the idea behind this,right, is again, we're combining sort of this knowledge graph, which provides this context and this enrichment, with the large language model, giving that reasoning and that creativity, and sort of the we're giving some liberty to the large language model to kind of take what's in the knowledge graph based on the customer's purchase behavior and other textual data and vectors, and kind of say, use your creativity to put something together for this customer, basically.

And while we're doing sort of a fashion recommendation thing here, this same process could work for different types of support agents, or a lot of different types of internal search and customer experience use cases. And this is what we're going to build today.

Sharing the deck?

What's that?

Really be sharing the deck.

Deck is in the GitHub repo. I need to it's not the most it's not exactly the one that I have here, but I'll update it tonight.

In the app?

PDF.

The workshop slides?

The PDF of the workshop slides is in the repo. Yep, yep, yep. So this is what we're going to build today. It's a little Gradio app, where basically we're going to enter in some customer information, their ID, a time of year, and some of the things they're interested in.

And we're going to generate an email that provides various recommendations to that customer. So imagine you're trying to create targeted marketing content, or whatever it may be. This is sort of an example of how to do that. And these are the specific technologies we'll be using.

So we're going to use OpenAI for the language model. We could, of course, substitute that out with Vertex AI, or Gemini, or whatever we wanted to there. We have LangChain that we're using to orchestrate everything. So that's going to be creating the vector stores and also the large language model chain.

We're using Colab, obviously, and then Gradio just to kind of show you an example of a very simple UI that you can put on top of that, just for demonstration purposes.

And this is going to be how we're splitting the course up. So basically, there's this journey that we're going to follow. We're going to start with building our graph. That's going to be built inside of that sandbox instance that you just made.

After that, we're going to be showing you how to do some very simple vector search. Then we're going to explore how to integrate knowledge graph traversal patterns into that to better personalize the retrieval data, so not just based on sort of text similarity, but also on similarity based on co-purchasing behavior.

And then when we get to recommendations, we're actually going to use an unsupervised machine learning model to generate graph embeddings. And we're going to use k nearest neighbor there to locate other recommendations that we can make inside of the graph.

And then we're going to put that all together inside of an LLM chain to show you how you can use that to create a better, more curated response with the language model.

Allrighty, so that was a lot. I'll go ahead and get started with graph building. Again, the way this will work is I'll spend a little bit of time in slides, just talking about what we're going to do. Then we're going to hop over to the notebook, and we're going to run through those cells.

You might already be running through those cells, because I know a lot of us, we it's going to be pretty easy to just sort of click through. There's not actual a lot of hands-on coding you need to do, but we're going to walk through that anyway.

So this is the data model that we're going to use. A reminder,right, that we're working inside of a graph database, which means that everything that we're working with is natively represented as nodes and relationships inside of a graph.

So we have these entities, we have customers and articles, and then products. So basically, what this graph is telling us is a sample of customers that exists, different purchases that they made of articles of clothing. So think of these like the halter tops, the jeans, the boots, all the accessories.

A variant of so an article is a variant of a product in the sense that you can have a certain style of jeans, but they can have different color patterns, different sizes, et cetera. So the product is sort of like the product, and then the article is like the variant of that product.

We also have each department that the article is sold in. We're not going to use that department node very much, though, in this specific workshop.

And of course, all of these things can have properties associated with them. So we have, for example, the customer ID to uniquely identify a customer. For articles, we have product we have color and different design patterns. And then for the product, we have sort of the description of the product, the garment group that is in, the type of the product, and the name of the product, et cetera.

So again, I think we've all done this step hopefully by now. So I will go ahead and

enter into our notebook. So hopefully everyone can see this. This was the first cell that we wanted to run before, just because it takes a while. If you haven't run the first cell for pip install, please go ahead and do that now.

Once you've run that, go ahead and just kind of keep going down here. Run the import. This just gets us all the libraries we need. And now here's the part where we're going to have to inject some things here.

So when you made that sandbox instance, I'll actually go to my I have to log out and go to a different account here.

You should get connection details.

So if I go and I just click this, and I go to connection details so please don't use mine, so that we're not fighting on the same database. But copy the bolt URL here, and you're going to go back into your workshop code, and you're going to put that here for your Neo4j URI is going to be that bolt connection.

The password as well

is going to be is going to be this oneright here, the password. And then you're going to keep the username as Neo4j, so you don't have to change either of these two. We're going to keep the LLM as GPT-4o.

And then for the OpenAI key, if you have your own, please do use that. If you don't, if you click on this link, it'll take you to a workshop key that we'll keep live for a little bit that you can use to run through the workshop.

And so basically, you're just going to copy it in, kind of like I've done here, and you're going to give that a run. And then this next cell is just if you wanted to do it the proper way and have a separate environments file, which you can skip, or you can do that if you want.

But that is not necessary. So

once you've run that, we get down to the knowledge graph building section. And we've already talked about the graph we're going to import. And basically, what you're going to do is you're going to connect to Neo4j. So this is our graph data science client.

We have multiple different drivers and different wrappers around those drivers to connect to the database. I'm using graph data science for this just to simplify things. It's basically you're just going to hand it the URI, the username, and the password, and that'll get you connected.

And there's some warnings there that we're going to get rid of. And then if you can run this cell successfullyright after it, this GDS debug, that means you're connected and you're good to go. So you're connected to a blank database.

Once that is run, there's one cell here that I have for pretty much loading 90% of the data. So you're going to go ahead and run this. What this is doing is I've gone ahead and there's another notebook inside of this repository that's prepared these CSVs.

So there's five CSVs here. There's department, product, article, customer, and transaction. And so most of these are self-explanatory. The department is the department node, the product is the product node, article is the article node, et cetera, customer the customer node.

And then transactions are all the purchase transactions between customers and articles. So if you were to dig into this code and because we only have two hours, I mean, we can spend all our time talking about this, but I decided not to, so we can get to more of the vector stuff.

This is going to create uniqueness constraints, which are going to be useful for querying later in Neo4j, so it's just a general best practice. And then if I go there's basically two functions here, one for loading the nodes and then the next for the relationships.

If you were doing this sort of in a non-automated setting, this is kind of what the cipher looks like. So just like

so just like in SQL, how you or for RDBMS, you have SQL. For graphs and for Neo4j specifically, you have cipher. So what this code is doing that I have above is basically for loading the different nodes. It's generating this cipher.

So you see it says, use this cipher query, unwind, merge, set. So that's the actual code behind the scenes,right, the cipher code that's being run to load this data. So if you are interested in how to load more data into Neo4j, you can take a look at that.

And we have other Graph Academy resources as well, if you wanted to dig into that piece in more detail around just how to model data and how to load data into a graph.

And then we'll stop there. And I think we're at 15 minutes now, so we can probably do, I'd say, five to 10 minutes, and we can wait for people to catch up. As a quick show of hands, how many people have made it through the loading?

Awesome. OK, that's really good. And then Michael and Tomasz.

I think it's the

load the CSV files. What is it better we could check if it got loaded somewhere?

Oh, so if you have yeah, if you go to this tab here, shoot. So instead of this

yeah.

So

actually, so you've got the credentials as well?

Yeah.

OK. So instead of single sign-on, there's username, password. So add the password over there.

You see this is where leaving it.

So this is like the built-in cipher query kind of thing. So when it connects, so one thing that's convenient is if you click on this database icon there,

it should end up looking for

it looks like it might be blank still. Oh, it just slowed down. Oh, OK. So end up updating. So for every node that has a label, it'll show the label that exists and the relationship types and obviously all the properties.

If you click on any of those, actually, you can go to the top, click on anything. It'll basically run a query. This is a cipher match query. So match for records that have a label article and a return in 25 hours.

Oh, it's now in the.

Yep. Cool.

Yeah, you don't have this is just.

You can have Josh Michael and

Josh use Graph.

And then you have the automation for that.

Well, what's happening is this setting is wrong. So

I can use setting is wrong.

So if you wanted to move your vector to.

Does anybody else need some help that I can tell Tomas to help with?

Everyone's good?

So there might still be some people having network connectivity issues. We've seen a couple of different variations on that. We'll try to help figure those out and get that sorted. I think we can maybe.

OK.

Yep. Awesome.

Can we just unplug and replug?

Oh, it's CrowdStrike. It's the stupid.

What's happening?

What? By the way, I have to say you're all pretty good for such a big room, everyone being on top of things. I'm really impressed. So well done, everyone.

It's the software they installed on the yes, fun security software on my laptops, preventing me from connecting USB devices. So hopefully it's there. If it disappears again, let me know. One thing that I wanted to point out, someone had a good question about what we were loading.

So there's a couple of things that we do kind of at the end of this block. One of them is I'm going to create a fake sort of dummy URL, because it wasn't included in the data. So we'll use that to help.

The LLM will basically provide a link to the product, which won't be a real link, but you can imagine substituting a real link into the database. So we're just creating a new URL property for that. And then this thing here is creating it's basically taking other properties that were in the database.

So when we looked at the product and we'll see what these fields actually look like in a second. But there's a product name, a type, a group, its description, all this stuff. And this statement here is just creating a new text property to kind of squish all of that together in one place.

And we're going to use that for our upcoming vector search. So we're going to index that. So that's why we're creating this one text property. So just wanted to mention that really quickly before we move on to our text section.

So now we're going to start talking about vector search, and we're going to do some initial search and retrieval. So when we think about what embeddings are, and I'm assuming a lot of people in this room are already quite familiar,right?

But an embedding is basically a type of data compression that takes some messy data, like text, images, or audio, and it puts it into a compact format that's machine learning friendly,right? And so these are going to be most often numeric vectors or arrays with hundreds or thousands of elements.

And they preserve information such that similar items have proportionately similar embedding vectors. And this similarity is going to be measured mathematically using something like cosine or Euclidean, et cetera. And another thing that I want to mention is we will and maybe I'll save it a little bit for the end.

But just like you can embed audio, text, and videos, you can also embed graph structures. And so when you have co-purchase relationships in a graph, or you're trying to do entity linking in a graph or anything like that, that can become very valuable for then using downstream in vector search.

Are you going to come to the models?

Yes. Yep, we will walk through that. So similarity for something like a text embedding is referring to semantic similarity, so the meaning behind a text sequence. For a graph embedding,right, similarity can mean similar in position or structure inside of a graph, and that can have semantic meaning too.

Andright now what we'll be using vector search for is just on text embeddings. And later when we get to recommendations is when we're going to use graph embeddings, and specifically node embeddings that we're going to use there inside of vector search.

So talking a little bit about just what's inside of Neo4j, so Neo4j has multiple indices. And you can use these in combination with each other. So we have things like range indices, which is just a general kind of index that can do range comparisons and equality comparisons on numbers and dates and things like that.

Point indices, which are geospatial or I guess point space indices,right? Text indices and full text indices. So we have a full text index that's backed by Lucene, for example, that is tokenization analyzers. And today what we're going to be focusing on primarily are vector indices.

And our vector index uses HNSW hierarchical navigable small world. Yes, there we go. For approximate nearest neighbor search.

And so a lot of this might be review for the folks in this room,right? But the idea behind this is that if we have some product, some text about a product, and let's say a search prompt on theright here for a halter neck top, we can start to identify things that might be more similar to that using that embedding vector,right, and sort of the semantic similarity inside of that space.

And that semantic similarity can be measured by either in Euclidean distance,right, or cosine distance. And like I said before, we're using HNSW to perform approximate nearest neighbor search, or we will be here in just a second.

So we're going to start

let me just go ahead and jump into the notebooks. I think it'll be easier to do live. So.

Question for you.

Yep.

For those who are not familiar with Neo4j, how to visualize a node from we have created something, but.

Yes. Soright when we're done with the vector indexing, I will go into Browser, which is one of our tools for visualization. And we can take a look at that. But basically, if you go into this page here, I can go to Open, and then it will take me basically into the Browser tool.

This is going to take a while because of the internet lag. And we can visualize in there. If you're willing to bear with me in creating a vector index first, because I want to just go over that very quickly, and then we can go into the visualization portion.

But yes, we will very soon be visualizing the data. What I wanted to do first, though, was just kind of get our index set up and just show you how to do some searching.

So basically what we do with our data, we're specifically going to match the product node. So if you remember, we have that product node, which is a variant of an article. And the way that this is going to work, you can call embedding models externally too.

But what we're going to do here is basically match the product where it has a detailed description. So basically, some of these products don't have complete information. So we're just going to take the ones that have complete information.

And then all this thing is doing is it's using this code, GenAI vector encode batch, to, on the server side, create embeddings and then set that embedding property on a property called text embedding. We're using OpenAI for this.

There's a few other models that we can use here, or a few other API providers, rather. By default, this is using text embedding ADA, or ADA. But you could also use Vertex AI here, as well as Azure OpenAI or AWS Bedrock.

And then you can also just generate your own embeddings externally and adjust them for any other type of model. So if you go ahead and give this a run, it will start creating your embeddings. And then the cell after that is going to create your vector index.

So this isn't creating an index. All this is doing is basically populating in batch a text embedding property. And then after you create that, this cell down here, if you run that, is going to create

your vector index. We're going to set it ahead of time to use cosine as a similarity function. And then this next call is just to wait for it to come online, since once you set it, it just takes a little bit of while for it to index all the data and become available for querying.

Yes.

Can you just clarify the difference between generating the embeddings, which is the semantic representation of the text,right, versus creating the vector index?

Right. So when you're doing the embedding generation, all you're doing is you're taking this text field that I talked about earlier. You are sending it to OpenAI to say, hey, embed this. You take that back, and then you set that as a property.

So all this cell is doing is basically making these vector array properties inside of the node. And then the vector index actually sets the index to make that searchable.

Once you've gotten that done, we can do a couple of examples of search. And again, this is sort of just using Cypher here, and you can do this externally as well. But in this cell, I'm taking a search prompt that is very simple, just denim jeans.

And then I'm passing it to this function, which is running Cypher. All this is doing, actually, is just encoding. So basically, it's just creating the query vector,right? So if I were to just encode denim jeans, this is the query vector that that creates.

And then if you take that query and you go to the next cell here, what this will do is it will do that same encoding. So this is the first part of the query that we just ran, which just creates a query vector.

And then inside of Cypher, I say, call database index vector query nodes. And I say, hit this index that we just created. Give me the top 10 with that query vector. And yield the node as product and the score.

And then give me some fields like the product text and the product code, et cetera. And so when you run that, it should hopefully give you some results that look something like this. And this is returning different products.

So you see all of these are denim jeans or denim products, or they should at least be jeans and/or denim,right? I think all jeans are technically denim, maybe not. But you see here we have the product descriptions and the product text that it brought back.

So you can see all the descriptions. And that's basic vector search inside of Neo4j.

Do we let you know if we have a problem with that one?

Yeah.

Sure.

Yes.

Do you build the HNSW on every node within the graph?

Do we build it on every node? Well, we build the index, and then every node has a property. So it's built on top of all those properties,right?

Is it just one HNSW or the entire graph?

Maybe I don't understand. It's one index.

HNSW is its own graph,right?

Yes.

So our graph representation is not the HNSW graph representation. Right. So I'm just trying to understand the scale. You build an HNSW, and now we're talking graph. HNSW, its own graph for every node in your graph, or is it one HNSW for the entire graph?

Wouldn't it be one HNSW for every index?

Yeah.

Yeah.

I'm not familiar with what are you talking about, a Lucene index?

We are using a Lucene index on the back end. Yeah. Yeah, this is a Lucene implementation.

Other questions?

Yes.

What if I already have embeddings? Would it be easier to just have a load code?

Yeah. If you do, I have.

If they match the size of the index.

Yes. Well, yes. If you already have embeddings so I commented this out. I haven't run this block for a while. But basically, this is just showing you how you can use

OpenAI embeddings here. I'm using LangChain in particular. But this is a workflow where you have embeddings external,right? And then you can import them into Neo4j.

Thank you. Same question for vector database. Do you have other systems where you have a vector database? Would it be easy to call the vector database and get an answer, or does the vector database need to be instanced, like mapping it?

So there's multiple different ways to do it. If you have your vector index inside of Neo4j, that can make certain things more efficient. But there's certain situations for scale where using something like LangChain or another orchestration framework, for example, you can query your vector data inside of your chosen vector database and then do your graph traversal stuff in Neo4j,right?

Yes.

Why do I need a graph database if I'm using vector search?

We're going to get to that in a second.

What was his question?

It's why do you need a graph database if you're already using vector search. And so what we're going to explore today is there's a couple of different things. One of them is using structured data or data that's been structured inside the graph to better sort of personalize or make the responses more relevant for a large language model for search.

And then we're also going to do some things with graph embeddings, which is a different type of embedding that we can create inside of this database.

Is there a possibility to do that?

Yes. We do that here. There's an example of that. We'll do KNN, and we'll show you how to draw relationships for that. Yes.

I was going to ask just a question just to wrap up that. Since it's structured data, what would be the advantage of using Neo4j or something else?

So it's a good question, and we should definitely have a longer conversation afterward. But there's a few different advantages. We have index-free adjacency, and we have very performant, like multi-hop patterns. So you'll see some of the patterns in here are very complicated, and it's easier to do inside of a graph, first for just writing the code to do it, but then also the performance on it will be better.

And there's a very flexible schema, so it's very robust in terms of being able to change and pivot things around very quickly.

That's not going to be very expensive.

Yep.

I have one more. With the HNSW, do you do scalar quantization?

Not yet. Not yet.

Allrighty. So I'm going to move on a little bit here. This next part of the vector search portion, I'm going to just integrate LangChain into the mix. So basically, the reason I'm going to start using LangChain here is because when we get to the LLM portion, it'll just be a natural fit where we'll create a chain for this.

But all we're doing here is we're going to create a vector store using Neo4j vector. So this is part of the LangChain vector stores. Again, instantiate an OpenAI embedding model. We're just going to use the default with 1536 as the default size.

And then you see we can create a LangChain vector store from index with that embedding model and our same credentials and the index name that we created earlier. And when we do that, basically what this will allow us to do is start running from our search prompt, which is denim jeans.

We're going to be able to run the results through this vector store here. So this is what we're going to be using throughout the rest of the notebook. Instead of just using our Cypher or query language directly, we're going to be using LangChain, which gets this done under the hood for us.

So that's that. And you can see it brings back the different documents with the product descriptions and the representative URL and all the metadata.

And if you want it to be in a pretty data frame so it's easier to see, you can run that to kind of get the view we had before. And this should match what we were doing earlier.

And of course, you can try yourself with your own prompt. So if you were to type in something else here, you can see it'll bring back a result that's relevant to that. So I typed in oversized sweaters, and it will bring back different oversized sweaters.

Allrighty. So how many people were able to get through the vector search section? If you just raise your hands. OK. Good. Very fast class. OK. Anyone having trouble, struggling, need help?

One person over there.

Allrighty.

Can I ask one more question?

Yep.

So if you're using one Lucene, how do you determine how many Lucene indices do you need? Do you put that on your side or does the user have to figure that out? Because each HNSW can only take up around 50 data.

Soright now, I might not fully understand your question. When you basically instantiate an index for one or a set of properties, that can be on a node, or it can be on a relationship,right? So if I make one index to cover the property of text, that's one index.

I can create another index to cover a product description or another name or something,right, if I wanted to. But there's nothing that would so it would be like it's one index at a time,right? Is that getting towards you, or is that totally off?

Let's say we have billions of documents. We're going to vectorize a lot of them, and I'm going to create an index. So you said you're using Lucene,right?

Right.

So Lucene is HNSW compatible. So each Lucene index takes 50 gigs of data. What do you do on your side? I'm just curious. What do you do on your side so it doesn't puke? Because after 50 gigs, there's diminishing returns.

Right.

Are you automatically auto-sharding it? What do you do to make sure?

There are ways to federate the database. You could create different shards based off of metadata values. But that's stuff that we're currently building on and experimenting with. And for very, very large, like if you truly had tens of billions of documents, you can also integrate with your vector store of choice,right?

And then just basically you substitute out the vector retrieval call for that vector database, and then you can use Neo4j for doing a lot of the graph traversals. But it's one of those things where we're continually improving. So we're coming out with new features to handle that sort of stuff.

Allrighty. So let's start with semantic search with graph patterns. So this is going to be basically building on top of vector search where we start to work with structured data. So when we think about Neo4j and semantic search, there's sort of multiple layers.

So the first part is vector similarity search, which we just went over, which is just finding relevant documents. After that, there's graph pattern matching, which is basically saying, hey, can I find entities associated to these documents and the sort of patterns in that connected data?

And then we have the more data science and machine learning part on top of it where we can improve search relevance and do other things by running graph algorithms like embeddings,right? So when we think about semantic search patterns with graph, there's connections that we have based on purchase behavior.

So basically, given that you have one person who purchases relatively the same things as another person in this graph, you can start to make inferences about what they might want to purchase next,right? So this is sort of like if you've ever dealt with collaborative filtering or sort of those types of patterns, this is a very common recommendation pattern that we can do with structured data.

And when we go back to the graph, we'll start exploring the data set now in a little bit more detail with visualizations. So what I want you to do and I don't have these calls saved, but I'll go ahead and copy them over.

Basically, you're going to go back to your browsing window, and you're going to open your blank instance. So I'm going to click Open. And hopefully, that'll work. And if not, I might actually start

hotspotting myself.

Does that part fail?

Yeah.

It took me like a minute to refresh the page.

Yeah.

Let me.

Allrighty.

You're hotspotting?

Yeah. I'm hotspotting.

Allright.

Allright. Thanks.

Are you on the Wi-Fi?

I'm on the I think I'm on the Wi-Fi. Allrighty. So basically, what I did is I just entered into something called Browser from our sandbox. So again, you just go here or I guess I'm back to this window now.

You go here, and you press Open, and then you can either do the SSO sign-on or you can use your username and password. And so once I'm here, I'm going to copy if you go back to the notebook, you can copy these calls.

This is just Cypher that you can put in to help visualize the data that we've ingested. So this will give me my schema,right? You won't have this relationship yet, but what you should have is these four nodes, which let me get rid of this window.

We'll give you an idea of your data model. That should reflect what we have in that picture inside of the notebook. We have our articles, our products, our departments, and our customers.

Excuse me. And then if we go back to our notebook, I'm going to pull a sample of that data just so you can see the graph. So this Cypher statement here is just saying, hey, get me all the products, match all of the articles that are variants of those products, and then all the customers that purchased those articles.

And I'm just going to return all of that, and I'm going to limit it to 150 results. And so if I do that, what's going to happen and again, the Wi-Fi is kind of hurting meright now. I'll get this pretty graph.

So this is a sample of the data that we loaded. So customers are in orange, articles are in blue, and products are in red. So what you'll start to see,right, is we have these customers. Some of them make multiple purchases.

So here's a customer making multiple purchases. Make that a little bit bigger so you can see when I go to Purchase, it should have a transaction date, a price, all that sort of stuff. Here's my customer with the customer ID and some metadata about the customer.

And then here's my article. And my article is a variant of a product. So what we've done so far is if you go down here, we've created this text field for every product. That's what the text description looks like.

And then we've created this text embedding. This is our OpenAI embedding that we have here. And if we zoom out, the graph structure that I was talking about before in slides, we start to see that. So you start to see that you have these customers that are co-purchasing things,right, at different times.

And so we can start leveraging that graph structure. And we can use that graph structure to start informing our search and our LLM how to return better responses. So for example, what I'm going to do is I'm going to feed it a specific customer ID.

So all this does is it basically caches this parameter for this specific customer ID so I can use that in later query calls. So I'm going to go ahead and put that in. And then I am going to take this query here.

And basically, what this query is just going to do is tell me the history of this specific customer, their purchase history. So I'm going to go back here, and I'm going to say, hey, get me the customer with that ID, and then get me all the things that they purchased, and then all of the product categories that those were in, basically, and just return some metadata about that.

So I'm going to go ahead and give that a run. And it will tell me this person purchased the specific product namesright here, the product type that they belong to, et cetera, and the description of those products.

And then the cool part about Graph is I can basically start taking I can pull the latest purchase history, and then I can start making recommendations off of that latest purchase history. So there's lots of more complicated and fancy ways to do this.

But the basic idea here,right, is that if I go with this, I can pull the latest purchases. So similar to what we did before, just say, hey, take this customer, get everything they purchased, get sort of the max date as the latest purchases.

And then I'm going to do this long query here. And basically, what this long match statement is saying is say, hey, take this customer, take their latest purchases,right, look at other customers that purchased the same thing, and then look at what they purchased,right?

That's what it's saying. So it's like,right, if ABK, if I buy the same things as ABK does, I can say,right, which OK, thumbs up,right? OK. And then I can basically say, well, for me, look at what ABK has purchased and a few other people that have similar purchasing behaviors.

And is there anything there that I haven't bought yet? That's basically what this query is doing. So when you run this query, what you'll see is you'll start to get these different sweaters and everything. And this will change for every customer.

And so the idea here is that this structured data that can feed us these you see here, you have these common purchase scores. This is basically scoring the amount of times that

you had common purchases between customers and then sort of the next thing that they buy. So for example,

there's been eight other common purchases between people that have bought this and between something that this particular customer has bought. And so these top

go ahead.

You can imagine this as your peer group,right? So the people that are like you that buy the same stuff or are interested in the same stuff. So that's your peer group. And you just look what else has your peer group bought that you haven't bought yet,right?

So that's kind of what is behind that collaborative filtering.

Yes, exactly. And these are the top-ranked things in the peer group. So this specific sweater, this t-shirt.

Can you just say slowly again what the common purchase score means in plain language?

Yep.

Thanks.

And let me look at it again just to make sure I have itright here. So we're basically counting the number of paths between the number of co-purchased paths. So basically, this customer, when we go out and look at what other people have bought in that peer group, basically, how many times has that item been bought in that peer group?

Does that make sense?

OK. So it's the score attached to an item.

Yes.

It says high score means lots of people bought it.

Yes, exactly.

Thank you.

Yep. Is that clear for everyone?

Sorry. How does it know it's a peer group?

Peer group being that it's sort of the local graph structure. So what I'm doingright is I'm basically

if I look at this graph, I'm basically going like I'm starting at a specific customer, then I'm looking at the article, and then I'm looking at everyone that bought this, and then I'm going out one more hop,right, to see everything they bought.

And then I'm going to count up how many times those purchases appear. And the more purchases there are, the more likely it is,right, that that individual customer is likely to like that product just based on the common purchase behaviors in the graph.

Does that make sense?

So in this case then, the customer that you were just pointing at, the well, yeah, before you scrolled away.

Oh, sorry.

I want to make sure I understand the graph. So you picked that one random customer, and you said you'd go one hop away from

so yeah. So assuming that person, then that blue product, I guess so that's the one they bought. And then the one to your left, I suppose, that would or I guessright, whatever. The closest blue dot, that's another product.

Yep.

Because there are links between

oh my goodness.

Yes. There's lots of products,right? So it's basically going bought this, this person bought that, and then for all of see, and there's a lot of interconnected relationships too. And then yes, for all of that, and then go out one more purchase relationship.

So you're basically going out, and then you're saying, OK, how many times do these other blue nodes show up,right? And the more that they show up inside of that graph, the more shared purchasing behavior you have.

Yeah.

Yeah. So it's basically just counting that.

You could write the query to go out further if you wanted to.

You could.

And you could build on it.

And part of the machine learning piece does that automatically. But yes.

Can I ask a question about time window? So for example, here you are considering the person, no matter if it's one month or one year, that's one transaction versus if I'm considering window,right? One month versus one year versus one way.

Is there a way to make this easier?

Yeah. Yeah. So you notice let me get out of this view. If I go up to the query that I just made here, and maybe I went through this a little bit too fast. But you should see in here, I'm saying I'm just going on max date.

But you could filter the date to say like TDAT, which is the purchase date variable. You could say do that for only the last week or only the last two weeks or whatever filter was relevant for that.

OK.

And so that's basically the idea that we're going to use to provide additional we're actually going to do a graph post-filtering pattern with vector search. So what we're going to do is if you come back to the notebook and you go down and run so here's the query,right?

So basically, what we're doing is we're going to instantiate another vector store. Except here, what we're going to do is we're going to say we're going to put a retrieval query at the end. So what this is going to do is it's going to do vector search like before.

But then after it does the vector search, it's going to try to match for that customer. It's going to create that sort of peer group, that local graph. And then it's going to score, well, how many times does that product show up?

So this match is maybe almost backwards, making it probably a little bit hard to read,right? But it's the same query that we had before where we take the customer ID, and then we go sort of one or I guess two hops.

But really, we're saying, OK, get me that peer group where we go to the article, we look at all the purchases, the common purchases between customers, and then we compute that same score. And there is some multiplication here that I do basically to combine the vector search score with what we're calling the purchase or the co-purchase score here.

But basically, what this does is it now allows me to pass a customer ID into the search result. And then when I do similarity search with that, it will start sorting things by basically this purchase score. So now instead of getting because Denim Jeans is like this really generic thing,right?

So different customers and we'll see when we run the app later. If you plug in different customer IDs, this result is going to be different. It's personalized now to this customer. So basically, with vector search, we're now using this post-filtering pattern in the graph.

And we can also do a pre-filtering pattern. But we're using a filtering pattern,right, to now rescore these things. So now, OK, we're just returning, for example, these different denim items, which are much more relevant for this specific customer.

Yes.

Can you explain the difference between search score and purchase score?

Yes. Let me see what I did here. So search score is basically just the vector score that came back from the cosine similarity. So when you're doing similarity between two vectors,right, for the text description, that's going to be your search score.

Purchase score is going to be what we were showing before, which is showing the popularity inside of the peer group. And then I'm just multiplying them together to create a combined score here. But there's different ways that you can do that.

So are you going to use this as a way of ranking those final results?

Yes. So if you look down here, these results are now ranked differently. And it's actually different from the query we ran before. So we ran the same search prompt with Denim Jeans. But here you can see we get like Rachel HW Denim, we get Jade HW Denim Skinny, and all this sort of stuff.

If you look at what we ran before, if I were to scroll up to the top here,

this one, you'll see that these are different. And actually, this was the wrong one. But this is theright one here. You'll see that these are actually different. These are different. It's a different ordering. There's some similarities here, like this is number three where it was number one below.

But the idea is that you're sort of re-ranking this,right, now that you're personalizing it to this specific customer. And this gets really powerful as you get more and more structured co-purchasing data,right? Yes.

Why are there a bunch of purchase scores zero in the result?

Because there's going to be if you it's an optional match. So if you notice,

the sort of literal reason is that this is an optional match. But the other reason too is that there could be a situation where they're searching for something that isn't bought in their peer group yet. So this is just organized so that even if you didn't catch anything in the graph structure, you would still have a way to return results based on the vector scoring.

So quick start from the vector.

Yeah.

That's probably the best way to get at least some results based on the vector search query. So there was another question. So the basic idea is like whether, for example, if I search Denim, so it brings up all the Denim-related products from the vector search and the number of people who bought in my peer group using the graph,right?

Yes. And it will try to see if anyone else has bought anything of that nature,right? And if they have, then it will help reorder the ranking.

Right.

Yeah. And you could do it the other way around too where basically and you have another app, and I might take it to our booth where you could do the graph traversal first,right? So like if you had a scale problem that was very large, one of the ways we can scale is through basically, hey, I would do the graph search first.

And then after that, I can do the vector search.

So whatever you are getting in this purchase score from your graph database, how is that different from the differentiation of products that you may have? Let's say you have a specific system competing with batch recommendation queries that, let's say, gives you top 10 recommended products queries.

Is that easy to just use those recommendations instead of traversing this graph to get the purchase score?

I think you should repeat the questions. Otherwise, people don't know.

Oh, people can't hear. So the question was, why can't you use an external batch recommendation? So you potentially could. What you're getting here is basically, as this data is updated in real time in a transactional store,right, you're going to be able to re-query it.

So you're getting everything kind of in real time,right? So if you're doing RAG and you want answers to be adjusted for what people had just purchased,right, this would be a way to accomplish that inside of a graph database.

And then there's a lot of flexibility. You could change the way this query is. You can shorten the time period. You could create a different type of traversal to go more than one hop out, et cetera.

There's somebody here who's been waiting.

OK.

Go ahead.

OK. Sorry. I'm hiding in the corner over here. I was wondering if you could speak more about combining the scores. It seems like just multiplying them together is a good way to get started. But is there a better theory there?

Or it seems like something that has pitfalls to just multiply the two scores together. You don't know. Are they normalized? What's the distribution between them? How are you going to achieve the results you want?

Yeah. Yeah, that's true. Yeah. So again, this is just an example to get you started. I suppose you could maintain a rank ordering where you say, hey, rank by the search score first and then rank by the vector score, which would be kind of independent of scale.

But yeah, there's a few different ways you can do it. This was just a quick way for me to put it together. And I could probably do it better. But yes. And with these systems, it's always good to have labeled data and sort of figure out how you can fine-tune things.

So in a production system, you would have recommendations and whether or not those recommendations were successful. And then you can try different methodologies and see which ones get the best historic score too.

For normalization, can you at least use recommendations? One second. One after the other is next.

Thank you.

Yes, please.

Maybe a follow-up,right? If I had this problem in front the first time, I would probably have gone filtering. So filtering on the graph relations and then filtering on fixed threshold on the semantic matching. Can you speak to how you would think about doing it my way versus computing scores like this and multiplying them?

So you're saying filtering. So basically, you would do the graph pattern first.

Yeah. I would register it on the graph to only the subgraph that matches these patterns.

Yes.

And then there filter to only the articles that have a similarity score above threshold.

Yep.

And return that.

Yep. You could do it that way. And in fact, we have a demo of exactly, I think, something very close to what you're describing. The issue is for this course, if there's a chance that whatever you put inside of the graph, inside of that local graph, could have a cold start problem,right, where it might not very well match the vector search.

So it is a way of doing it. It just might end up that you could be over-restricting to a local area of the graph that might not have, in this case, good Denim Jeans, for example. But you could do it that way.

You could also do a filtering where you said, for example, give me I guess you could do vector search filtering first and then filter down to the graph. And then if there was nothing in the graph, default to the vector search,right, results and the scores there.

So you could do it that way too.

Any intuition on what you expect to work best, these filtering steps or multiplying to one score?

I think multiplying into one score is a good way to it's just a way for me to get started. And some of it will depend on the scale of your graph,right? So the more sort of purchase transactions you have, the more well-connected the graph is going to be.

And the better you're going to be able to sort of match things, almost like with a pre-filtering pattern where you just restrict to that area of the graph first. When it's a little bit more sparse, then you're going to probably not have that same reliability.

And you're going to need to either do an optional match or some post-filtering pattern where you make sure not to over-filter things.

So another question.

Yep.

Just two real quick. I wasn't going to ask you about scoring, but since there's been a couple. Normalization, a typical way to do it is reciprocal rank fusion. Is that available here?

Not that specific one, no. We have some normalization. It's the graph data science library. There are some normalization functions. But they're not going to normalize things at query time in the way you're thinking.

OK. Yeah.

Yeah.

That's the most popular way. OK. And then the other one is just and if I'm getting ahead of myself, just tell me and I'll be quiet. But everything we've done so far, it seems like creating the peer group, you can do that with a graph database,right, and infer relationships.

But what part of this is actually the knowledge graph part of it, which is infer unknown relationships based on unsupervised understandings? Have we gotten to that yet?

Yeah. So we'll get to some of that in a little bit.

OK.

Yes. So we'll get through to that.

By the way, there are also some empty chairs here if someone wants to sit down.

Thanks.

Any other questions? Otherwise, we probably need to progress. By the way, we also have a really big booth with really comfy chairs over there. So you can always come the next two days and ask Zach and Andreas and Tomash and me many more questions.

And we have Swag at this booth as well.

Awesome. Thank you, Michael.

Allrighty. So now we're going to start on semantic search with knowledge graph inference. So this will get a little bit into what you were talking about. There's a lot more. So we have a whole knowledge graph builder where we build relationships from unstructured data that we're not doing in this course.

But we do have it at our booth.

So basically, this is where we're going to start using the graph data science portion of Neo4j. So we have this thing called Graph Data Science inside of our product. And basically, what it does is it offers various graph algorithms to you and this special workspace and data representation.

So you can effectively create a projection of part of your graph in sort of this sparse matrix representation. And that allows you to run a lot of algorithms very efficiently. So things like pathfinding, centrality, community detection, different machine learning algorithms, and of course, embeddings and KNN similarity, which is what we're going to talk about here with a lot of focus on this graph embedding piece.

This is a customer who created a very nice sort of periodic table of all of our algorithms for reference. But there's a lot of different areas,right? So there's a lot of things that we aren't going to get to talk about here.

So things like community detection, pathfinding, centrality, topological link prediction, and directed acyclical graph algorithms. But we are going to talk about similarity and embeddings. So basically, a node embedding works just like any other embedding, except it's for a graph where the things you're embedding are nodes in a graph.

So this is a picture of a very small graph. I think it's Zach's Karate Club is the name of the graph. And it's basically the relationships between different people in a karate club. But the idea is that I can take this graph and I can turn it into a set of vectors.

And in this case, it's a 2D vector, which is a 2D representation. We're going to create longer vectors for the notebook. But the idea here,right, is that the yellow nodes that are similar in the graph in the sense that they're really close in the graph are going to be really close in the embedding space and the orange nodes and the green nodes and the blue nodes, et cetera.

So that's the idea of what we're doing. We're basically embedding what we call homophily, which is basically this fancy term for the locality inside of the graph structure. And once we do that embedding, what we're going to be able to do is start inferring relationships.

So someone mentioned being able to do sort of inference beforehand so calculations are available at query time. And we're going to use k nearest neighbor for this. So basically, this is happening outside of an index. What we're going to be doing is creating graph embeddings specifically on the articles.

And then we're going to be inferring these k nearest neighbors between them based on KNN, which is this semi-supervised or unsupervised algorithm. And you can do this for a lot of different things. We are going to use co-purchase relationships for this and kind of scale that process for multiple hops.

But you could do this for other forms of link prediction, for example, things like entity resolution or if you're trying to resolve. Maybe you have people and locations in the same graph and you're trying to resolve whether or not a person is in a specific location.

You can use these sorts of models to start making those knowledge graph analytics and drawing those relationships.

So let's dive into the notebook to see how that works.

Any question?

Yes.

You mentioned you're embedding the nodes. Are we essentially throwing away the edges? We're not embedding the edges?

So we're doing a node embedding, which is specifically focused on nodes. We could do different types of knowledge graph embeddings, which embed the edge and it embeds the what is it? The head and the edge to the tail,right?

So there are those types of embeddings. And there's different types of subgraph embeddings. For today, the relationships are used to understand where nodes fit inside of that vector space,right, in terms of clustering of, in this case, what's going to be co-purchase behavior.

So we're not throwing them away. They're going to be used.

And we'll use the yeah, you'll see as we go along. We'll continue to use the relationships.

Sounds good.

So basically, what we're going to do here is we're going to create a customer's also-like relationship. So basically, this cell so it's going to be the first cell after you get to augmented semantic search with knowledge graph and ML.

And if you run that cell, it's going to run our GDS algorithms. So I have some utilities to kind of help clean things up. And then basically, we're going to create something called a graph projection. And we don't have a whole ton of time to go into detail on these in a two-hour course.

But the basic idea,right, is that I'm going to look for these common co-purchase relationships between articles and customers. And then I'm going to project that as a co-purchase relationship to an in-memory graph. So I'm creating basically this graph that contains articles and then a weighted co-purchase relationship between them.

After that, I'm going to run our node embedding model. So we have supervised models that we can use. But in this case, I'm going to use something called FastRP, which stands for fast random projection. And basically, what this is is a graph embedding that is going to use a lot of sort of this matrix multiplication with sort of this sparse random matrix.

So the math behind it basically allows it to calculate and embedding really quickly without necessarily needing a neural network. I'm going to make these embeddings 120 dimensions. I'm going to specify certain things like an iteration weight that will tell me how many hops to go out when looking for other similar nodes.

And then once I create that embedding, I'm going to write GDS KNN write. So basically, what this is going to do is it's going to find nodes with similar embeddings. And it's going to write a customer also-likes relationship between those nodes.

And then there are some sampling procedures that I can use inside of KNN to make it run faster, which I'm doing here. And then once I have that, I can write both my embeddings back to the graph. And in KNN, I just wrote the relationships back.

So now we have these customer also-like relationships between articles inside of the graph. And I get some stats when that's run telling me about whether or not it converged, the number of new relationships written, which is going to be generally a lot because it's doing this quadratic comparison.

And each relationship is scored. I had a similarity cutoff at, I think, 75. And this was using cosine similarity. So it would be negative 1 to 1, I think, that it was using. And we can visualize node embeddings as well.

And that's interesting to do in the sense that it will tell me clusters of purchase behavior. So these cells that I have here, I've commented out because they take a while to run. But basically, this should still be able to be rendered here, this visual.

And basically, what this visual will show you is you see we get these clusters. And basically, what I did is I took those 128-dimensional embeddings. And I used a TSNE, which is just a way to condense that down to a smaller space.

And I condensed it down to a 2D space. And basically, what you'll see is you'll get these clusters of commonly purchased things. And so this is sort of doing the multi-hop thing that you were referring to earlier where now it's starting to look at, OK, well, these different clusters,right, these clusters represent items that are often co-purchased together.

Baskets,right?

Baskets, yeah. And yes, baskets in a sort of sense. Right now, it's just co-purchasing. But you could organize into orders inside of the graph. And you could do co-purchasing baskets as well.

And so with that information, we can start to sort of scale how we power just plain recommendation. So I can create a new query here where basically, instead of going to the vector database, I can just say, OK, we'll match the customers.

Get me everything they purchase as the articles. Get me what else customers also like to buy. And get me the product behind that. So it's similar to what we did before, except it's scaling it to sort of a multi-hop pattern.

And you can substitute other machine learning models in here if you wanted to, too. So if you had, for example, a supervised model that you wanted to use for recommendation, that could be substituted in here to make these inferences.

And then you all of a sudden have these customer also-purchase relationships. You could also create a vector index for that graph embedding, which we're not doing here. We've sort of done the KNN beforehand,right? And we've written the relationships to the graph.

And that allows me to do this customer also-liked relationship that I can draw in the query. And that gives me a set of recommendations, again, not necessarily based off of a search pattern, but just based off of what the customer had purchased before.

And to your point earlier, we could create time, date, and date cutoffs and things like that.

So this is basically just a way to kind of scale things with the machine learning model and leverage graph embeddings. So that was it for the graph embedding and sort of the graph machine learning section. And again,

you can use a lot of different types of machine learning models here to basically infer these relationships, some of which exist inside of our platform through Graph Data Science and others that you can bring in externally. We're at 4:48.

So we have, what, 45 minutes left, a little under. I mean, I haven't taken a break yet.

I'm wondering if we should power through.

Yeah. How about people too? Keep going?

Yeah, keep powering through.

OK. Yes.

I have a question regarding the graph embeddings.

Yep.

Do you always have to recalculate it if there are any projections on the customers that are coming in?

Soright now, you do, yes, have to recalculate the graph embedding. Ideally,right, if you over time, if customers have been in there for a while, that graph embedding, well, it'll slow down how much it changes. We are working on things for the future that might make that easier in terms of getting things in real time.

Butright now, for calculating graph embeddings, it is something where you have to recalculate it for the graph. Thankfully, though, with something like FastRP, that'll work. I mean, I've seen it work on graphs with billions of nodes and relationships.

And it can calculate in like half an hour. But yes,right now, you do have to recalculate it.

You could use other methods too, like averaging the neighborhood as well for sort of intermediate embeddings as well. So if you didn't want to recalculate itright away, you could take the average of the neighborhood and use that as a stand-in embedding.

That's not good.

Allrighty.

Is there an app on embeddings? Like how large?

What's that?

Is there an app on embeddings? Is there graph embeddings? Is there an app on embeddings? Now you have 220, which is not a lot of.

Yes.

Do you want me to?

Oh, yeah, go for it. I think

you had a question about dimensionality, sorry.

Yeah, about the dimension. Is there an app on embeddings to the dimension? I think you had now 128, which is quite small.

Yeah, there's no upper limit to dimension. Right now, inside of this graph, we only have let me see how many customers we have.

So this graph only has 1,000 customers,right? There's been other examples and demos that we have where it's like millions or tens of millions of customers,right? So the more transactions you have, the more customers and articles you have, the higher that dimension needs to be,right?

For demo purposesright now, and we're working off of sandbox instances, I kept it pretty small. But you can expand this to 256 or keep going up in orders of two. I found that these generally don't need to be as big as the text embeddings that you get from OpenAI because it's a smaller space,right?

Even if you have billions of nodes in your graph, it's still a smaller space than the entire human language,right? So they tend to be on the tend to be smaller. But they can go up. And there's no upper limit that we inherently have inside of the product, no.

And what is the biggest use case you have at the moment deployed?

For embeddings?

Graph embeddings, yes.

I'd say entity resolution, different forms of classification, like for fraud detection, and then also for recommendation and things like customer segmentation where you want to create clusters based on behavior or other data inside of the graph.

OK.

Is it possible to semantically retrieve the information about the graph structure itself? So let's say that you have 10 different types of pens. But you want to ask a question, how many types of pens do I have, which is like it's not in the data itself specifically.

It's kind of metadata about it.

Yes, so very good question. So there's different types of retrieval patterns that are better at those sorts of queries. So we have something called text to cipher. We have it. It's a pattern,right, where you can actually get the LLM to generate a cipher query to query the data.

And those tend to work well for aggregation. So if you were building out a more mature system, you can have a semantic layer with different tools. And so if you knew that you had more of an aggregation question about types of things, then you can send it to that tool, which can do a text to cipher or another sort of a cipher snippet that was pre-generated or template that was pre-generated.

Does that make sense?

Yeah. Have you tried using that with function calling? So you don't have to specifically know in advance that you'd like to use it, but rather like give the LLM the ability to do that.

Yeah, that is something that Michael, if he's still here, has done some work on for function calling.

Yeah, so

yes, in general, you can make many of these graph operations available as functions to an LLM. And then basically, it depends on the selectivity of how you describe the function and what you pass in to make this available,right?

So you can do this kind of embedding work. But also, other graph queries can be made available as functions to an LLM. And then it just selects them. So for instance, get a neighborhood of a node or find the shortest path between two nodes or find similar nodes to this node using graph embeddings or whatever.

And then you have these kind of some generic functions that you offer to the LLM. And then you could have also specific functions that are basically built by subject matter experts that, for instance, for a legal domain or for a fintech domain, they have specific graph queries that are really specific to that domain and then add them as functions as well.

So you can have 5, 10 functions available. And then the LLM picks the ones that are necessary to answer the questions as such. But it should work with all the LLM models that support function callings and such.

Yes.

Thanks. If you have more questions, you can always come to talk to us in the demo.

Yes.

Tomorrow and on Thursday.

So I was just trying to gain a little more intuition about the idea of graph embeddings. I'm pretty new to graphs in general. It seems to me like the whole motivation for using embeddings has really taken off in the last four years or so.

And it's because you've got transformers. You've got unsupervised pre-training where you can leverage all of the text on the internet or all of the images on the internet and whatnot and train these embedding models that are just general-purpose foundation models.

And this is very different from that because you're just completely training your embeddings or if it even is training from scratch. I'm wondering if there's a deeper intuition to why you'd be using embeddings for this instead of it almost seems like there's a lot of more traditional machine learning techniques that you could be using.

I just don't know what the state of the art is. And I'm curious what the intuition is for using embeddings for this.

Yeah, I mean, so embeddings here basically is helping you kind of scale these other graph traversals,right, to go because when we do that embedding, what we're basically doing is we're taking a graph traversal that's similar to the one we went over and going multiple hops,right?

So you're sort of taking this structure of the graph. And you're scaling your ability to search for related things inside of the graph without having to do a cipher query every single time. So it's basically and it can help you also infer information that you didn't know before.

So in this example,right, we're making these relationships on customer also-liked. We could also do things like, say, hey, if we had an identity graph of people and different identifiers and transactions, we could start resolving people in the same household, for example, using embeddings.

So even though it is kind of local to your graph, the idea behind embeddings is that you're able to infer relationships between things inside of your graph that would be very hard to do without some sort of machine learning or just traditional methods, if that makes sense.

A little bit?

That makes sense.

Yeah. Yeah, because you're basically using depending on the graph embedding you're using, you're basically taking the structure of the relationships and that topology. And you're using it as features,right, to power what something should look like in a representative space.

And then you can then use those to basically predict where links should exist inside of the graph.

Yeah. OK, that makes sense.

Yep, hopefully. I don't know if it really did, but.

It's a question. I mean, it's that you're.

You're providing this sort of intermediate representation where you can do more general-purpose things with it.

Yes.

Is that good paraphrasing?

Yes, I think so.

OK, understood something.

Allrighty, awesome. So this is going to be the final step for tonight. And then we'll see. Maybe we'll even be able to get out a little bit early and get to the reception. So we're now going to take what we have.

And we're going to start using a large language model. So we're basically going to use the retrievers that we were just working on. And we're going to take them into this prompt. And we're going to have a language model generate an email for us.

So this might be a little bit hard to see. But it's also in the notebook for when we go there. This is the prompt that we're going to use. We're going to instruct an LLM. We're going to give it a name.

We're going to say it works for a certain company. And we're going to tell it to write an engaging email to a customer to promote and summarize products relevant to them, given, one, the current season and time of year, and two, recent searches and interests, which is going to be our search prompt.

And so all of sort of the red things are where we're going to inject things inside of the prompt. And then the RAG part comes in. So we're going to tell it, hey, please only use the below-relevant products.

So there's going to be what we're calling search products and recommendation products, which search products is going to be based on a vector search combined with a graph traversal. So that will give us something that matches the customer interest.

The recommendation products are going to leverage our graph embeddings to give us additional recommendations that we can add to that. So there's going to be sort of two lists that it gets. And there's some other instruction there to tell it to only pick a certain number from those things and kind of be creative with that and make sure it aligns to the current season and time of year.

And this is kind of what it looks like. I'll just go to the end of this thing. So if you were to think about the chain that we're going to put together, we're going to come in with basically four things: that search prompt, the customer ID, the customer name, and the time of year.

The search prompt is going to go with the customer ID for the personalized search, which is going to be that sort of peer group pattern. The customer ID is going to go directly to get recommendations based off of graph embeddings.

And then these four things here, the products that were recommended here and the customer name and the time of year are going to go into the prompt to then generate an answer. So we're doing basically two retrievers here, one for recommendations and then another for basically search results.

You can think of it that way.

Allrighty, so let's go and dive into the notebook. So

we're going to use LangChain for this. We're going to instantiate the LLM. That's going to be based off of GPT-4 Omni, which we have here. And then we are going to recreate two retrievers. So the first retriever here is going to be for the personalized search.

This is going to use the text embeddings. And then it's going to do this sort of post-filtering pattern with that combined score where we do this optional match to look at the peer group to kind of filter those results.

We're going to create another function here based off of that. I'll zoom into this a little bit more

where basically we are going to take

this function is just so we can pass a customer ID parameter in. So this is the KG personalized search function, which is going to use this retrieval pattern. And then we have a KG recommendations function, which is going to use that customer also-liked relationship that's going to just basically go off of customer history.

And then both of these will sort of join. It'll do some formatting for the documents and return them for injection into the prompt. Here's the prompt itself. So this is just the prompt we look at in the slide before.

It mentions customer name, time of year. And then it has the relevant products where you put in the search products and then other recommendations generated from the graph embedding.

And then when you create the chain, so you can see here on this chain, we have four inputs,right? So the search products, which again comes from the personalized search pattern, the recommendation products, which comes from the recommendation pattern.

So there's these two retrievers that are supplying each of these respectively: the customer name and the time of year and the customer interests as well. So I guess all five of these things,right, get passed and injected into that prompt, go to the LLM.

This is all LangChain expression language. This is kind of a LangChain thing where we have this pipe operator that's just basically taking this dictionary of things and then passing it to be injected into the prompt, calls the LLM, in this case, GPT-4.0, and then just gets outputted.

And so if we look at what it looks like when we invoke that chain, again, using our search prompt, which is denim jeans, our customer ID. And then I gave it a customer name and a time of year.

And the customer name is probably something that could be stored in a database too. It's just that this data was anonymized. So we don't have that. We don't have that in the database itself. But anyway, what this will do, you see, it'll give me its responses.

It'll make a subject header for the email. Hi, Alex. I hope this email finds you well and enjoy the start of the summer as the days get warmer. It's a perfect time to refresh your wardrobe. And then it will give you the denim jeans.

So these are the items that came from the search that was personalized to this user. And you see we get a set of five jean products here, summer essentials to pair with your denim. And then this is where it uses the graph embeddings to provide those other recommendations down here.

So it's basically,right, using a combination of this vector search to pull things back, personalizing it so that it becomes specific to just this user, and then adding additional recommendations to that as well. And of course, you can change the prompt to be just the recommendations portion if you wanted to or just the search portion or whatever was appropriate for the application at hand.

And if we just looked at so basically this cell, if I run it, all this next cell will do is show you the prompt, the full prompt that was set to the language model. So if we look at this, this is basically showing you if I were to scroll all the way to the top, it provides quite a few products.

You can see here, this is the data that it retrieved and formatted from the database for the relevant products, all the different garments that it selected. And then if I were to go down more, it would eventually bring me down to the recommendation products, which I think it already did here.

So if I go

up further, yes, relevant products. And then it should give me about 10 of those and then other recommendations from the graph embeddings that go along with that going down here. And then it basically chose from those products to generate that given response.

So we are limiting the number of products because of the context length?

Yes, yeah. If you look at the way those queries were written in the cells before, I think the first one for the search responses, I limited them to 10. And then for the recommendations, I'm actually passing 100. So I'm passing quite a few.

But yeah, eventually, if you provide enough of them, you will hit a context window issue.

And then so basically, once we have that, we can run a demo app. So in this case, we're using Gradio. This is just some examples, preset examples that we can use in the application. And this is running the application.

So I'll go ahead and give it a run. And if you have it in your notebook, it should just give you a link that you can go to.

And basically, let me make this bigger so it's seeable. So here's just a UI of what we had before,right? So this is the example that we ran earlier. So if I say Submit for this specific user, it should pull back hopefully in the next 20 seconds or so because the internet's slow.

Normally, it's around 10 to 15 seconds. Yep, and this will pull back the email response. And the interesting thing about this is if I took you see this customer ID is like DAAE. If I select a different customer ID but the same prompt, so say it's this person.

So I click that. And then, yes, this changed to this other customer. This answer should be different. And if it isn't different, it's because they have a very similar peer group and a very similar piece in the graph.

But if you see HW denim and skinny jeans and then this, you'll see they get a bunch of different recommendations and search patterns. So the idea here,right, is even though these two individuals had the same interest, based on their purchase behavior in the graph, we're able to give them very different recommendations because they exist in very different parts of the graph and have different interests based on their purchase history.

And you can play around with this. Like if I you'll see it'll give me some summer things here. If I change this to like February,right, this has nothing to do with graph. This is purely an LLM thing at this point.

And I submit it. It should the email should change to winter. And then it should give you instead of summer must-haves, it should give you something about winter. And it should be able to select

different products down here that should hopefully, if the LLM is doing its job correctly, go for more winter-related items like sweaters that you see here.

So that's the full app that kind of leverages all the things that we just went over.

Cool. Thank you so much, everyone. This was really great.

Thank you so much.