Wisdom-Driven Knowledge Augmented Generation at Scale - Chin Keong Lam, Patho AI

So, hi—hi, everybody. My name is Chin Keong Lam, um, I'm the founder and CEO of Patho.ai. A bit of background about my company: Patho AI started 2 years ago with an invitation from the National Science Foundation, from the SBIR grant funding investigating LLM input.

We did an LLM-built driven drug discovery application. Since then we branched out to leverage what we learned about building AI systems for large corporations. We are currently building expert AI systems for several clients. Currently the system we built goes beyond the RAG system.

Many of our clients are asking for AI systems that perform tasks like research and advisory roles, based on their area of interest. Today the talk is about sharing with our fellow AI engineers what we learned so far building this kind of system.

Okay. What is knowledge? Okay, generally philosophically I say: knowledge is the understanding and awareness gained through experience, education, and a comprehension of facts and principles. And that leads to the next question: what is knowledge graph,right? So, knowledge graph is a systematic method of preserving wisdom by connecting them and creating a network of interconnect relationships.

That's important. The graph represents the thought process and comprehensive taxonomy of a specific domain of expertise. That's why this is very important for people moving forward. It's about AI systems that think a lot and return advice instead of just retrieve, you know, data from your database,right?

So that comes to the development of this KAG. Okay, what is KAG? KAG stands for Knowledge Augmented Generations, and it's different from RAG, okay? It is an enhanced language model by integrating structured knowledge graphs for more accurate and insightful responses, making it smarter, more structured approach than a simple RAG.

KAG doesn't just retrieve; remember, it understands. This is different. Okay. After interviewing a lot of my clients, okay, or we also are experts in a certain area of scale, I found that there are common ways of their thinking, decision-making process.

The way that make them experts in their area knowledge graph seems to be a perfect fit. So here is a graph, or state diagram, if you're a computer engineering grad like me. So, um, it shows wisdom. The wisdom node, as you can see, is the core,right?

It's wisdom. It just isn't static. It actively guides decisions and fuels by other elements.

The output from the wisdom actually goes to decision-making in the blue,right? Wisdom isn't passive. It guides decisions, helping us choose wisely. Okay. And then the decision-making analyzes the situation. It's given in the circle in the green. And decisions aren't made, you know, in a vacuum, okay?

They analyze real-world situations. That's the difference, okay? So look at the wisdom input. Okay. Look at the relationship: feedback from the knowledge to wisdom, in gold color. An example of that is knowledge to wisdom. Like all your books, smart encyclopedia, Wikipedia, whatever you store.

Plus, once that data gets absorbed by LLM, whatever model you use out there, it needs to regurgitate that and understand. That's why it's very important that wisdom is able to synthesize the data after you ingest the knowledge. Okay, that's the kind of abstract, but I'll come to that later.

What I'm talking about, okay? Okay, from inside. An example of that is wisdom derives patterns from chaos. Like some of my clients have a lot of social media. Their product, how do they, you know, track their product settlement from social media,right?

Sometimes it's very chaotic. And from X, tweet,right? So from that, you can see some pattern of their competitor versus current what my product is. That's like an example of that, and I will go to that later. Okay. When all these connected nodes matter together, why do they matter?

All the nodes relate to one another to ever increase, um, enriching a wisdom storing system. Okay. This talk is about storing wisdom,right? So knowledge tells you what it is,right? And experience tells you what worked before. Insight invents what to try next.

Like a pizza, knowledge is recipe. Experience is knowing your oven-burnt crust. Insight is like, hey, it is adding, you know, honey to the crust, you'll make caramelize perfectly,right? So the most important part of the knowledge graph is feedback loop.

Okay. Feedback isn't one-way street. It learns from itself. Look at the feedback from the, uh, going back to all the nodes from insight to wisdom. Okay. Situation informs future wisdom. Experience deepens it. Insight sharpens it. Like a tree growing roots.

The more it fats, the stronger it gets. Now, I want to ask you a question in general. Where do you see this circle in your life? Maybe a tough decision that, you know, taught you something.

So one practical application for leadership is wisdom: avoid knee-jack reactions by learning from feedback. As for personal growth, ever notice how past mistakes make you wiser? That's the loop in the action. All this. So the takeaway from the slide in this is: wisdom isn't a trophy you earn.

It is a muscle you exercise. The more you feed knowledge, experience, insight, the more they guide you. Now, I will show you how it's being mapped to my current clients. You know, all this is like very abstract,right? So how I—one of my clients is actually doing a competitive analysis.

They used to have a marketing department doing that, but they want AI to do that,right? They asked me to build a system. This is exactly what I did, with the same taxonomy of storing all this. So this taxonomy will be later on.

I'll talk about how multi-agent is going to handle all that. Here is one of the chatbots that I built for my client to do, you know, not just some, uh, wi—not just some chatbot, okay? It's our Wisdom Graph Power AI, designed to turn data into strategy,right?

Dominant. So what kind of question are we talking about? We're talking about how do I win my competitor in this market space. That's kind of a very sophisticated question,right? So without, uh, if you do simply just RAG, but first speaker talk about RAG,right?

So it's not going to cut it. They're not going to be able to answer that kind of question. Okay. What I did is this: we retained the same taxonomy, and the wisdom is then mapped. The same engine there, the Wisdom Engine.

Wisdom Engine is like an orchestrated agent that does a lot of decision-making, including advising what the AI is able to see based on the current situation. What to do next,right? So, um, what I did is, uh, for the decision-making, I mapped it to a strategy generator.

So this customer is talking about competitive analysis,right? So, um, I mapped the knowledge in terms of knowledge. What do they have? They have market data,right? So I mapped the experience to HP. It's one of our clients. Okay. Past campaign.

So they have a lot of campaigns, doing a lot of marketing. And then, um, the insight is actually mapped to, uh, industrial insight. They have a database doing storing that. And then, of course, the most important is the situation.

The situation is, how am I doing? How are my products selling,right? So that is like a situation. And then I mapped that to a competitor weakness. That means to say, if you make the LLM aware of that, you probably get a very good answer, and the chatbot will probably be doing theright thing, advising.

So from here, very high-level, you know, state diagram or that. How do I map it to a system that drives? Well, here comes the trick. So anybody here heard of N8n? Allright, allright. That's all good. So, so I first encountered a similar situation when my past IoT project, which is Node-RED, developed by IBM,right?

So it's the same kind of thing. It's like no code, but underneath the hood there's a bunch of code, okay? It's all Node.js code, okay? So, uh, but for proving your concept and all that, it's very, very, very flexible.

And I highly recommend that. And here, here you can take a look at the workflow, the workflow. I enabled the implementation of this complicated state diagram with, um, what I say is, there is a different community node. One of the very powerful nodes is the AI agent.

Previously, N8n is just a workflow automation tool. I'm not selling for anything here. I'm just telling you I'm using it for prototyping. Further down the road, maybe the client says to, like, I really need to go lightweight. Maybe we'll switch over to some other language chain or whatever.

But, uh, we actually use this. Like, I mapped the previous state diagram from the Wisdom Engine. I actually mapped that to our Wisdom Agent. Okay. Wisdom Agent now has the option to drive different models, like OpenAI model, Anthropic model, and even on-prem model.

And then that, the key in making the state machine work is that my Wisdom Agent is now overseeing, like a supervisory agent, all these other agents that do whatever I say on the state diagram. Like, for example, the state of going into a node of insight.

Insight agent will task to do, go to the social media, look for the settlement of all your products, and then collect that. And then pump that. You can see that at the bottom that I connected to a

centralized graph. That centralized graph will be able to get updated by different agents. Insight agent will update their perspective, like part of their graph for the, as I say, for this particular insight node. So all the unified knowledge graph will contain the taxonomy that eventually just think like the marketing strategies.

The way that it was probably, if you are doing manually, they probably would think in a SharePoint where all these, you know, folders will store the same kind of, you know, wisdom, I call it, to make decisions based on that.

So the final decision is LLM also depends on the model that you use. But I pretty much think that not really the way that I think the final decision comes when you make theright decision from the advisor output is basically depend on all the taxonomy, the graph structure.

That's very important. So come to that. I want to go deep down how I implement one of the nodes. Just to go a bit technical on this competitive node. How do I implement that? Okay. Before I do that, okay, competitive analysis.

Well, you can actually just use RAG. What do you want to use knowledge graph like Neo4j? Well, if you ever being asked that question, tell them this is five reasons. Okay. First reason is knowledge graph, you know, system excel at capturing and representing complex relationships with the entities.

That is covered by the first speaker, but I just reiterate that this leads to a deeper contextual understanding, which is crucial for competitive analysis, where in this case, the nuanced insight can be significant, make a significant difference. Okay.

You want to find the gap in your competitive weakness. Now, this is very important. The second is improve accuracy. By leveraging structured data and semantics relationship, knowledge graph can provide more accurate and relevant information compared to traditional vector RAGs.

This ensures generated content is not only relevant, but also precise and reduces noise and improves decision-making. In this case, the bot is supposed to help the guy that is in the marketing department make decisions. So you better make this work.

Improve accuracy. Any inaccurate data, you will be out of the contract, out of the door. Right? So it's very important. Okay. You're talking about contract work like me. I have to make the RAG as accurate as possible. So the third is scalability and flexibility.

They are graphic, you know, knowledge graph are inherently scalable and can integrate to new data sources and relationships. The flexibility allows the continuous improvement. As I say, if your taxonomy is correct, you will continue to improve and reach,right?

So that is important. And also rich query capability. Knowledge graph supports complex query. Traverse to multiple relationship entities provides richer and more detailed insight. This is particularly advantageous for competitive analysis when multifaceted query, like what the first speaker said, it is super authority good in answering things that normal RAG will fail.

It's like multi-hop question. Okay. This is very important. And then the final one is the enhanced data integration. Knowledge graph can seamlessly integrate diverse data sources, pictures, graphics, videos, however it is. Now that LLM is so powerful, we have OCR capability can do that.

As long as you have aright structure of the graph, semi-structure, unstructured, the holistic approach ensures a comprehensive view of the competitive landscape and enables more informed strategic decision-making. Okay. So one of the, this is, I'm going to just very briefly go through this.

It's just an example of some of the things, like problem of a vector RAG, you know. Vector RAG is really, really bad in answering limited numerical reasoning. Vector store excel, you know, adds semantic similarity but struggles with complex numerical calculations.

This is why for marketing analysis that I'm building the chatbot for, they actually rely on numbers instead of just, you know, returning examples like this. If you ask like, what is the Apple revenue between two, you know, what's the revenue in 2022?

They probably will give you a bunch of this kind of passage,right? Retrieval graph. Instead of this kind of very, very precise thing, like the answer is, you know, knowledge graph is able to, because the data is already there in structured form.

The data source assumes a knowledge graph name. This particular, in this particular case, Apple financial data. The query will be able, the query engine will be able to select the revenue figure from 2021 to 2022 and then do a function call.

The function call will eventually come out with 15.23, which is exactly what the marketing guy was looking for. A very quantitative stuff that most of the decisions were based on that because you have evidence, not just some passage that you retrieve from the data.

It's basically evidence-based decision-making. It's very important for this kind of complicated RAG system that, you know. So, um, there's a jungle out thereright now. You can use different kind of

things to build yours, you know. This is just a snapshot of that, you know. You can actually use LangChain plus Chroma to build your own RAG. And then you also can combine that with your knowledge graph. Depends on your user case, okay?

This slide shows that the RAG and the KAG can be built with money. Okay. I adopt that Wisdom Graph in red color. Normally you will see if the client is just asking for a simple RAG that performs product information query, you can just use a simple Chroma DB with LLM agent.

And if you start to ask so complicated questions like, how can I beat my competition based on my current market share? Well, this will be ever the thing that I will probably be adopting is Knowledge Graph here with Graph DB plus Cypher Query and then do a quick Q&A and also train my RAG to perform several loops of, we call it multi-hop query.

And this probably will give you a very good answer. So, and then it comes to that another question. When I was trying to extract my whole, I think my time is almost up. Okay. So anyway, this is like to say, the first speaker talked about the extraction,right?

There's a very simple way to extract. On theright side is like automated, totally automated LLM graph transformer. On the left is like manual. I would probably recommend the center hybrid model, which is like after you use the LLM to extract your graph, you ask the interview, the expert that you're going to build to build your taxonomy,right?

To prune the graph. We call it pruning the graph. Remove a lot of relationships. Then that will be okay. And I will try to just highlight this. This is the result of benchmark that we did. Okay. Anybody ask you, you know, why you want to use Graph RAG or KAG?

Okay. First is accuracy. I had achieved 91% because it's really good in extract structure. Second is flexibility, 85%. Third is reproducibility, deterministic, and then the fourth one, traceability. And finally, most important is scalability. So in conclusion,

by leveraging structured nature of Wisdom Knowledge Graph, we can significantly enhance the quantitative capability of KAG systems and ability more accurate and insightful response to complex queries. By using Wisdom-Driven systems, as highlighted, together we can build smarter AI systems that can scale and store wisdom with theright framing, potentially supports the intelligence of the initial expert that we meant to serve.

So talk to Jesus, you know. What do Jesus do? Talk to Jesus. He's in a knot booth. This is my good friend. And anybody that wants to build graph, we have a good so-called LLM Graph RAG stack on GitHub that is sponsored by Neo4j.

And out of the box, just spin up your Docker. The next thing you know, your text is going to be converted to your graph and you can start happy pruning your graph. Thank you.

Thank you so much.