Your LLM Ran Out of Knowledge — Now What?

So, it wasn't that long ago that we were all talking about chatbots, and it felt like only yesterday that we were excited about agents. But today all we're talking about is reasoning, and how these models can now reason through complex problems.

But how is a model going to solve a problem when we haven't been able to give it any training data or knowledge in that specific domain?

So what I want to do here is share a technique that you might want to try in one of these low-knowledge domains. So what do I mean by that? Well, we know that the models are great at coding, and the models are great at math, and that's not surprising because we have large bodies of trainable material, well-structured information and data, that we can give to the models to train them for math and coding and the law and physics, because everything is structured in a consistent way, and it's in a format where we can easily convert that into training data and give it to a model.

The problem that we have is that we also have other areas where we don't have well-structured information. So, as an example, think about corporate negotiations. We know what the start point was because there was a story in the Wall Street Journal that Company A wants to acquire Company B, and we know what the end point was because CNBC or the Financial Times had a headline explaining what the outcome was.

But what we don't know is what happened in the room. What did those negotiations actually look like? We might have some rules or guidelines that come from an MBA course, but we don't know what those discussions were. So there are a lot of domains where that specific knowledge, that wisdom, is locked away.

It's in people's minds. It's never written down. And where this is taking us, or where this could take us, is that we still have these areas where this knowledge gap is not being filled. So actually, we have sectors or professions who are becoming better served by these models, who have more powerful models at their disposal.

They can do more and more. But then we have other domains who are almost being left behind.

They don't have models that understand what they do and aren't able to apply them to the problems that they come across. So what we want to do is actually close that gap. How can we close that gap where we do have this lack of knowledge?

And as practitioners in the sector, I'm from risk and security management. It is the fault of the practitioners. We don't write things down. So this is not the fault of the people training the models, and this is nobody's particular fault, but the sheer fact is we don't write things down, and therefore there won't be, there isn't, and there won't be the material to train a model.

So what do we do to overcome that? How can we overcome this knowledge gap in order to make sure that we have models that can work in all of the domains where we want to apply their expertise, skill, and ability?

Well, one of the things we can do is we can use a technique that we already use. We already use this in the workplace. Someone who has just left college, they're a recent graduate, highly intelligent, but probably lower in wisdom.

They haven't had the experience. Now, there are lots of things we can do, but one of the things that works really well is giving them guidelines, giving them rules of thumb, heuristics, whatever you want to call them. We can give them some guidelines to apply in specific domains.

What that then means is they can use their intelligence, apply these rules for that domain in order to solve problems, in order to understand what's happening, and start to work through things in that specific domain while they build up their personal knowledge.

And so the technique that I'm talking about today takes that same approach, except we do that with the LLMs, with the models. We're going to give them a set of rules to follow for specific domains and ask them to apply those on top of the very powerful reasoning capabilities that they now have.

And that's going to give us the benefit of the powerful LLM, the benefit of the reasoning, but overcome these wisdom gaps and give them the subject matter expertise they need to start working in these other domains. Allright, so let's see what this looks like in practice.

So I'm going to toggle between this front end, which is just an easier way to show the inputs and outputs, and we'll go and look at the back end as well and see what's happening behind the scenes and how the logic's working.

But this is an easier way just to show the process. So I'm going to drop in a prompt, andright away you'll see this is quite a long prompt that concerns an intelligence estimate. And so what's happening in the background now is it's trying to figure out what kind of problem do I have and what rules should I apply.

And we'll see how that works in a second. The other thing it's going to do, it's going to reformat the question that I give it, and it's going to put it into a standard format because, again, what I found is that standardization really helps these models work most effectively.

Okay, so here's what's going on behind the scenes. The heuristic match, this is basically the same as tool use. I found that tool use wasn't working out as well as I would hope, and so I've taken a little bit more of a long-winded way of doing it.

So there's a specific LLM, a specific prompt that is going to match up the user input and look for a match with one of the heuristics. And you'll see that these are listed down here. It's analyzing for those.

And so that's the first thing that it's doing. Whatever you've presented to this model, it's looking for a match for the specific rules that you want it to apply. We want it to be consistent in the way it's presenting the information to the actual simulation.

So we haven't got to the simulation yet. When we get there, though, we want to make things as consistent as possible and make sure that they're formatted so that the kind of material the simulator needs is contained in the prompt.

And so we've asked it to parse the scenario and return it in the structured format, and you'll see at the front end, you actually have the opportunity to go in and edit it if necessary. But we're just trying to make things consistent so that each time it gets a scenario, it's in the standard format.

So the first two things it's doing is looking for a match, standardizing the format. So once we have a match, we're now going to come over to the rule set and find the appropriate rules and behavior for that particular type of problem.

So we have two different types of scenario rules in the demo: negotiations, geopolitics. So these are the rules. These are written in a fairly binary way. You must do, must not do. They're provable. It's provable if you have prioritized agreements that create highest combined value.

It's provable if you've got these three-plus independent paths for critical resources. And that's deliberate because we want to make it nice and straightforward rather than having sort of abstract, vague rules. We want to give it quite clear rules.

But by the time you combine these together, you've got a sophisticated model for these are corporate negotiations or geopolitical strategy. So these are the rules. These are the same heuristics we might give to our consultant that we were imagining before.

And then we have this heuristics list, which is a higher-level prompt, a sort of system-level prompt, how you want it to behave. And you can see at the end, it then pulls in the rules either for negotiations or geopolitical strategy.

And the only reason these are split out is just because it helps avoid the prompt becoming too unwieldy. It's a little easier to go in and edit these. And so you would have these rules for the rules you want to apply.

You might want to adjust these in different scenarios and different situations. So you've got a lot of flexibility around here. But what this gives you is this high-level, very comprehensive prompt set of rules and guides that really focus the model in on how you want it to behave.

Okay, so we've now got the result. So here's my original question, and you can see it's done two things. First, it's identified the type of problem as geopolitics, and now it's reformatted the question into a specific format, and this is just so that we've got consistent inputs to the actual reasoning or planning model.

So a lot of work's happened up front to take everything that the user has presented to it, identify what that is, package it up correctly, and then it's going to send it off to the model for analysis. So as far as the model's concerned, you can send it to whichever model you're comfortable with, whichever model you think will perform best.

In this case, it's actually going to Anthropic because I'm running this world sim, and this is something I owe a great deal to Noosh Research and Karen because I first saw this from one of his demos. And so it's running this world sim version to give it this sort of realistic view of what the world looks like because we're dealing with geopolitics.

So it's particularly relevant to this particular example, and that's just going to add on more realism and give it some additional constraints to think about. So the physical geography of the world, things like the UN and the WTO need to be considered where appropriate.

So we're just sort of very quickly giving it a whole set of other parameters to consider or to lean upon. And so particularly, as I said, for this example of geopolitics, this is useful. But at this stage, you could point it towards whichever model you see fit and whichever one you're comfortable with.

So when we come back and look at the results, obviously we've got these in a presentable format. They're easy to read. We can seeright away it's following the structure that we've given it. So we know that it's obeying the general rules, and as we go down and look at the output, you'll see that it's starting to reference two times leverage, greater than 50% control.

And those are all rules that it was given in that large rule set for geopolitics. So we can see that it's applying those in our thinking. We know it's applied the rules that we asked it to and the considerations.

And then when you come down to examples like scenario five, it would breach one of the rules, and therefore it's being eliminated. And so not to get into the detail of whether this is the best approach or not for US semiconductor development, the point is we can see now that it is not only using its reasoning ability, the intelligence that we know the model has, but it's also overcoming perhaps domain expertise or a lack of insight into how these decisions might be made.

But it's overcoming that by applying the rules that we gave it, and we can see that the rules are being applied. So by this point, I'm hoping that it's given you a sense of one other approach. It is an approach I'm still working through.

However, so far, the first results are very positive. It is able to identify the type of problem, and using a larger prompt instead of just tool use gives us a lot more granularity there. We're able to then work with a subject matter expert to give us really tight rules and heuristics to follow, and then it can apply those on top of these powerful reasoning tools and things like the world sim to start to iterate through lots and lots of different options and come up with the optimum solution or give us multiple scenarios.

And again, I would always want a human in the loop. I would want the subject matter expert to be part of this discussion. But obviously, these machines are able to work so much more quickly that a human expert could generate dozens of scenarios to review in a fraction of the time it would take them normally.

And that allows us to either challenge problems or take on more challenging problems, to work through many more options in a finite amount of time where time is at an essence, or even look for new approaches that we might not have thought about.

So if you come across one of these areas, I'm hoping this technique, this approach will help you where you're using the parsing engine, you're using the rules of thumb, and applying those in that problem solving in addition to using all of the LLMs.

And I hope that that helps us move these models into other areas that are currently a little underexplored.