Agents Building Agents - Alfonso Graziano, Nearform

Hi everyone. Today we will talk about agents, building agents, so how we are leveraging coding assistants and spectrum development to build reliable and secure agents. First of all, just a couple of notes about myself: I'm Alfonso, I'm a tech lead at Nearform.

We are a services company, um, I am currently working on some of our AI agentic projects. I'm supporting multiple teams adopting AI native engineering, and I'm also a, uh, O'Reilly author of the book "Learning AI Native Software Engineering."

So, we are seeing in the industry thatright now everyone wants AI agents for workflow automation, search, you know, every use case that you might have in mind. But AI agent come with a, sometimes, a very high cost, they have hallucinations, they have an entire new set of problems.

Of course, they are non-deterministic. So today we will see how we are building and improving iteratively AI agents. And of course, as you may guess, how can we do that,right? AI is very powerful and very good at building any type of software, and given that AI agents is just one type of software, as you may guess, we are using AI to build AI.

Um, as I was mentioning, there are a lot of problems, a lot of different problem classes with building AI agents. Non-determinism is one of those: latency, cost, hallucinations, a lot of stuff, really. But today we will see how to solve them, at least partially, by leveraging agents and the repeatable process that we developed over time on our projects.

Just as a very, very quick refresher, we can say that an AI agent is basically just an LLM, which is like the brain of the agent. Then the LLM is connected to a bunch of tools, it has access to a bunch of context, and basically lives into an agentic loop.

So that's the, that's the gist of it. Of course, there is way more into AI agents. There is, um, everything around observability, how do we ensure that, you know, it's doing what's, what's necessary. So everything around evals that we will discuss in a minute.

But like, just from the first principles, we can say that an agent is an LLM inside an agentic loop, which is connected to tools and can retrieve context. That's it. Now, there are a couple of classes of problems that we will analyze today.

So the first one is bad performances on the evals, and we will see what the evals are in a minute. And the second one is bad performances on live data, which sometimes is similar to what we have on the evals.

But you know, live data coming from real users with, you know, real expectations from the system can be a lot wider and sometimes a lot messier as well. So let's analyze the first, um, the first failure mode. So we have bad performances on evals.

Before we analyze this failure mode, I just want to introduce you to the context of a golden dataset. Basically, a golden dataset is a file or a set of files that we develop together with the subject matter experts when we are building our AI agents.

And basically, this file defines what is the input that the system should retrieve, um, and should get, sorry, and what is the expected output that we should get. Now, in the naive case, the expected output, as we can see here on theright, can just be like impossible or like a number or a text.

So it can be a lot of things. In real-world scenarios, the expected output can be, for example, "I do expect the system to call this tool," or "call this tool with this parameter," or "call this tool in this chain,"right?

Because maybe I want to do a retrieval and then do an update. You know, so how we define the output is very different in every, in every agent. And the idea is that we want a way, basically, to ensure that our agent is working correctly.

So that's why we are building the golden dataset. You can see the golden dataset as a test suite, but in a non-deterministic scenario. So when we build the golden dataset, we also have to build a scorer or a set of scorers that can basically, um, go through the golden dataset together with the LLM, and then can give us basically a number in the end, which is saying what is the current accuracy of the, of the system.

So that, you know, we have a baseline, we can look for regressions, and we can further improve the system over time. Now, let's, let's assume that we have a very simple hello world agent. In this case, I'm using Mastra.

It's a very, very simple agent, as we can see. I've, I've called it mad agent, but in reality, in the golden dataset, we have multiple types of questions,right? But let's assume that this is our very simple agent,right? So it has an aim, it has just a couple of instructions, has, has a model, of course.

As you can see, it doesn't have any tools yet. So it's like really the bare minimum. Um, we also built a naive evaluator. So if you remember from the golden dataset, we only have inputs and outputs. And what this very naive evaluator is doing is checking whether the, um, LLM output contains the ground truth output that we are expecting.

Of course, once we run the eval suite here, the results are very poor. So as we can see, um, we have a pass rate of 18%,right? Just because a lot of questions are simple enough, like additions, multiplications,right, are simple enough so that the actual, um, LLM has this knowledge in its training data, and it doesn't need access to any external context or like any particular tool to answer these questions.

So 18% of the questions can be answered by the weights of the LLM, the rest can't. So we need a way to actually improve this pass rate.

Some failure modes that we can expect are, well, first of all, the agent is missing theright tools,right? So let's assume that I expect from my agent to be able to surf the web, fetch web pages. If I'm not giving those specific tools to my agent, of course, every eval that is expecting that, it's going to fail miserably.

So first of all, we are maybe just missing some tools, or we don't have theright system prompt. Because the system prompt basically contains all the instructions, all the behaviors that we should expect and that we shouldn't expect from our agent.

So in a lot of cases, um, a lot of the optimizations will be just to tweak the system prompt and update it so that, you know, our agent has all the information it needs to work on our domain.

And then, um, of course, context retrieval, which is maybe our agent doesn't know where to look for information. Of course, usually context retrieval is implemented as tools,right? Um, maybe it's, I don't know, an hybrid search in a rug system, maybe it's a fetch on the web, you know, all these kinds of things.

So we need to give enough, um, let's say, tools to our agent so that it can retrieve all the context it needs, of course, in a secure way. Now, the question is, can an AI agent improve another AI agent autonomously or semi-autonomously?

And, um, Karpathy, Andrej Karpathy, which is one of the most well-known AI scientists on the planet at the moment, implemented something which is called auto-research. Basically, auto-research is a loop which, um, updates the code, updates the hyperparameters, updates the, the actual Python code of machine learning algorithms.

And it actually shows that a coding agent tweaking the code of a machine learning, um, of a deep learning algorithm can actually improve the results,right? So that's the, um, that's, that's the loss that we can see. And basically, every dot here in this chart is an example, is a test that the system is doing, um, by changing some of the parameters.

And we can see that the baseline has a specific accuracy. And while we go, um, while we go with the number of experiments, accuracy is improving, and therefore, of course, the loss, um, the loss is decreasing. Now, the question is, can we do the same with AI agents and not just like machine learning models?

The answer is yes. So I built something which is called auto-agent, which is basically the same idea but applied to, um, actually like AI agents,right? So basically, this is a loop which is able to run the evals and then, um, update the code of the system, try new system prompts, create new tools, doing everything autonomously, and then check again whether things improved or not.

And it actually works quite well. So here we have a few iterations, and this is the AI agent, this is the naive AI agents that, agent that we have seen earlier. And in that agent, as you can see here on the bottom left, um, we have the baseline accuracy, which was 18%, and we managed to reach up to 83%, um, in like something around 10, uh, 10 iterations.

Now, of course, that's a, um, that, that's a very naive case,right? Because we started with something which didn't have any tools and nothing else. So it's relatively easy for the system to improve this agent. But, uh, we also improved, um, some evals by 10% on a production agent that was already humanly optimized.

So actually, the, the machine, uh, the, the agents, the, the coding agent found new ways that humans didn't find, um, to improve the agent, and we got plus 10% on some of our internal benchmarks. Now, how does it work?

Actually, I just, I gave you already like a few insights of how this works, but let's see it in details. So basically, the core idea is that we have a coding agent, in this case Claude Code, but it can work with multiple, uh, coding agents.

It builds the agent, so it writes the code of our target agent, and then the target agent is giving feedback to our Claude Code or whatever, uh, coding tool we are using, um, as evals,right? So it's giving some information as, "Okay, here I got some regressions," or, "These evals are now passing."

Um, also, Claude Code can read the traces, the thinking traces, or like the full trace of the target agent to see whether something broke or something is not working as expected. So it can do like self-improvement. Of course, there is the human in the loop.

Uh, and the human in the loop is usually in structuring the initial agent and giving as much context as we can to, um, our coding agent about what it can do, whether the relevant files, uh, what it cannot do.

Like, just to give an example, updating the golden dataset or the scorers, uh, just to let the evals pass is not a good idea. So we want to enforce, we want to tell the, um, we want to tell the AI agent to not do that,right?

So as humans, we can, um, we can steer a little bit our, our coding agent so that it can optimize the target agent. Now, there are a few steps,right? The first step is to create an optimization job. Of course, it's very easy.

Everything is a Markdown. Uh, we all love Markdown now. So we have, we can define the objective, we can define the target repository, the metrics, and literally everything,right? So in this file, we can give as much context as we want to our coding agent so that it can actually, uh, run and improve the target agent.

And then the second step is to run the loop. So basically, we run the evals once. So we, we, we generate our baseline data. So we understand what's working, what's not working. You know, the system will generate the first report, and then we run the loop, the optimization loop.

So we can see that in the first case, uh, we get a regression. So basically, the system works by creating a hypothesis. So it's tackling one class of problems at a time. It's updating the, the agent, and it's running the evals again.

And then it's seeing whether we got a regression, an improvement, or something else, uh, happened. And so the agent can decide whether to roll back the change or move on. So in this case, in the first iteration, we got a regression, so we want to roll back.

Um, then we see a 5% improvement, which is great. So in this case, we are getting, uh, we are creating a new hypothesis. We get a 0% improvement, unfortunately. But of course, the agent is keeping track of all the hypotheses.

Uh, and so in this case, can go and create a new one, and we can see a 12% hypothesis, and, and so on and so forth,right? Of course, we define the number of iterations that we want to run.

Um, of course, the way we create the baseline data is, you know, we just, uh, run the evals once, and we generate basically a document. And this is super helpful because it gives the coding agent the information about the current status of the system.

Then this generates a baseline report, which contains, you know, all this information. Uh, it contains what are the cases, it contains a summary, what's working, what's not working. So basically, um, our coding agent, it's going to have a clear picture of what's happeningright now.

And so it will try to understand, "Okay, how do I improve this system from this moment?" And so from now on, we can start with the first iteration. So every iteration starts with a new branch. We create an hypothesis, um, the system changes the agent to implement that hypothesis.

We run the evals, we run our eval suite, and we generate a report.md file, which contains everything that happened after we run the evals. We update the memory file, uh, which is like a global memory file across all the runs.

And if the metrics improved, then we continue from this branch. Um, if the metrics didn't improve, or we have a strong regression or something bad happened, uh, then we roll back to the previous branch. Of course, um, the generated hypotheses are based on what the agent reads when it starts the investigation, which is the memory file, other reports file.

So it has access to pretty much everything. And that's what happens when we run multiple iterations,right? So let's assume that in the first case, I create a branch, I create an hypothesis, uh, I run the evals, everything goes well.

So I get an improvement. So I am fixing one failure mode. Um, therefore, I continue from that branch. I create a new one so that I can keep track of every change. Um, I create a new, that new branch.

I try a new hypothesis. This time, it didn't really improve. So maybe I get a regression. Maybe things didn't improve really. Uh, so what I do is I roll back to the previous branch. So the first branch where we had a real improvement in the system.

And then from here, I create a new branch, and I try a new hypothesis, and so on and so forth. At the end, what we are building is a full change log of all the changes we have done, um, every improvement in, or improvement or regression, as we can see here on theright.

Um, we are seeing all the hypotheses that we are doing. And of course, if we want, we can also see all the hypotheses because every hypothesis is building a report. So let's assume that something didn't really go well in terms of evals.

Maybe the evals didn't improve after an hypothesis, but maybe that hypothesis was promising,right? Maybe the agent was onto something, but it just didn't implement the system, um, the, the change in a correct way,right? What we can do is, as humans, we can just go back into the hypothesis, read, understand what the agent was trying to do, and then maybe steer it in theright direction next time.

Of course, um, this is the real task on a, on a real agent that we did have. Um, and as we can see here, the baseline accuracy was 67%. Um,

but then in something around 10 iterations, we managed to reach 86% in our evals without actually cheating because it found edge cases. It improved the system prompt. Uh, it improved the tool descriptions to catch more edge cases, and it also fixed some tools logic.

So that's a real agent that is now running in production. Uh, and actually, this system found ways to improve it more and more. Now, the second class of problems that we are going to tackle today is not the evals data, but bad performances on live data.

By live data, we mean data that we collected by our real users, beta testers, subject matter experts, you know, all the people that are actively testing and using the system, and they're giving us feedback. Just as an example, you can see here on theright, was this response helpful?

And you got yes or no, and then you get a note. And basically, we leverage this data to, uh, further optimize our agents. Now, how do we do that? Uh, this is the full flow that we will see in detail.

So first of all, of course, we will need our users to actually use the system. Uh, then we collect all the traces,right? So we collect all the information from our users. Our users can either give us feedback that we've just seen, so thumbs up, thumbs down with the, with the comment, or, uh, our subject matter experts can annotate the trace.

So basically, they can go on the platform that we are using. They can look at the trace, look at the input, look at the output, uh, look at basically like all the tools that have been called, uh, look at the agent behavior, and then annotate the trace by giving us feedback about how the agent performed and how it should have performed.

Once we collect a, a relatively good number of traces, once we have, um, enough information, we basically run an agent workflow that we built. So basically, we analyze all the traces with both the negative and positive feedback, but we are more interested about the negative feedback here.

So we analyze all those traces. We do clustering of the failure modes,right? So based, based on all the feedback that we collected, we try to understand, "Okay, given all this feedback, what are like the five, six, seven failure modes that we have here?"

And then we analyze these, um, we analyze these clusters of failure modes with our subject matter experts. So we say, "Okay, uh, based on those 10 traces, we can note that the agent in this specific case performed poorly because of reason X," for example.

We validate all this with our subject matter experts, and then we do have a fix, uh, proposal, which gets generated and implemented by our coding agent. Then we implement this, um, and then we ship it to production to see whether it actually improved on real data.

And of course, we use the traces that we do have as regressions. So we test our fix against our traces first of all. But let's see that in detail. So first of all, the user tests the system. So in this case, we have a very simple example, uh, how can I optimize a React component?

Uh, here we have the answer. Here we have, you know, the information about the component, whatever,right? So that's the first step. Second step is, uh, we are going to collect all the tracing, um, information. So we are collecting everything from user interaction, um, the tool usage, the, uh, how much time the AI took to respond, you know, how many tokens we burned, everything.

The second step then is, uh, the user, once we collected the, the user behavior, we do expect the user to give us some feedback. Um, as I was mentioning, thumbs up, thumbs down, uh, and then a, a comment saying what went well, what didn't, um, and what's the expected behavior of the system.

And then, of course, uh, the other option that we were discussing is that the actual subject matter experts can annotate the trace. So in case I don't have feedback from the real user, I can ask the subject matter experts to take a look at that trace, invest their time, and give us feedback as they were users.

The third step is now to collect all the traces with feedback and download them locally in a JSON or whatever format you're using. As you can see here, in this case, we have 114 traces, and we are using those traces to then do the cluster analysis.

How do we run the analysis? Well, this is actually a skill that we built. Basically, the skill, it's going to instruct the coding agent properly to go in the, go in the JSON file, look at all the traces, run some clustering.

Um, then we do an adversarial review of that. So basically, um, once we have all the clusters, we do also a little bit of root cause analysis because, of course, our coding agent has access to the actual code of the AI agent.

So it can do some root cause analysis. It can try to understand what happened, uh, especially because it has also access to the details of every trace. So it can go within the trace, understand why the agent answered that way, you know, trace it down to the root cause.

Um, and then what's going to do, it's going to, um, propose possible improvements. And what we get at the end of this process is a full markdown report, which is very helpful for us. Uh, it's going to give us like the positive feedback, the negative feedback, the negative rate.

You know, we are just trying to collect as many data as possible. And so we get this very complete, um, executive summary and this very complete report, which tells us what are like all the clusters which are not working as the user was expected.

So in this case, uh, we have that, we have a small markdown formatting failure, which is saying URLs are not hyperlinked and there are some formatting inconsistencies. So it's linking some of the traces with the trace ID. Uh, it's mentioning the user feedback, is showing like some root possible root causes, and it's telling us, "Okay, how do we fix it?"

Right? Once we have this full report, uh, our work is not done yet because this report will contain all the possible failure clusters reported by users. But now what we have to do is to triage everything and validate with subject matter experts.

Once we are doing, once we do the triage and the pre and the validation, then we prioritize and we understand what we want to fix now, fix later, don't fix. Um, and so either we, uh, fix it with the agent.

So we give to the agent, you know, all the context. We give it the failure mode. We give it the, uh, traces, and we ask our agent to fix it, or we discard it because sometimes it can be the, the failure cluster can be a false positive or can be an intended behavior.

And maybe, uh, the user gave us, um, a feedback which is not really useful for us at this moment. So, you know, there is a little bit of human judgment here. As I was mentioning earlier, all the failure modes that we are finding during this investigation step, they will become part of the golden dataset that we mentioned earlier.

And the eval suite is updated to spot those regressions. So in case this failure mode, for whatever reason, is introduced later on in the code, we will spot it very fast because now it lives into our golden dataset and our scorers.

In terms of how often the report gets generated, it really depends on your use case. Uh, we found out that one sparse print is good enough, but it really depends on how much data you have,right? So if you have 100 traces, it's, uh, you might want to generate at, at 100 traces, but if you have like 10,000 traces with feedback, it might be too much.

So you might want to, you know, generate multiple reports. So it's really on your use case. We found out that one sparse print is actually reasonable. Uh, in multiple use cases, what we found is that a coding agent, while instructed, uh, has been able to fix an entire suite, uh, of, of issues like the one that we have seen earlier, uh, with just one prompt.

So once we give enough context and once we give the coding agent the ability to, uh, test its changes against some regression tests, uh, then everything becomes way, way easier for the, for the coding agent. And then, uh, in some complex cases, uh, we have paired this approach with auto agent.

Uh, and so we basically pointed out auto agent to some of the failure clusters. Um, and we let auto agent build some draft PRs for us. Now, of course, both auto agent and this pipeline to improve, um, agents based on live data, they're possible thanks to Arnes Engineering.

So basically, the core idea is that we are leveraging a set of things so that our coding agent has a powerful enough harness so that it can change the code, validate its own changes, which is like super important, uh, and then propose new changes if the updates that it did, didn't lead to the, to, to the outcome that we were looking for,right?

So just very, very quickly, Arnes Engineering is the idea of building the environment around our coding agent so that they can work reliably. Uh, so we are giving all the constraints, uh, all, all the tasks, the feedback loop.

We are giving the governance to our, um, coding agents, and we are doing that with a few things,right? Which we already mentioned. The first one is spectrum development. So for example, when we are trying to fix some, um, failure modes, every failure mode becomes a spec.

Um, we are creating a spec for the expected behavior of the agent, and then we are implementing that. Then we have quality gates, uh, like linting, uh, our unit tests. We have our, our evals. Um, we have LLM, uh, code review, so a bunch of stuff, uh, everything related to context engineering.

So giving theright context to our agent. And then, of course, observability, um, because we need to know what's happening,right? Um, if we don't know what's happening when we ship in production, we are basically blind,right? So we want to ensure that we know what's happening so that our coding agent can fix any bugs or anything once, once it's found.

So, um, I hope that this has been interesting or helpful. I hope you're going to implement some of those techniques in your agents. And in case you have any questions about this talk, uh, please feel free to reach out on LinkedIn.

I would love to have a chat with other people which are building agents. I know how hard it is. I know how interesting it is. So in case you have like anything that you would like to talk, uh, I would love to get a message on LinkedIn.

So I hope you enjoyed this talk and wish you a great day. Cheers.