MCP Agent Fine tuning Workshop - Ronan McGovern

Okay, welcome to this workshop on MCP agent fine-tuning. I'm Ronan from the Trellis Research YouTube channel, and today I'll be showing you how to run an agent that has access to tools via MCP servers, and then take traces or logs from the high-quality runs and use them to fine-tune and improve the performance of a model.

By the end of the workshop, you should be able to: generate high-quality MCP agent reasoning traces, save the tools and the multi-turn traces from each of those runs, and fine-tune a model. I'm going to show you how to fine-tune Qwen3, but you'll be able to fine-tune other models too, and then run the fine-tuned model and see if the performance is any better.

Now, all of the materials are available online in this repo here, Trellis Research AI Worlds Fair 2025, and they're in the MCP agent fine-tune folder. I'll be cloning this and working through it very shortly, but first I do want to give a brief introduction to what MCP is, that's Model Context Protocol, how it allows a language model access to tools, and I'll emphasize what we need to understand about those interactions when we're building both the fine-tuning scripts and also the agent scripts and saving of those logs.

So, what is MCP, Model Context Protocol? It is a protocol for providing services to LLMs, namely access to tools. The tool we're going to focus on is browser use, so using an LLM to navigate through websites, but there are MCPs for things like Stripe, GitHub, Gmail, and many more you can find.

You can sub those into these examples today, but I recommend running through first using the MCP tool that I've selected until you're comfortable with how everything works. MCP does a few things. It's, first of all, a store of information on tools.

That information helps the LLM to know how it can make calls to the tools or make use of them. But the MCP tool service also runs the tools. So when the LLM decides to make a call, MCP, the tool service, will take the action, whether that's adding numbers or navigating to a page, and it will then return a response which contains details of the result, or maybe some extra helping guidance for the LLM, so the LLM can then loop back and make another tool call or provide some kind of text-based response.

Now, to bring it to one more level of detail, we're going to wrap the language model here, or we're going to expose it in the form of an API that we can hit, and we're going to expose it as an OpenAI endpoint, which is pretty common.

Of course, it's the one OpenAI itself uses, but using that library, there are many models and many libraries that will make use of that style of API endpoint if you want to set up an endpoint for your language model.

So this is what we're going to use. And in integrating this API endpoint, there are a few points of integration or translation we need to take care of. Don't worry if you don't understand this first time around, because it'll become apparent in the code.

The first one is, we will get tool information from the MCP services, and we need to convert that into lists of JSON tools, because that's what OpenAI endpoints expect. Second of all, we will need to convert the tool response into a format that the language model expects.

And lastly, when we have the language model calling a tool, and it will call it just by emitting tokens or text, we need to extract, detect, and extract whether it wants to make a tool call or not. And the form that the text will take when it wants to make a tool call is going to be in Hermes format, actually, even though it's a Qwen model, and I'll talk about that in just a little bit.

To give you a better idea of how all of this fits together, it's helpful to look at a prompt. We can take a look at this little chat I've prepared. It just shows a little pseudo prompt. It's the string that is being sent into the LLM.

It starts off with a system message, just after a system start tag, and it starts by describing to the LLM how to make tool calls, namely by passing them as JSONs within these tool XML tags here. And the LLM is being told, "Okay, you have access to these tools, there'll only be one in our case, which is the browser, please return each function call as JSON inside tool call tags if you wish to call a function."

So we're telling the LLM, "Here's how to call a tool by giving us these XML tags and then the JSON." After the system message, we have the first user message, which would be the first message, like navigate to trellis.com, and then the assistant will respond, and it can respond in a few ways.

Typically, it might do some thinking here, because it's a thinking model, and then decide to call some tool, like navigating with the browser to a certain URL, or if it's finished navigating, it might decide to think and then just provide an assistant response as text.

So this is a little pseudo prompt here, and we'll see this come into play in the code when we want to set up and inject these different parts. Okay, so that brings me to the end of my very quick overview of MCP and the prompt structure.

If you're not following fully, that's okay, because it'll become a lot more clear when I run the examples, which I'll do now by cloning this repo, AI Worlds Fair, into Windsurf. If you clone it, you want to open up then the MCP agent fine-tune folder and go to the README.

Now, in this README, there are two sections. There's the data collection section, where we run the agent to generate these sample runs, and then there's the fine-tuning, which will be done in a notebook. So starting off with the data collection, we need to make sure we have an endpoint.

We need to make sure it's an OpenAI style endpoint, and if we're not using OpenAI itself, which we won't be, because I'm going to generate the traces using a Qwen type agent, because I'm going to fine-tune a Qwen model later, I do recommend keeping some consistency between the model to generate data and the model you're going to fine-tune.

In principle, you could just use a stronger model, any stronger model, but we want the stronger model to share its reasoning traces so we can train on those, and OpenAI models don't share their thinking traces, so we're going to use a Qwen model.

Specifically, we're going to use the 30 billion parameter one with 3 billion activated parameters. It's a mixture of experts model, and I'm going to run it on RunPod. I'm going to use this one-click affiliate template here. You can run it, though, on any service.

In principle, I think you could run it locally as well, provided your local app supports tool calls. I'd need to look into whether LM Studio does support that. But if you click that one-click link, you'll pick a GPU, maybe a H100, and you can inspect then the template.

So a few things here. We're running a Docker image for VLLM. We're running this Qwen model. We're going to enable reasoning and a reasoning parser. What this means is that when we see a response that includes think tokens, we're going to detect those and extract it as reasoning into a JSON type style in the response.

Next, we're going to set the max model length to 32,000. We'll set the port. We're going to host it on port 8,000, and we're going to enable automatic tool choice. This means the LLM will decide whether or not to call a tool and which tool to call.

Here we have the tool parser that we're choosing. You'll remember in my presentation, I said we need to specify how to extract the tool into JSON format. What this means is when the LLM makes a tool call like this, we're saying, "Okay, the tool call is in this format, Hermes, extract it into this JSON here, and provide it to the MCP in a format it's expecting."

So this is specifically this point here where it converts from the language model string into a JSON format that OpenAI, the API, expects. Okay, so these are all of the arguments we'll use, and then we'll click here to start the server.

We do need to expose port 8,000, because that's where we're serving it. We're going to hit that port and server from outside, and the way we're going to hit it, by the way, I would click set overrides and start the server.

I actually have a server that's running already here. It will take about three minutes to start up. And to hit the server, we need to specify the RunPod pod ID and also the port number, which is 8,000. So you'll see here, when we run this agent, and I'm going to start just by running it without explaining it, because it'll be more intuitive when you see an example.

I'm going to run the agent with this model here. I'm going to run it with this base URL, which has got the pod ID and the port number, and it's got the v1 that is required for the OpenAI style endpoint.

And there's one more argument here I'm using, which is truncate. This is truncating the length of the tool response. So when you have a tool response, for example, here, there's a tool response coming back. If we call a browser navigate tool to navigate to trellis.com, it's going to return an accessibility tree, which is a text description of that page, and that will be provided back in the tool response so the language model can read the page.

But this could be really long, and so I'm just naively truncating it, so we just take the top part. Now, bear in mind, that means that the LLM won't probably be able to see the full length of the page, and you would need to do something more complex in a deeper implementation of what I'm showing.

So we're going to run the agent here, and first run UV sync, just to make sure you've synced the requirements. If you don't have UV, do pip install UV, and then you can run the agent here. So you'll see the agent has started up.

We've started the MCP server. This is the only server that's configured, and it's configured in the config file here. You can add in other servers here, and it will load up those tools. Playwright offers 25 tools, so there's quite a few, like I think navigate forwards, backwards, switch tab, navigate to link, different things like this.

We'll see more of them later. And those 25 tools are all available to the LLM. For open source models, probably you don't want more than maybe 25 to 50 tools. I think for Claude, you can probably handle 200.

More than that, the LLM will get confused by a lot of context. So now it's asking me to put in an input, and I'm going to say navigate to trellis.com and read out the top two lines. Let's see what it does with that.

It should start to think now. That user message has gone to the LLM. It should be thinking now, generating thinking tokens. They won't be shown here, just for brevity. I'll save them later on. And then, yeah, it's asking for a tool call to move and navigate to trellis.com.

Okay, and actually, before I say yes to that tool, which requires my approval, I'm just going to put this screen on the side. So I leave space here, because the browser is going to pop up. And the reason the browser will pop up is because I'm running this browser.

I'm not running it in headless mode. If you want the browser to operate in the background, you would just put in -headless. But I actually want to do it, because I want to see the browser and what it's doing.

So I'm going to ask this question here, which is navigate to trellis.com and read out the top two lines. Okay, so I'm expecting the model to start thinking here. The tokens won't be printed, because I'm saving them just to avoid clutter on the screen.

And then, yeah, it's decided to call a tool, which is trellis.com, navigate there. I've built in approval to the agent, so I'm going to say yes. And now it has opened up that browser, which is good. And the structure, the accessibility structure should be sent now to the LLM.

And it is picking out the top line here, and it's also picking out this second line here. So this looks really good. I'm just going to exit, and that was our first quick demo. So I've got a few little demos here.

I'm just going to clean up two demos from before, where I didn't have my screen set up. And you can see the logs here. So I've got a feature where by default, when you run the agent, things are going to be logged like this, with two parts, with the messages part and the tools part.

And this is exactly what we need for fine-tuning. We need to know a list of the tools, and we need the full conversation history. So you'll see in the tools a list of the 26, browser close, browser resize, etc.

And under messages, you can see my user request. Then the assistant starts thinking, and yeah, there's quite a bit of thinking. User wants me to navigate to trellis.com, etc., etc. So it goes through all this logic, and then it decides to make a tool call here.

The tool call is sent to the MCP, and then Playwright basically opens that up, as you saw, and sends back the accessibility trace, which includes details on the page, but truncated. And then the assistant takes that and says, "Oh, I have the information I need.

Here's some reasoning, and here's my final answer." So this is a very nice trace, and we're going to keep it. We're going to keep it, and we're going to use this for fine-tuning. Now, it wouldn't make sense to fine-tune using this on the same model, because the model can already do this, but it can make sense to fine-tune a smaller model that doesn't have this kind of capability.

So your goal now is you should set this up, and you should run this, and I want you to run it multiple times and try to get some good traces. And then we'll save those traces, and we're going to upload them to Hugging Face Hub.

Okay, so we'll run one more example here. I'm going to make it a bit more complicated this time, and I'm going to say navigate to trellis.com, and then find the third product, navigate to that page, and tell me what is on.

You know what? Let's try and ask a specific question. So let's go to trellis.com and manually do the navigation first. Oops. Oh, I can't type. Here we go. trellis.com. The third model is training and fine-tuning. We'll navigate there.

It opens in a new tab, which, by the way, is tricky for navigation. And let's ask it what the first line is. Yeah, let's ask what is in the feature section. Okay, we'll try that. So the model should be thinking.

We should see the browser pop up. In a second, once we approve that navigation, it should get a snapshot, and we'll see what it decides to do next. It should try to do this dropdown menu here, which is a bit tricky.

It is clicking on something, so that's good. It did get the dropdown menu. That's also good. And okay, it's actually just given an answer, which is not exactly what we want. Yeah, well, the answer is correct. It says, "Do you want to click the learn more section?"

Yes, click that, and then navigate.

And then make sure to switch tabs, because it will open in a new tab. So yeah, I mean, this is kind of working out well. There's always going to be noise, so some of the traces won't be good.

And I'll show you some ways to deal with that. One is to just manually adjust the trace. Allright, it's going to click on learn more, and that has opened a new tab, which is good. And now it's going to think.

It's thinking in the background. We'll save the thinking logs. And it's asking to select that tab too, which is this tab. And I think at this point, it'll get a browser snapshot, but I'm not entirely sure of that.

And let's see. So

the core features are GitHub repo access and ability to post issues. So yeah, this is correct. It's getting the correct content. So it's actually fulfilled this task very well, and I'm going to exit. Now, before I show you the trace, there's one more trick.

If it's not following what you want it to do, you can kind of guide it like I did in the user prompt and adjust retrospectively. But you can also pass in a system prompt. So you could pass in a system prompt like this that literally explains very directly how it should do this task and what tools to call.

And this can also help performance. The idea is you get it to create a nice, tidy trace, and then you don't have to include the system prompt in your training data. You just train on that nice trace, because our goal is to get nice traces for training data.

So I'll just show you that, and then you should go yourself and try to make some of these different traces. Let's find that more complex one which I just made. Let's see if I want to make some adjustments here.

So we have the tools that are saved, and I'm not going to change those. Butright here, when I say, "Make sure to switch tabs,"

let's just expand that thinking. I need to find, I'm trying to find the section where I said, "Yeah, click that." And yeah, okay. So one thing you could do is literally delete that user turn

and combine kind of these two sections here. So you could kind of do that manually, or just literally copy-paste it into ChatGPT and ask ChatGPT to generate just a single version. You would need to delete the assistant content, just provide reasoning.

But another approach here is just to say, "Yes, click that," and proceed.

And instead of saying, "And switch to a new tab," I may then need to switch tabs if it opens in a new tab. So you can see I'm cleaning up the trace here. And yeah, this is some nice data.

So we have a few data points that I've saved, and what we want to do now is push this up to Hub. So we want to create a data set that has the tools and that has the conversations that we use for fine-tuning.

So for that, I'm just going to run push to Hub. And quite simply, it's going to grab the tools, and it's going to grab the messages and push them up to a data set. Now, there's one subtle point here that I'll show you, which is called unrolling the data, because we want to train on multiple turns, not just the first response, but the second and the third.

So actually, if we have, say, three back and forths, we want to unroll that into three roles. One role with all of them, one role with just two, and one role with one. So actually, we get kind of three for the price of one in terms of training data here.

And there is a subtle point that the template of Qwen will always include the reasoning from the most recent turn. So even if we include reasoning and save it for all of the turns, it will only inject the most recent reasoning.

And that's another reason why doing this unrolling is important. Ask me a question in the comments if you want more clarification on that. It's kind of a subtle point. If you want to push to Hub, you can just look up in the README.

There's a function here, push to Hub. You need to specify the repo ID and then pass in unroll if you want to do that unrolling. And this data set should get pushed up. Now, I do need to log into Hugging Face.

So I'm just going to do that. So I've put in my token. It needs to haveright permissions. And then I can run push to Hub again. And it's a pretty small data set. There's nine examples, so it's unrolling these four into nine.

One of them has three, the others have two. Qwen unrolled, and then I can check out my data set, which has got ID, timestamp, the model, the messages, and then the tools. And you can see, I think there's also a flag whether it's been truncated, which means unrolled.

So say this last example.

Well, it's not the last example, but there's an example here where we have unrolled it twice. So the first example here, second example, third example has got two, it's got three turns, so there are two unrolls. So you'll see if you look at the messages here, this will give the full three turns.

So it'll be fairly long. It will have all of the messages. And notice how the reasoning content is extracted. That's because we've turned on the reasoning parser. But if we scrollright down to the bottom of this, and I know it's kind of long, that's because the tool responses are long.

You can see the last response is the answer, or at least it should be. Yeah, it's saying what's on that page. Whereas if you go up to the one above that, where it's been truncated, it should just be a tool call, I think.

Or it may actually be that assistant response we got. Yeah, it's got some reasoning content, and then it's making a tool call down here, where it's deciding to select the tab. So you can see we've got nine rows of data here, and we will grab these messages, and we will template them along with the tools in our fine-tuning template when we want to fine-tune the model, which we're going to do very shortly now.

Okay, so at this point, you should be able to go ahead and run this a few times, try to modify some of the traces, try to push some of that data up to Hub. I will maybe very briefly show you the agent.

Let me see how I can do this quickly. I'm basically setting up an OpenAI endpoint. And what can I say that won't be too long? I need to set up the server. So I have server management here. Any MCP service that I want to run, I need to boot it up when I start the agent.

So that's one of the first things. I need to do schema conversion then. So I talked about how you have MCP. That has a specific syntax, but OpenAI expects the endpoint, expects a different syntax. So there's conversions that need to happen.

There are some kind of helper functions, but then here's how we convert the tools from MCP to the OpenAI format, and then we need to go OpenAI to MCP. So we need to go both directions. Here's where we execute the tools, and here's how we wrap the result in a format that's going to be suitable for OpenAI.

So that's all those connections I showed you in the PowerPoint. This here is discovering all the tools, then booting up the servers, and this is the chat loop, which you saw operating below. There's not a whole lot that's to explain beyond what you saw me doing.

You can see reasoning has been parsed separately to the assistant content.

And we have a helper here for trace logging, just to save the details. And yeah, we have some logic here for doing the logging, and then some different arguments that we can pass. So I'll let you dig on that in your own time.

In the meantime, let's move to the fine-tuning step. So we've got these traces, and we want to use them to improve the performance of the LLM. We'll go to the fine-tuning section and open up this notebook. I have it open here.

My runtime is disconnected. I'll just restart. This is based on an Unsloth notebook for fine-tuning Qwen, but it's got some things that I've changed. First thing we'll do is install Unsloth, which has got a bit more of a detailed install if you're on Colab.

And then we'll load a model. Now I'm going to train the 4 billion parameter model. You can train a larger model if you want better performance. The max sequence length needs to be fairly large. If you want to train on the free GPU, you probably can't train a larger model.

You'll need to run maybe on RunPod by starting up. You can find a fine-tuning one-click template, actually, in the OneClickLLMs repo. You can start that up and then open Jupyter and then download the notebook here and upload it.

We're going to run it in full precision in 16 bits. And once we've loaded the model, we'll want to set it up for training. We're actually not going to train all the parameters. We're going to only train adapters that we connect to certain parts, certain matrices.

They're called low-rank adapters or LoRA. And I'll talk about applying those adapters in a moment, but I want to actually get a benchmark on performance without fine-tuning. So I'm not going to apply the LoRA adapters just yet. I'm going to prepare the data, and then I'm going to run inference without having done fine-tuning.

And the data we're going to load is that data set that we've been looking atright here. So we load it up, and we'll just print it out to see how many rows. It should be nine rows now. If you want, you can take a look at the format of the messages or the tools just by running these to inspect the zeroth row.

You can also take a look at the chat template. The template is what takes in the tools and the messages and puts it into a single long string. In fact, it's probably worth looking at. So we'll just run that.

You can also find it if you go to the model. You can go to the tokenizer config on the model on Hugging Face, and you can see a copy of that there as well. By the way, my runtime restarted, so I need to redownload the model weights.

They're just downloading here. It'll take a few moments. And yeah, as I said, we take a look here at the data set. Sorry, I've got a really long data set row saved. Once we have the data set downloaded, which means we're going to have access to the messages and the tools, we basically pass them into the chat template, which will make a single long string.

And that single long string is basically this. It's the formatted data set as a single long string, and it is very long because it contains all of those tool responses. But you can see it starts off with a system message.

It starts off with those tools. It then gives a list of the tools available, which is very long. There's 26 of them. Then it'll give the user message, the assistant message, the tool calls, etc. So that's what it's going to look like.

When we rerun this, we will have nine rows, and then we'll just print out to check what the length of the longest row is, because we don't want it to be longer than 32,000, which is our max length.

So we'll do a check there. Now, we will train the model, but I said first we'll just do a quick run to see how performance is without training. And for that, we'll just run on the first row of the data set, and we will see if the model decides to call a tool correctly when I ask it to navigate to trellis.com.

So yeah, if we print out, yeah, if we print this out, we should start to get it to stream a response. Here's one I streamed earlier with a fine-tuned model and it correctly called a tool. But if I call this again now, let's just quickly check the length of the data set.

Oh, the data set is not defined. Why is that? I don't think I ran all of my data set rows.

So let's just check. Downloaded data set. All these rows will run quickly. It's a quite small data set. Yeah, nine rows. The longest is 18,000. That would be fine. And yeah, you can see if I print out the zeroth message, I'm asking it to navigate to trellis.com.

And if I run inference on that, it should just start to stream its output string, and we'll see whether it decides to call a tool or not. So we're just testing here the raw 4 billion parameter model, which should be capable of tool calling.

This is a lot of tools for a 4 billion parameter model, though. 26 tools. So, you know, it's going to blab on here with a lot of thinking, and then we'll check whether it gives a tool call. It may or may not.

It may decide to blab on for a full 2,000 tokens and then just stop when it hits the limit. So we'll come back to this. But what you're going to do now that this model is loaded is you're going to prepare it for fine-tuning by applying LoRA adapters.

And we're going to apply LoRA adapters to the attention modules and also to the MLP layers. We will use a rank of 32, which means the adapter matrices will have effectively a width or height of 32. We'll use rescaled LoRA, which will adapt the learning rate of the LoRA adapters according to their size.

Larger LoRA adapters means lower learning rate. So I'll go ahead and apply that. The data should all be prepped, so we can goright to training the model here. We're going to pass in the data set, and it's going to train based on the text field, because we've prepared that text field.

I went through this quickly, but if you recall when we formatted the messages and the tools, we templated them into one string of text. That's the text key there. And when we did that, we essentially prepared it so we could pass in just rows of text to our trainer.

Batch size of one. This is not ideal, because you don't get much smoothing. It's going to make it very jumpy in training loss, but we don't have enough VRAM here. If you run on an A40 or on an H100, you'll have more VRAM, so you can use a larger batch size.

We just trained for one epoch. We actually shouldn't have warm-up steps, because we don't have enough steps here at all. If I have nine rows, I'm only going to have nine steps with gradient accumulation one and device batch size of one.

Learning rate is fairly high, because it's a small model. Using the AdamW8 bit optimizer to save a bit of VRAM, and we'll use a constant learning rate. Everything here is quite simple. I'll probably do a follow-up video with some more advanced techniques on the YouTube channel, on the Trells Research channel afterwards.

So keep an eye out for that. But I wanted to keep this fairly simple, so we can go through it quickly. So yeah, we'll run the training. That's actually just setting up the training. We'll check the device memory.

It's got 15 gigabytes. 10 of it's reserved already, and we can start to train. After training, we'll show the final code, and then we'll run inference again. Now, let's just take a look at what it did without training.

It actually pretty good. So on this basic task here, it more or less got the problem correct. Would it be able to read out the top two lines? Probably it would, even with the 4B model. But I can tell you that if you do a more complicated task like multi-step, it will struggle.

It already struggled when we did the 30B model, as I showed you. It was kind of lucky I got it to work when I did. But this 4B model will definitely struggle on multi-step. So after the training, I'll rerun that, and we can see what it does.

Hopefully, it'll also get itright. You can see the training is underway here. We've got nine steps. We're only training 1.62% of the parameters, so those adapters are tiny. Everything else is frozen. All the main weights are frozen. And it'll take a little bit of time for this to complete and then inference to run.

And when that's done, we have the option to save the model in the tokenizer. We also have the option then to push the model up to HUB. If you want to do that, you can just log into Hugging Face here.

And then I recommend merging it to 16 bits. If you merge it with this name here, for example, you can actually go back into your pod. You could edit the pod and just drop in here the name of your fine-tuned model, and now you've got an inference endpoint ready to go.

So let's see how our training is doing. About two minutes left, and our training loss is falling. In a more advanced implementation, you would want to have an eval set. So you want to have much more data, not just nine traces.

You want to have probably a few hundred. You want to split some of those off into an eval set. You also want to do logging with TensorBoard. I go through that in most of my videos, but I wanted to keep this fairly simple.

Okay, so we have the training completed, and generally the loss is kind of falling. It's very bumpy, and that's not quite completed, by the way. It's just finishing up. But it's not helped by the fact that we only have batch size of one.

Ideally, we'd have batch size of like 32 maybe, which would require quite a bit more data. You could use gradient accumulation, but if you have a very small amount of data, you actually just have to set the batch size smaller.

You don't want to do just one update step with all of the data. So we're kind of doing the best we can here, given the limited amount of data. And this is something you can definitely add a lot more traces to, hopefully.

Now, just one other comment, because I know one question will be around reinforcement learning. Can we automate this so that we generate traces automatically or use some reward-based system? And my recommendation there is start off with manual traces and curate them like this.

Because even if you want to do rewards afterwards, like gRPO, and you can check some of the other videos on that, it's going to be beneficial to SFT, supervised fine-tune, like I am now, on some traces that are high quality.

If you just start with a model that you want to do reinforcement learning on, but you've not done any supervised fine-tuning for that domain, it's going to struggle. It's going to generate lots of traces and very rarely get to the correct answer where it sees a positive reward.

So it will speed up your training a lot, even if you do reinforcement learning afterwards, if you first generate some high-quality traces and then do the fine-tuning. Now, if you want to do gRPO, you need to define rewards.

That means you need to have a data set where you have answers. That could involve picking some random niche websites and finding maybe the text on the top and then asking the language model to navigate to those kind of obscure pages, and then you have a ground truth of what those lines of text are.

That's just a really simple example, but you need to come up with a way to systematically generate data that's verifiably correct and then reward the model when it gets that answer correct. Okay, that was a bit of a sidebar on reinforcement learning, which is not the focus here, but it's very much related.

We have finished the training. The loss, I'd say, generally has gone down. We used peak memory 75%, so we didn't have space for probably doing a batch size of two. And now we're generating response. So the model is thinking, and it's got about 2,000 tokens to think.

Let's see if it finishes up with recommending a tool call. It may or may not. It's possible that with this noisy of a training, we bumped it off. What was a good answer, at least the first time around.

By the way, I'm using default parameters for running inference. You can set them manually here. There's recommended parameters for Qwen for the thinking model. And yeah, it looks good. It's calling the tool as expected. So you'd want to do a much more elaborate setup on the evaluation.

You probably want to run your model on the endpoint and actually try to run some of the rollouts over in the workspace on Windsurf that I showed you earlier. Just in case you want to see what the default configuration parameters are, you can see the temperature is 0.6.

And I think these are the recommended parameters if you're using a reasoning type model. Okay, with that, it brings us full circle. Hopefully, you've had an opportunity to create some good traces and understand now how to do fine-tuning, at least in principle, even if we didn't use all that much data.

This is something you can build on by adding in different MCP servers, curating many traces. And I think it's probably quite powerful, even without moving to ORL, if you've got a small number of examples, and I mean maybe 100 or 50, something that you can realistically generate yourself.

I think you can get significant improvements in performance, particularly if there are some kind of narrow trees that are common but important for your use case. As I said at the start, all of the materials are in Trells Research AI Worlds Fair 2025.

You might also want to check out the two MCP videos that are on Trells Research on YouTube if you want to understand in a bit more detail how those servers are not just set up, but how you would make a custom one by yourself.

In the meantime, any questions, let us know by posting down below in the comments. Cheers, folks.