Autonomous Agents for Scientific Tasks - Sina Shahandeh, Radicait

Hello everyone, my name is Sina Shahandeh. I— my pleasure to present you this talk about running autonomous agents for scientific tasks. Um, let's— let's dive in. So here, I think everyone is quite familiar with the concepts of Autoresearcher, or Auto research.

This is the original Andrea Carpathy's

GitHub repo where we have an ML model, and we ask a coding agent to find a particular metrics and then optimize the code in order to minimize the error. Basically, do a hell climb over model optimization. Now, for many of these coding tasks, this works very well, but when the problems become very much open-ended and,

sometimes, long-horizon— like most of the scientific task— you have this case where AI agents usually kind of saturate to a certain level. Simply, they— they are very good at implementation of the code, or changing the— or running the experiments over lots of data, and so on.

But the problem is, they run out of ideas, or, you know, what people call them research taste. Now, you can see that in these situations, you know, good humans keep going higher, and the top 1% humans, you know, they keep even improving better and better over time.

Now, the— the really the difference from here is the way that good ideas can— or good hypotheses and how the model could be improved, or how the problem could be solved, keep, um, keep coming— coming up. Humans keep coming up.

So in the scientific task, you know, you have this scientific method where, you know, we observe a situation, we make a questions, we come up with a good hypothesis and come up with a hypothesis of how to solve this problem, and then we do experiments and implement and do the experiment, do the loop, and we— at each of the iterations we learn something and we improve.

Now, the components of the learning component, I think those are all questions of memory and implementation from learning the mistakes, but— which is one of the bottlenecks of using coding agents. But I think this is quite solved by just simply organizing patterns of activity.

I think what is much more difficult is coming up with a hypothesis. So how can we come up with a good hypothesis, good ideas, for our coding agents to keep improving better the process? And this is something— core things that I would like to kind of focus on this talk, and plus a little bonus at the end.

So let's look at a problem here we're trying to achieve as an example, so you have a good— a good idea of what we're trying to do. So here is what we're doing at Radicait: we're building in silico pairs, meaning you have a CT and we want to generate

a pet— pet image, a pet scan from the CT scan. Example of this, you can see here, we have CT scans and of slices of— scans of the— of the— of a patient. They might have a nodule in the— in the lung, and the question is, is this cancerous or not?

Is it a lung cancer? So they do a pet scan, which is a difficult process and very time-consuming to do. But here, we do an ML— through ML model image translation, one can kind of change the modality, learn the structure of the body, and infer what would happen in a pet scan if the hyper— the activity of the tissues.

So, you know, certain tissues absorb more radioactive tracer and they shine up in this pet scan. And the— the tumors, usually that's the case. Now, to generate this relationship, we need— we need a model to— to do the translation, but the problem itself has many components.

So really, this, like any other scientific task, the problem is decomposing that problem, entire long-term horizon, 2 years, 10 years, research process, into steps. And each of those steps is really fundamentally our goal, our— our loop. So I'm going to focus on one of these particular onesright now, and that is on training of a machine learning model.

So we have here, you know, decoder-encoder type of situations.

So encoding the CT and then decoding it into pet. So that's the typical GAN model, which kind of generates the image. Now, for this, we can kind of define these kind of, you know, the architecture, and we try it, we capture data, and do all the 80% of the work to basically bring in the good data set.

And here, we create the metrics and so on, and around the image, you know, fidelity of synthetic pet to real pet, and so on. But the challenge is, how can we improve this situation given a certain initial point?

And we'll go back to our idea of hill climb around this optimization. So you can see an example of iterations coming from a real run in Codex where, you know, we improve the data and so on, and then we— the model goes around and tries to do the optimization.

And you can see there's a range of possibilities, and some of them become dead end, some of them don't improve anything, but we kind of desaturate at a certain point. And then you need— you really need a good idea.

A good idea has to come up. So in this case, we have slices of CT, and we feed these as a channel into the model. So initial problem, initial model that was trained was 2.5D, so treating each CT slice as a— with a 2D convolutions, but stacked over a channel.

Now, if you give this to a typical ML model, they would not think about it as, you know, go through hyperparameters or, you know, some sort of, you know, playing around with problems that it knows, but it wouldn't do a very radical change.

For example, to come up with a 3D idea, convolutions, or change the whole problem upside down. So to create those ideas for— for— for the model to try, I had to kind of, in the midst of the Codex loop, say, oh, what about— what about this idea?

What about that idea? Oh, go read papers out there and see what the papers are saying, what other people are trying. So to induce that hypothesis generation, we need to do something about— about— about our ML model, LLM models.

So this is a trick that I found that works very efficiently, and it's very similar to that chain of thoughts step-by-step problem. First is to decompose the problem into its subcomponents, but it's an explicit action. So, you know, you can ask LLM, actually, go through your problem here.

In this case, you know, create a— translate polyme nodule CT patches into equivalent pet. That's our top problem that I just explained to you. And then it has components into it. So different domain, in this case, you can see, you know, the— the data component, the actual core, the learning, the architecture, the training loss, the, you know, the operational part of the ML modeling, the metrics and evidence, and, you know, the peripheral scripts that kind of run the model.

And data preparation itself is very important pieces. Now, what we have here is this hierarchy of components of this model that's induced. So this itself can be induced very easily using a prompt. So basically, you know,

coding agents can itself go in with this prompt of, go in through this code base and create this series of hyper documents that are linked to each other. Now, what we're trying to do is— is to give our— our coding agent ability to look at this problem in its component where it might not do so, and then induce

a change within those plethora. That's exactly what human researchers would do. The question is, look at what's important, what's not important. So let me show— let me show you this graph here in Obsidian. So the documents that's generated out of this hierarchy generation would look like something like this,right?

So there's an initial README that says, here's a problem statement, the level one, and here is our, you know, the documentation. And then when we go to a next level

of the— of the problem, you see here the level two. Level two itself goes in and has other level three components going on. So one of these pieces, for example, is the model architecture. There's all those kind of components I showed you in the graph, and I can see it here.

And each of these documents are very simple. We just ask them, the model, to document different levels of this hierarchy as it goes in through the process. So here's— you can see the generator. Now it says, is it 2.5D generator and so on, and we go show the details.

And at the end of these hierarchy components is the actual code that is— that is in our system. Now, what we do during— before we actually start the loop, the question becomes, okay, now that you have this hierarchy, now that our model can start thinking step-by-step through the hierarchy and going forward for the actual implementation.

So we ask then the task is, given this hierarchy that you have, come up with ideas that can improve— improve this problem. And you can see here, before, if I just say, here's our code base and here's my objective goal, optimize this process, similar to what originally Carpathy's README file in this program.md, it would not— it would not generate.

It would saturate after a while. But now that we have this hypothesis generation process that can work in an adversarial and collaborative way, now you have a process where you can create, say, you know, you basically create 100— you could create

100— 100 different solutions. So this goes in, and because we have all the components, we actually have a— have a process in which going and modifying every component of this. So it becomes a very much more comprehensive search because you have a scaffold that our reasoning LLMs can go in and make a decision around improving each of these.

And very much you can see here that it can come up with an idea around, you know, creating a three-dimensional structure rather than a 2D. So basically creates a much more radical change that it otherwise would have not come up or— or would have missed.

Now, you can also review this, of course, this plan, with a— with a adversarially and/or collaboratively with another agent or another model to basically loop in and improve this. Ultimately, the idea would be to go through this in— in— in a loop like— similar to like this.

So we have basically, you know, a code base, a metrics, goal achieved, you know, then you hypothesize a required change and then kind of go in and implement. And this loop can go much faster and much better because here we create a better hypothesis.

Now, let me give you another example where we also, in the parts of data preparation, another way to kind of improve this scientific loop is around the concepts of— so you can see an example of it as a registration.

I want to show you how you can use this additional skills, additional kind of collaborative and adversarial

process within your loop. Problem is that we have this case where we have to do image registration. That's another type of optimization loop in the sense that you— you— the idea here is, you can see two process, two— two scans are overlapped here.

So the CT scan, the gray, and then the pet scan is the color one that shows the metabolic activity of the— of the tissue. So here, as you can see, liver is high and the lung is— is black, is not absorbing that radio tracer.

Now, at the beginning, these two scans are not aligned because they're scanned at different times. At the beginning, they look like this, it's off. And through that process of optimization, which we call image registration, we need to align these images together.

This itself has a goal to make a good registration, but there are many different components to it, many metrics, and sometimes the— this is a very long process. It— it can fail in different aspects of this as this, you know, the bodies may not be the same.

Well, the— the body's not the same because the— because respiration, because changes in the system, they— they're not always aligned. And the patients moving and coming in the next— next day or next week do the same scan with a different scanner would be a very different positioning.

So to do so, we have to kind of align these, and we go through the same loop here. But we have many different types of metrics here. Some of them are not necessarily

quantitative, but qualitative, and also hard to know which bug or which issue has— has come across. So as part of that loop process, one of the idea is that as a— as a scientist, I would come here and look at these images one by one myself.

I could have tried to say, oh, is this kind of quality control metric kind of story where you look at this image, say, does this make sense? So here you can see a lung mask. Is the mask applied correctly on the lungs or not?

If this process, next steps is going, is the scans properly truncated or cropped? But each of these processes, there's a loop in itself, and it could be— this concept of adversarial could also be collaborative. So as part of the metrics, you can kind of call in another model that has a better multimodal capability to review the image and what's the opinion about this image.

Does it scan correctly aligned?

And then as part of the hypothesis generations or critique of the changes that has happened, you can invoke the another model such as that is much more better at reasoning, 505 Pro, to generate the hypothesis because they have much bigger test time compute and inference time compute.

To do so, you need to equip your loop with additional skills to do these— these processes by themselves. And it's very easy now in the coding agents, of course. So here I build a skill for review of that image generated with, say, with a Gemini model and so on.

So as part of that loop, you can kind of implement these— these loops here. And of course, to use GPT-5 and Pro, that's something I really like is Peter Steinberger's Oracle CLI, which packages the code and packages the data and sends them to API of the— the GPT-5.5 Pro, and then we'll be able to kind of close the loop on the— on the outcome.

And you— I've seen a much better improvement during the— especially the hypothesis improve generation, but also the critique of is this implementation was really effective because every loop that goes in, the loop basically asks, hey, this is what we implemented, this is the outcome of it, does this make sense, what would you do next?

So the process, the loop is very important to kind of keep going and improving the model. Out of this story, I think what you see as this back to the concept of autonomous researcher is that you have the ability to go through this loop of scientific discovery with much more rigor into the hypothesis.

And— and I think implementation is already quite a bit solved, especially if your problem is, you know, you know, in a— in a basic can be done in a basic simulated environment in a box. Of course, the big thing in science is better observations.

The multimodal model currently, they lack very much understanding of these components. So for example, here, most

understanding of these tiny— tiny changes, for example, the existence of this nodule, no LLM today is able to identify these very well because it's just not simply trained on scientific images and scientific data. So there's a big space for future work for people who build these models to be able to fine-tune them or be able to adjust them so they can— the model themselves can close the loop by observing the same way as a trained scientist would be able to observe that particular issue.

And then, so that's I think one of the biggest bottlenecks for— for not having a full, you know, scientist in a— in a— in a data center. And then the next thing would be this strength of these hypotheses.

But I feel the same way as the chain of thoughts came on top of the basic

base model, like a, you know, 4.0, GPT— GPT-4.0, and allowed it to do much more better reasoning. This type of trick with this type of hierarchical structures is the same trick basically applied on the current model. And I think going forward with the newer models that are much better post-trained to compartmentalize the problems and break down the problems, we probably get need less and less of these tricks down the road.

But in the meantime, and the absence of very good models, I think you— you can do something like this where manually, well, in a different explicit steps, decompose a problem into hierarchies and then going through those hierarchies for— for improvements in each component of it.

And basically that allows a very structured way to scale the test time compute to— to generate more and more tokens on this problem and close the and continue the loop. Well, thank you very much for this talk. Hope to see you around.