RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI

Hello, and welcome to this online track talk for the AI Engineer Worlds Fair 2026. Today we're going to explore the concept of RLM, also known as Recursive Language Models, and how we can use those concepts for larger codebases.

My name is Shashi, I'm a founder of Superagentic AI. First of all, let's be clear that the RLM paper has been published by MIT and Friends. As you can see, there's a full paper you can read about it, but the purpose of this talk is how you can use the concepts of RLM and you can use into your own workflow to implement your own harnesses.

So, first of all, what's the problem? If you're using the coding agents for smaller repos or monorepos, they work exceptionally well. But if you have ever tried it with the monorepos, with the large context, you know there's a context problem.

As the context grows, the performance degrade. And if you're working with the monorepos, this problem gets worse. In this talk we will see. We selected the codebase, and the concepts of RLMs are relevant for the larger codebases. If you've used the coding agents, then you probably saw that there are different approaches that other coding agent harnesses have been taken to solve this problem.

The most common approach is searching using the tools like Grep. So basically, there's a file system, and the coding agent harnesses search using these tools. The second approach, you've probably seen, that's the semantic search, or the local search.

So the idea here is basically you can search through the code and curate the context. Another approach is the long context gets compressed, and you can use the summarized version of the context. And there are some memory solutions available in the market as well that you can use to persist the memory for the coding agent.

First of all, let's explore the RLM idea. The core thesis of the RLM is you need to externalize the context management into a programmable execution environment. Meaning, you should have a separate dedicated environment so that the model can operate on that.

In this case, for example, your whole repository is treated as data that the model can operate on. Then the model can write the code to inspect, slice, and compute the relevant chunks. The value you can then feed into the main context window.

So basically, rather than putting everything into the model's context, create a separate dedicated environment, give them a coding agent or REPL, and then the model writes the code to curate the context that can be used into the main.

So it's another context management technique proved to be very effective. It could also be used as a memory layer for your coding agents. Let me summarize this with a simple analogy. Imagine you're a lead software engineer and assigned to the new project with a huge codebase.

Imagine that's a monorepo. How does that lead engineer deal with the code? So rather than reading a line of code line by line, the engineer probably inspects the codebase, makes some notes, sees what the project's dependencies are, how it is structured.

If something else is not understood by the repository, the engineer probably asks another engineer or expert to get some ideas. And the same concept is applied in RLM. So large projects, like the files and docs and texts and configs, because the repository has a lot of things.

And the programmable REPL, it's kind of a notebook that the engineer makes a note about the codebase that can be used for researching. He may be using other techniques, or maybe he's writing some script to search something from the repo.

And then if he's stuck, then he asks another engineer or specialist where it comes to the llm_query. And llm_query is basically asking another model environment to get an answer from. And once they get an answer, then the loop continues.

And at the end, it returns the clean note. Synthesis. So the recursion part here is the engineer asks another specialist using llm_query. That can be one question, or that can be a number of questions. So this is where the recursion comes in picture.

The loop is basically you have a repo as your context, and then the model writes the REPL code to get some relevant context. That returns the bounded observation. And if the loop needs more information, it passes through the llm_query where it asks another language model or another system to get the response, returns the value, and continues the loop.

And the loop gets terminated until we get a final result.

Why are we talking about the codebase and not the big contexts in terms of, like, other things? For example, books or dictionaries. The codebase is different. It has directories, it has tests, it has some imports, it has dependencies, it has tests, it has pictures, it has configuration files.

So the codebase is not only just the text, it is structured data. And the model needs to understand and reason over the text. That's why I chose this scenario, to use the codebases to prove these concepts of RLM.

Now let's switch the gear and talk about our own library that we created at Superagentic AI, called RLM Code. You can see RLM Code's landing page here, where this is just a research playground where you can implement the concepts of RLM.

We have documentation that you can take a look at, and there's the GitHub repository. It is a completely open-source project that you can use

and play with. RLM itself is a concept and a pattern, and you can implement that concept and pattern in your own way. There are official authors who also wrote some implementation in their GitHub repos called RLM and RLM_minimal.

You can refer to that implementation of RLM in dspy.rlm. So Omar is the author of RLM, and he's also the author of another popular framework called dspy. So dspy got RLM implementation inside it. However, you should treat. They are completely different.

So RLM is a pattern, and you can implement in your own ways. You can find there are various other people who implemented RLM in their own way. And in a similar way, we implemented RLM Code as our own independent harness that we will be using in this live demo.

RLM Code is just a reference implementation to demonstrate how the RLM concept works under the hood. So we have implemented something called RLM mode. We are using RLM as it is. We are not adding anything on top of RLM's ideas and RLM's paper.

We are using the same concept of recursive calls, REPL execution. However, you can run it with a local model, you can run it with a cloud-based model, you can plug into any observability framework of your choice. And that gives you, like, a lot of flexibility around RLM.

You can also plug it into the framework of your choice. For example, you can use Pydantic AI, or Google ADK, or something similar framework, and implement ideas of RLM over there. In order to demonstrate this, we have created a source code repository where you can try this concept by yourself using MIT's RLM paper and RLM Code.

And we will see how these things work in practice. So basically, we will show you the loop. You will understand this once we see this live demo and what all these files are doing, where the context has been created, where the Python REPL has written a code, and where it's passed to the llm_query, and how we get the final results.

So that will be covered as part of the live demo. So we can cover everything here. So in a nutshell, how it looks like is basically it creates the REPL, and then observation, and the final recursive language output.

So let me jump into the live demo now. Okay, let's do the live demo of these concepts of RLM and RLM Code, and how it works in the larger codebases. So I have a code repository here I have checked out, and let's open it into the editor so that we can see what's inside it.

So as you can see, there's a demo target, which is we're using RLM Code source as a demo here. And then we have some instructions that you can follow along yourself. So basically, you have a README file that you can use with your local model or.

So we are going to use it with Gemini. So we will try this script and see what happens.

Soright now, you can see we're using the Docker as a sandbox. If you see,

the Docker container has been just started for this RLM. And now coming back to

our execution, you can see that execution has just finished. And in this execution, what you have seen, basically, in the first step, the model has written the REPL code that you can see here. And then it's built the evidence.

And after that, it also made the calls to the llm_query

with some prompt and got the result back. And after that, it gives the final answer. And as you can see here, we can have

all this step coming back to the final answer. And here you can see that it made two tool calls, and how many tokens are used for this model that you can see here. And the good thing is that you can see all these traces in the RLM Code repositories.

So for example, you can see all the runs. This is the run that we just did. You can see all the sessions and all the observability that you can plug into any of your favorite observability platforms. So this is the CLI path we just demonstrated, but we also have this

kind of coding agent style experimental harness where you can try the same thing. So first of all, let's connect with the Gemini model. So you can connect with the Gemini model using the command connect. You can have a provider and the model name.

Now you are connected. So you can also run the Docker command

and see if everything is okay.

It seems like the Docker command prompts some warning, but this is related to Deep Agent ADK and other frameworks, which is not relevant to this demo. And the interesting part where we will be seeing is basically you are sending the prompt.

So for what we did now, we ran the command and we asked the question. We specified the budget so that we don't know spending too much on this run. But once we do that, as you can see, you have maximum steps, recursion depth, and it completed this run and coming back with the results.

We can also see this thing into the research lab where we can see the spend has been completed. We can see some rewards. We can also see the trajectory, which is an important part where we can see all the RLM loop.

For example, the REPL and the code and the final output.

And also we can see the events, when it started and when it ended. So you can play around with this RLM Code terminal user interface, which is kind of a harness, and you can experiment with your RLM ideas in here.

So I'm going to quit this for now, and let's switch back to the slides.

In a nutshell, what we just saw, basically, our context has been loaded. We have some REPL code written to extract some snippets. We also saw the llm_query has been called to get some more context from another model. This is where the recursion comes in picture.

And we got the final result. And we got the traces in JSONL format that you can import into any of the observability platforms of your choice.

We also saw these results coming from different files. You can take a look at the source code that will be available for you. Let's talk about the real thing, how an AI engineer could use these concepts in real life.

And there are a few things. For example, if you're dealing with this large source code and you want to. For example, root cause analysis, or onboarding of the repositories, or some unfamiliar repos. So there are a few use cases you can from here and probably try to use RLM concepts over there.

Basically, you can design your own harness based on your needs. So that should capture the whole trajectory, all these things like the planning, coding, observation, sub-call, budget, and the final output. Now, coming back to the final point about RLM concepts and where it's been used.

I have recently come across a lot of posts on X saying the RLM concepts have been being used into some of the proprietary things, like the managed agent, dynamic workflows. Using the RLM concepts under the hood. So they have implemented one or more forms of RLM inside their agent harnesses.

Recently, I saw that the Codex harness is writing the Python code in the REPL that you can see to curate the context. That is one form of RLM I have seen myself. And obviously, the Claude's managed agents or Gemini managed agents, they are all kind of concepts of RLM.

So basically, you can get the harness in the sandbox, and then you can do the stuff. And the recent things about the dynamic workflows where one agent. Given the task, you can spawn multiple agents that have their separate sandboxes.

They can work together and give back the final results. And the idea is basically generally coming from the RLMs. A lot of software factories concepts are probably using the RLMs, but we are not sure yet. However, some of the cloud code engineers from Anthropic have accepted on X that they have used concepts of RLM.

You can use this RLM concept on your large context repository. And if you have any questions, then feel free to reach out to me. And finally, thank you so much for listening to my talk.