Proactive Agents – Kath Korevec, Google Labs

I'm so excited to be here. I love New York, and I love meeting everybody here. And I am Kath Korevec, I'm from Google Labs, and I work on this little team called Aida. And I'm going to be talking about some of the stuff that we've been doing on this project called Jools.

So a few months ago, in my household, our dishwasher broke. And while it was being repaired, my husband decided that he was going to do all the dishes. And so he told me he was going to do this, but every single night I found myself reminding him to do the dishes.

And you can imagine, that got old pretty fast. And I realized that even though I wasn't physically washing the dishes, I was still carrying this mental load. And I know a lot of you can probably relate to this.

I was keeping track of whether or not that task was done, following up, making sure that things kept moving. And I realized in that moment that that's exactly where we are with asynchronous agents today. They can handle some of the work, but we're still the ones, as developers, carrying that mental load and monitoring them.

So here's the truth. Humans, we are serial processors, not parallel ones. We can juggle multiple goals, but we execute them in sequence, not all at once. When you manually kick off a task in Jools, you are usually waiting to be able to move on.

And it's that pause, it's that gap in attention where we really lose momentum. And this is actually backed up by science, where humans actually think we think we're multitaskers, but we're actually executing many tasks very rapidly. But switching between these tasks comes with a huge cost.

It can cost up to 40% of your productive time. So that's like half a day lost to switching contexts and reloading. So if humans are unitaskers, what's the solution here with agents? So for async agents, in order for them to succeed, developers can't be expected to babysit them.

We've all seen that post on Twitter of 16 different Claude code tasks running in parallel on 16 different terminals on three different huge monitors. And when I first saw this, I thought, god forbid that is the DevEx of the future.

I don't want to manage work. I don't want to manage my agents. I want to be a coder. I want to build. And so we need to think we need collaborators in our system that we can trust, agents that really understand context, can anticipate our needs, and they know really when to step in.

And then I think finally we're reaching that point with models where they're getting better and better at executing end to end, as long as they understand what our goals are clearly. And that's where trust really becomes this unlock, where you can trust the system to know what's missing, to fill in the gaps, and to really keep progress moving forward while you manage on something else, while you focus on what matters most.

And essentially, we want Jools to do the dishes without being asked. So most AI developer tools today are fundamentally reactive. You open up your CLI or your IDE, and you ask the agent to do something, and it responds.

Or it waits for you to start typing, and then it auto-completes a suggestion. And there's a benefit to this model. It's very efficient. It only uses compute when you explicitly ask for it. But the real question I'm asking myself is, is this how I want to manage AI?

And if you think about in the future, imagine a world where compute is not a limiting factor anymore. Instead of a single reactive assistant for instructions, you could have dozens of small proactive agents working with you in parallel, quietly looking for patterns, noticing friction, and taking on the boring tasks that you don't want to do before you even ask.

It could do things like fixing authentication bugs that you've been avoiding, updating configs, flagging potential errors, preparing migrations. And all of this can happen in the background, triggered off of things in my natural workflow. So I really think there are four essential ingredients that make up proactive systems today.

There's observation. The agent has to really continually understand what is happening, what your code changes are, what your patterns are, what your workflow is, et cetera, to get context about your entire project. And then there's personalization. And this one's difficult.

It has to learn how you work, what you care about, what you tend to ignore, what your preferences are, the code that you absolutely don't want to ever touch. And then it has to be timely as well. If it comes in too soon, it's going to interrupt you.

And if it's too late, then the moment is lost. And it also has to work seamlessly across your workflow. It has to insert itself into spaces where you naturally work already, in your terminal, in your repository, in your IDE, not forcing you to go somewhere else to some application that's secret or that you forgot about.

So bringing all these tools together, you can imagine, is not trivial.

Oh.

So I was running this presentation. And

you want to be able to ask your agent to understand your workflow and anticipate your needs, and then intervene at exactly theright moment without breaking your workflow. And that's when it really starts to feel like magic. The interesting thing is, these proactive systems, they're all around us today.

One of my favorite examples is Google Nest, where you put it in your house, you install it, and then you configure it. And then it starts to learn your habits as you leave the house, as you come back, as you go to sleep, as you wake up in the morning.

And then pretty soon, you don't have to think about climate control in your house anymore because it's learned what your habits are. Another one is your own body. Your heart rate elevates as you go for a run or start to work out.

Or it anticipates that you're about to fall, and so it reacts before you consciously think, I'm going to put my hand out. So when you look at it like that, proactivity is actually not that proactivity for AI is actually not that futuristic.

It's very familiar, and it is very human. And that's exactly the point. What we're building is tools that behave more like a good collaborator and less like command line utilities. So we're already doing this in this tool called Jools, which is this proactive asynchronous autonomous coding agent from Google Labs.

And we're doing this in kind of three levels of proactivity. Level one is where a collaboration really starts to emerge. And this is how Jools works today, where it can detect things like missing tests, unused dependencies, unsafe patterns.

And then it starts to automatically fix those things as it's doing other tasks that you've asked it to do. This is sort of like this attentive sous-chef in your workflow, where it's keeping the kitchen clean, the knives sharp, the kitchen stocked, so that you can focus on what comes next.

And that's the beginning of proactive software. At level two, the agent becomes more contextually aware of the entire project. It observes how you work, the code you write. If you're a back-end engineer, maybe you need help with React.

If you're a designer, maybe it wants you to maybe it'll help write the database schema. And then it learns what your frameworks are and what your deployment style is, et cetera. And this is the kitchen manager. This is the person in your workflow keeping the rhythm and anticipating what you need next.

And then comes level three. And this is what we're working on pretty hardright now going into December. And I'll show you a little bit of what we're going to be shipping in December in a minute. But level three is where things start to converge around that context.

It's where the agent starts to understand not just context, but also consequence, how these choices are actually affecting the users of your products, the performance, and the outcomes. And at that level, we have this thing Jools. We also have an agent called Stitch, which is a design agent, and another one we're building called Insights, which is a data agent.

And they're all coming together to build this collective intelligence across your application. Jools can see what's breaking in the software. Stitch understands how users are interacting with it. And Insights connects behaviors from real-world signals like analytics, telemetry, and conversion rates.

And then together, they can propose improvements across boundaries of how the system all works together, doing things like performance fixes to improve UX, and then design changes to prevent regressions. And then all of that is organized based on live data.

So the trick here is that the human stays firmly in the loop. You're observing what the agents are doing. You're refining when you need to intervene. And then you're redirecting it when it has been misdirected. So level three isn't really about autonomy anymore.

It's actually about alignment to your project, agents and humans collaborating together across the full life cycle of your project. Soright now, Jools is focused on this code awareness piece. It understands the environment, the frameworks, and the project structures.

And we're moving towards more of that system awareness. So things that we're introducing in Jools now, we've added something called memory, which I'm sure a lot of you are familiar with. It's the ability for Jools to write its own memories, and you can edit them and interact with them.

It can edit them, and it understands and builds this memory and context and knowledge of your project as you work with it. We've added a critic agent, which works adversarially with Jools to make sure that the code is high quality, but then also does a full code review.

And then we've added verification, where Jools will write a Playwright script, take a screenshot, and then put that back into the trajectory for you to validate. And then we're also doing things like adding a to-do bot that will look through your code and look through your repository and pick up on anything where you've said, this is a to-do I want to get to in the future.

And it will start to proactively work on those things with that context. We're also adding in things like best practices, where Jools will understand best practices and start to suggest those, and also environment setup. We have an environment agent that we use internally for running evals.

And we're extending that externally to better understand how your environments work and set those up for you. And then we also are adding something called a just-in-time context. It's like a Jools cheat sheet, where if it's doing something very specific and gets stuck, it can just immediately look at that cheat sheet instead of reaching out to you.

So this is all moving Jools very close to being that proactive teammate, not just this reactive assistant. OK, so this morning, I was talking to my team back in San Francisco. And I was thinking, OK, I'm going to do a live demo.

But the live demo gods did not align with me this morning. We still have CLs that are being pushed to stagingright now. So I'm going to walk you through a little bit of this. And if you know Jed, he's going to, I think, be talking tomorrow.

We're going to affectionately try to fix Jed's code here. So this is a view of proactivity. And this is Jools, where you prompt it. And the first thing that you do when you configure and enable proactivity is Jools will index your entire code base.

It'll index your directory and start looking for things that it can do. And then that'll show up on the screen. Soright here, we're looking at a little bit more in this repository 80k Python. And it's indexed the repository.

And it's found a bunch of to-dos. It's found a bunch of best practices that it can update. And it's giving me some signal about what it's finding. And so you can see the signal is high confidence, medium confidence, and low.

And so it's actually telling me what it thinks it can achieve based on what's in my code and what it wants to do. And so it has high confidence in green, medium in purple, low in yellow way down at the bottom.

And so I can go through this, and I can manually click these and say, I want to start these. And so I don't have to think about the prompt. I don't have to look at the code. I can do kind of less cognitive load here.

We're working on something to just start these automatically. And so that's coming in the future. But I can also delete these. I can say, hey, this one isn't for me, isn't good. And so once it gets started on a task, I can kind of drill into it and see a little bit more.

I can peek into the code that it is suggesting that it's suggesting it work on. I can find the location of that code. And it also gives me some rationale about why it wants to work on that code, what it's doing, et cetera.

And so it's giving me a lot more context and helping me trust that it knows what to do here. OK, so that's proactivity. That's coming in December. And hopefully, we'll be able to give that to everybody here. We're very excited about it.

And I want to tell you a little story about something my husband and I were working on just to kind of wrap things up. We tinker a bunch with hardware. And we live on this slow street in the middle of San Francisco in Haight-Ashbury District.

And so on Halloween, we get a lot of people walking by our house. And so we were trying to take advantage of that with our Halloween decorations. And so we built this six-foot animatronic head that sits in the front of our house.

It's this old Victorian house. And he sculpted it out of foam, epoxy, and fiberglass. And then our kids also called this lovingly the Baldhead. And it's based off of if you ever saw Pee-Wee Herman from the '80s, it's based off of the Pee-Wee Herman, Pee-Wee's big adventurous head.

So while my husband was doing this, I was spending my time working with Jools on updating the firmware, controlling the stepper motors, working on the LEDs and the sensors. And for me, that's the fun part for me is really getting creative with what the LEDs are doing.

So I wanted to focus on that, the LED animations. But I ended up spending most of my time actually fixing bugs and swapping libraries and doing things like that. So what I would do is I would prompt Jools.

I'd wait 10 minutes, and then I would repeat. And I found that process very, very tedious. And what I wanted was actually Jools to do the research. I wanted it to handle the ugly parts, where it was researching how to fix a bug, doing the debugging itself.

And I wanted it to do this so that I could focus on the creative parts. I wanted the eyes to move and follow people as they walk down the street and have lasers coming out of its eyes and stuff like I mentioned it was Halloween.

It was very scary.

But I couldn't really do as much of that. And I ended up actually not shipping as much as I wanted to with this animatronic baldhead. And so it's that gap that we actually want to close. It's the space with Jools.

It's the space between that tool friction and creative freedom that we're trying to unlock with these kinds of proactive agents. So what I really want you guys to take away from and I give this advice to the folks on the Jools team a lot is that the product we build today actually won't be the products that we have in the future.

And I think a lot of us know that. But in reality, I want everybody in this room and everyone building, working with AI to be able to take those big steps. I think the patterns that we rely on today, Git, your IDEs, even the code, how we think about the code itself, might not exist a year from now, might not exist six months from now.

And that's the exciting part for me. It's sort of we get to invent the futureright now. We get to describe and decide how software is made and built, kind of all the people in this room. So my challenge to you is to not be afraid to question the old ways of how you're building software.

Because really, the future is coming faster than any of us know. It's probably already here. And the cool thing is we get to build it together. Thank you.