Why Eval++ Is the Next Great Compute Primitive — Sunil Pai & Matt Carey, Cloudflare

Uh, yeah, well, welcome. I'm Matt, and this is Sunil. We work on agents at Cloudflare, this, this thing. So yeah, maybe you want to introduce yourself a little bit better than me.

Uh, this is Matt Carey. I only hire my friends for our team. Say something, make sure it's on the microphone.

Yeah, yeah.

Okay, yeah.

Once, once. Is mine on?

Like, I keep thinking about Cloudflare as a friends-and-family company. We love doing favors. All our closest friends, no one pays for Cloudflare bills, which is already dirt cheap, by the way. It's ridiculous. So we started building out AI agents seriously a little over a year ago, and it's gone remarkably well.

How many folks here are, like, orange-peeled? Which is to say, Durable Objects. Okay, yeah, like, worst named technology, but incredible,right? Everyone's like, it's a database. I'm like, no, but it has a database. And I'm like,ugh, but. But we started using that, and it turned out to work out really well.

There are some unique capabilities about it. Some might call it vendor lock-in, some might call it innovation. But, like, and ever since then, the team has grown. We are based in, like, most of us are in London. Well, you're British, but you just moved to Lisbon, which is amazing, by the way.

Folks have you spend your days drinking wine, and you do fish, potatoes, bread. It's just king's life. I live in Newcastle. The weather is way worse, but I do like it. I tried watching Jordy Shore, and I regret doing that.

I will no, that's not what I think of. Anyway, so we've been building agents and.

Back on topic.

Sorry, yes, of course. And I don't know, we can tell you about our journey so far, some of the tech we're using, some of the new upcoming stuff. Thankfully, none of this is recorded, so we can tell you about some secret features that are coming out.

But I'd love to put it out there for a couple of minutes if you have any questions. Otherwise, we have, like, five, like, topics we thought would be interesting to talk about. But what do you want to know about Cloudflare and AI agents?

Tell us about this.

Amazing.

So it starts with Durable Objects. OK, so you know how everyone believes the way you build servers nowadays is you write a function that takes a request and returns a response,right? Simplest model. And you decide you want to do you want to build a hit counter for your website.

So what you do is you say, let x equals 0, or let counter equals 0, and for every request, you'll say counter plus plus. And it works locally, and you deploy it. And it immediately fails. Why does it fail?

There's no state.

There's no state. Like, it spins up, it does the thing, and it disappears. Counter will always be 0, most times, at least. Like, on traffic, it might reuse the isolate. So now you have to use a database to store state.

You need to do a whole bunch of things. Instead, for a given ID, what if you could have a class that spun up once for a given, like, spun up once, and then every future request, every WebSocket connection lands in the same place?

So we call this stateful serverless, which is the most confusing thing, because servers are stateful. But it works like serverless in that it scales across the planet. And because of the Cloudflare magical planetary network, it means, like, in London, you get, like, 15 millisecond latency.

For contrast, 60 fps is 16 milliseconds, a little over 16 milliseconds, which means if you are using if you folks have used TLDraw, it's built on this tech. If you open up a bunch of phones on the same shared screen and you start, like, drawing on it, you'll see, like, perfect sync on all phones.

It's I know I'm biased, but so good. Anyway, so we it turns out that that's the perfect compute unit for building AI agents as well. They're addressable. You can run long-running things in it, even if there are no requests going to it.

They can run in the background. They hibernate. They go to sleep. They have persistence. And they can connect to, like, everything else. So that was the original bet we made. And I wonder if I can scroll down. I wonder if, like, the API is available on this page.

Something, something. Marketing, marketing, marketing. Oh, yeah, see, that's kind of, like, what it looks like. You basically make a you import from agents. Please don't ask us how much we paid for this npm package. You extend it, and you can add.

You're like, oh, when it starts up, schedule these things. You can make little callables. So agents also come as a full-stack thing. We give you, like, React hooks. And, like, it doesn't have to be React, but I assume.

We have plain JavaScript clients as well. You can define things that are actually called from clients. You can run things in the background. There's, like scheduling is my favorite thing, because it means you can say stuff like, every Friday at 9:00 p.m., look through my entire Git history and my wiki and the Notion, compile it, and send it to my manager.

Make sure you mess up the spelling so it looks like I wrote it. So you can do all of this. Like, it's nice. You can and you can, like, it spins up to millions, trillions, some absurd Cloudflare number.

That's where it starts. And over the last year, we have built a bunch of things. So we have a first-class backend for Vercel's AI SDK. And it turns out the AI SDK is great. Is Niko around here somewhere?

No, anyway, we don't allow Vercel employees into our chat rooms. So it turns out there's a lot more to make it production-ready, to do tool calls, to do synchronize, to do resumability, things across tabs. So we have the world's best backend for, like, the AI SDK.

I'm saying a lot of things. So we have that. We have I want to talk to you about code mode, which is, by the way.

About maybe this point in the journey, like, MCP got mega-popular around April last year. And so it turns out Durable Objects are actually a great way to do MCP servers, because if you know anything about MCP, when it was originally launched, you needed to have a stateful connection between client and server.

And this was one of the most annoying things about deploying MCP servers to production, to the cloud. And Durable Objects maintain a stateful connection, like, very, very easily. That's kind of the whole point. And so we, like, jumped on that really early.

We had some of the, like, first MCP servers, like PayPal, Sentry.

All the big ones are on this.

All of them.

Linear, Intercom, name it. Yeah.

All of them. Some we can't name. And yeah, so it turns out MCP was, like, really, really good for this. And this led us down, like, very many more rabbit holes, really, where I think this is our lead into code mode, really.

Sure, yeah.

Yeah. And then so then we have so we have Durable Objects. We have Workers, which is sort of like a serverless function as a service, like a Lambda, you think. Like, imagine a Lambda was designed 10, 15, 20 years later, something like that.

Like, it's better.

Then we have this new primitive, which we're calling Dynamic Workers, where it's like a worker, but instead of having to deploy the worker to the cloud, you can go from one worker and be like, I have this string of code that a customer sent me, or a user sent me, or an LLM generated.

And I'm going to run that piece of code in its own isolated worker. So from a string, you can run the code. And this is, like, kind of breaks a lot of people's brains. I was at MCP Dev Summit last week, and there was a startup next to me.

And this guy was just trying to convince me that no enterprise environment would allow people to run generated code that they've never seen. And then someone from Lockheed Martin came up to me and was like, oh, we love this, like, generating code.

And just, like, the dichotomy was really funny. But basically, you can run people's code that they generate in this thing called a Dynamic Worker. And we think it's a very, very good sandbox in the sandbox use of the world.

Not a VM. Not a full it doesn't have a full file system. It doesn't have, like, all of the VM-like heaviness. But it is an isolate that you can spin up. And you can spin up billions of them on demand.

And so that's kind of cool. So anywhere where you want to have, like, user-generated code, or you previously would use something like a DSL, like, you would have some JSON file that was generated or was built from some frontend, from some big form.

And then you would take that, and you would convert that to code. Now, just write code. At least I hope this is it. Yeah.

So I love so I hate being that guy who's like, oh, you should read my blog. But you should read our blog, by the way. The Cloudflare blog, really nice, incredibly technical. Like, the only part where we try to sell you something is where the marketing folks ask us to put a CTA at the, like, very bottom.

You can ignore that bit. But everything else is dope. We go into details about this. So you know how, like, sandboxes start, like, from a VM or a container, and then you try to add security, like, from around it?

So, like, we start the complete opposite way. The only thing you can run is JavaScript in it, but it has no access to fetch, no APIs, nothing. And from the outside, you can decide, OK, here are, like, four APIs.

And only outgoing fetches to github.com are allowed. Hell, github.com/xyz. Or math.random. I don't know. Some requests fail. You can write code that does it. And the way we recommend it is, no, just block outgoing fetches. You give this sandbox explicit capabilities that you can run in.

No environment variables exposed to any of this code. You should check this out. Should I do a demo of the MCP server? Like, can I show the MCP server a bit?

No, no, no, because I'm going to do it in my talk. We can tease our talks now.

OK, oh, we get to tease our talks.

Yeah, we tease our talks.

OK, OK, OK.

So basically, using this, we kind of claim that we fixed MCP, not once, maybe twice. And that is both of our talks. Yours is.

That'sright. 5:40 p.m., I'm giving a talk about code mode, what we call code mode. I have the hat. For people who are asking where you can get code mode hat, we have a careers page on cloudflare.com.

And I'm doing a talk tomorrow about how you can use code mode in your MCP server to give access to all 2,600 API endpoints of Cloudflare in just only 1,000 tokens. So the code mode, like, this idea of being able to generate code that you then run is super, super powerful.

I don't think we have enough time here to, like, fully explain how that works. So I would just ask to come to our talks.

Yeah, you should have. Like, so the way I've been telling it to people is, for the last 30 years, 30, 20-something odd years, you go to being such an old guy in the room so fast. They've told you never to use eval in code.

In fact, on Cloudflare Workers, eval you don't have eval. It's dangerous. But we took it, and Dynamic Workers are, like, Eval++. So I think about it how there's an entire branch of the tech tree that we haven't explored in, like, 30 years.

And now we are giving you a fast, secure, and cheap way. Is there such a thing as fast, good, fast, and cheap?

Yeah.

Maybe it's not good. I don't know. You have to try it out and let us know.

I know it's fast.

And you get to reconsider the way you build interfaces and how things happen, especially in a world where all your users can write code,right? So that's Dynamic Workers. That's, like, it's really cool, man. Like, I've been having a lot of fun playing with this.

It's very new.

Should we tease some stuff? Or do you want to talk about we had some other releases previous recently. Go on, yeah, question.

Just a quick one, because I think it sounds like you guys are I mean, you obviously are, like, super you know, maybe this is just me. I'm struggling to find a way to, like, map all the things that I'm building to this world.

I'm not quite whether it's just me or I'm trying to find a bridge. I just want to hear sort of, like, what you think people are building in the industry that, like, they really shouldn't be building it like that.

They should be building it this way.

I've got a good anecdote.

That would be very helpful, too.

Oh, I'd love to give examples. OK.

Yeah, let's give examples. We do one each?

I mean, we'll do five each. Go ahead.

OK. OK.

So, I mean, we are here to chill, bro. Like, that's it.

Yeah.

So my favorite example is there's a big thing going around about generative UI at the moment. And people are basically have you heard of JSON render? Yeah. You're, like, generating JSON, but then you render. That's kind of in the name.

Why are you generating JSON? Why don't you just generate HTML, or even generate React, and then just render that? Well, why can't you do that? Well, some platforms don't have a primitive to render untrusted code. Well, what happens if you did?

You could literally get your users, get your customers, get your clankers, get your agents living in the cloud to generate code that they run that then is rendered in a UI. Like, anyone tried Claude artifacts? That's basically, yeah, that's it,right?

It's just HTML, but it's rendered on your browser. So people don't care so much about that security. What they care about is when they're rendering it on their servers. That's when it starts getting a little bit dodgy. And now they're like,right, you need JSON.

You don't need JSON anymore. You could render React. And this is how you build it. So this is, like, there's one fun thing here. Yeah, do you want to go to the next one?

Wait, which feature?

Dynamic Workers.

Dynamic Workers.

Yeah, Dynamic Workers. I'll go, like, a little simpler. OK, so because our agent thing is, like, not is more the execution environment than the library use, like, you can use LangChain, AI SDK, anything with our agents SDK. We give you that.

But because there's this durable object model, it means let's say you wanted to build resumable streaming. OK, so you tell the LLM, give me a long story. And you hit refresh in the middle of it. How do you make sure it continues streaming in a serverless world?

OK, now you need to do a database. You need to do replication. You need to do sticky sessions. In the durable objects world, in the agents SDK world, you just reconnect to it. And if there's a stream going on, it just says, well, here's the beginning of the stream, and I'm just going to continue giving you bytes.

It means that you automatically get multi-tabs, multi-browser sync. Like, you're looking at your phone and the laptop same time. All of these things come out of the box for you. In fact, like, the killer use case where we started with durable objects was for building real-time collaborative sync.

Now, I'm a believer that AI should be a multiplayer game. Have you noticed, like, why can I not share a link to my ChatGPT chat with you, and both of us work in the same conversation? It's because OpenAI hasn't, like, built that.

And they're like, oh, we have, like, bigger problems to solve. Half our people quit, like, every three weeks or something like that. It turns out, like, with primitives like this, you don't have to patch it in user land and become this crazy distributed systems engineer.

You make it Cloudflare's problem. Well, our problem. Well, I say our problem, but we take credit for the people who actually build it. We do the JavaScript on top of it. So I really like that. Like, the fact that you get sync, streaming, et cetera, out of the box, it just it means you can play with you can build your you can focus on the things that, like, you care about.

That's the killer demo, isn't it?

I like it so much. Like, just showing, like, the sync is just so fun. Because then it, like, unlocks. They're like, oh, OK, fine. If you can do that, I'm going to worry about making money. It's a fun thing to do.

So would it be fair to say you could rebuild a closed code cloud interface with just use.

Should we start leaking secrets?

Yeah, we can start leaking secrets now. OK, yes.

You have the problem of starting a VM.

Yes.

Cleaning the app and resuming the app.

Yes.

But it's always top-level.

Yeah. So there's some cool things you can do here. Because imagine an agent loop that ran in the agents SDK that did the backend of Claude code. Now you have a server that runs, that's persistent. And you can connect to it from any client, so a terminal, from a chat, from a phone, from an iOS app, from a web app.

It doesn't really matter. Like, everything is synced between all of your clients. Everything is resumable. Everything is stateful. And I guess.

So I guess it's up to Cloudflare to now build a harness,right? A coding agent that runs purely on Workers and Durable. Hmm, should we build that, man? Should we be building this? Well, we are building it, just so we're clear.

If I wasn't obvious enough. We hope to ship it imminently. Like, the conference is taking my time away from burning tokens, is what it is. As quickly as I can go back to that. No, but no, we believe in that world.

Look, everyone's building a harness. And everyone's leaning into the benefits of their infrastructure or their philosophy. This is ours, where not only do you have these spin-up, spin-down, stateful agents, but with capabilities of generating code on the fly and running it, like, instantly, really fast, resumability, et cetera, out of the box.

And if you want to delegate to a big container, you should absolutely be able to do that. Well, our team also handles Cloudflare sandbox SDK. You want to run it in a browser. Well, we have a browser. You really have to use our stuff.

You want to use Daytona, great product, by the way. Love their sandbox stuff. You want to use browser use, or you want to run what's the Light Panda? Is that what it's called? You can do that. But Workers and our agents SDK becomes the nexus of where, like, it connects down to all of the things.

That's the vision we have. It's happening sooner than later.

And specifically, how would you then create a cloud clock, particularly one that can, like, create its own skills? Because I think that, for me, is the thing I love the most. The ability to just say, like, send it a voice message, and it starts, I've got a cron that does a really great thing now.

So

we have a bunch of storage solutions. We have one in the durable objects. So we have, like, SQLite directly in the durable object. So you can store stuff there. You can store stuff in R2. And all of that, at the moment, you have to kind of wire up.

But imminently, you won't.

OK, so here's the thing. What is Open Claude? You want a thing that has a heartbeat. You want a thing that has a virtual file system. And you want a thing that's connected to services.

Is it, like, control secrets? I can say.

Exactly. And you want extensions, which means you want to generate code that's run in a safe, sandboxed environment.

Exactly.

I implemented extensions this morning. It works really well on our thing. By the way, I'm friends, like, with Mario. Hi, Mario.

And yeah, 100%. We love Pi, by the way, the Pi coding agent. In fact, we want to run Pi directly on Workers as well. We've been talking to him about it, because why not? But this is the future.

Yes, we are working non-stop on thisright now. If it wasn't for you people, I'd be burning tokens. Do you have any more questions? How much time do we have? I don't even know how long we have.

Maybe done.

So of JavaScript and React Cloud, what about other languages?

Would you like Python?

Sure.

Amazing. So we'll do Python as well. Dynamic Workers and Workers itself supports Python. You just need to spend a little time, like, polishing the rough edges.

First class, or?

Python is first class. JavaScript is first class. Everything else is WASM. The thing we have been playing with lately is Zig. Our boy here is one I think you're the first person to bring Zig into production in Cloudflare Workers for a bunch of things that we actually cannot talk about today.

But Zig, the nice thing about Zig is the WASM bundles are, like, tiny compared to, like, Go and Rust. So other languages, we do WASM. Or if it is that important to you to run it, like, natively, you spin up a container instead.

Again, like, we have a first-class SDK that lets you spin it up and do things. That being said, brother, your LLM is writing code at this point. Like, why does it matter,right? Like, whether it's JavaScript or not.

That was kind of my point as well. Like, what if it can write whatever it wants?

Yeah.

It's good to write whatever it wants.

Yeah, yeah. So the infrastructureright now, first class, is JavaScript, Python. And as you could tell, I'm a React programmer. So I'm like, eh, Python too hard. But no, we'll polish off those rough edges, and we'll do that as well.

Hell, we have a thing which lets you run a bundler inside a workerright now. It's literally called Worker Bundler. It pulls down dependencies from npm, strips out types, JSX, TypeScript, all of that, and generates, like, the thing that you would, like, run inside it.

So fun, because then it uses the Cloudflare cache for dependencies instead of worrying about npm downtime. It's quite nice.

When you said you have to wire up the SQLite storageright now and soon you won't, what do you mean by soon you won't?

Oh, no, it actually works. It's called@Cloudflare/shell. The APIs shouldn't break, but it's usable today. It gives you a full file system inside, layered on top of both durable objects SQLite and R2 for, like, bigger files.

I see.

It works today. You can use it today. Actually, that's just it. I don't want to talk about the history of that project. Anything else?

If you search Sunil's name on Twitter, you'll find the history of that project.

Yeah, it was a little dramatic. Anything else we can help you folks with? Anyone want free credit and shit? Like I said, friends and family company, so.

I don't know if you wanted to talk otherwise about the new CMS that we launched last week. It's called MDASH. Great name for a product in this day and age. It runs fully on Workers, durable objects, et cetera.

And, like, the nice thing about MDASH is the plug-in system is built fully on Dynamic Workers. So you know how, like, WordPress has a bunch of security incidents because of its plug-ins? Well, you just lock down where you run the plug-ins, and it just works out of the box.

MDASH otherwise deploys actually completely on any platform. It's not Cloudflare specific. And we are working on other platform support for this Dynamic Worker bit that we haveright now.

Yeah.

Other things is WeeNext. We have a version of Next.js that runs fully on Weet. And it's so fun to use. What else do we what do we do, bro?

What do we do? Well, the problem is, it's like, next week is one of our it's our agents week. So you'll be seeing a lot from us next week.

Oh, yeah. Like, twice a year, we have these announcement weeks where we get really noisy. And next week is going to be particularly noisy where we announce a bunch of fun stuff.

Yeah. Any more questions? Go on.

How do we become friends and family? What's the.

Send account ID and optionally bribes.

See, you can consider it a bribe or protection money. Like, I'm trying to also develop a little, like, mafioso vibe around our relationship. But no, dude. Like,

we the way that Cloudflare is constructed, the reason it's so cheap is because of decisions made 10 years ago. We don't build massive data centers. Instead, we install hardware near ISPs. We buy bandwidth in bulk from level zero, like, top-level ISPs.

And we do agreements where bytes that cross boundaries, you don't really pay for it and stuff. So our free accounts aren't marketing expenditure. It's just how the business is constructed. And I need more people to know that. Like, I spent a bunch of time.

I was like, how do they give it out for so cheap? So it's otherwise $5 a month, and we know people building multimillion-dollar, like, SaaSes on top of that. But if you want less than $5, well, you become friends with us.

That's cool. I think.

Other fun stuff we have.

We're at time.

Is that time? Oh, well, I guess we are at time. How is no one knocking at this? Oh, you are I'm so sorry, especially after we made a thing about the previous session. Thank you so much. We have a booth on the expo floor.

I'm speaking today at 5:40 p.m. He's speaking tomorrow. You should watch both talks. Like, look at us. Like, I got a haircut for this, you guys. Like, it's going to be fun. Also, just come hang with us. We love just, like, chatting with people.

agents.cloudflare.com or developers.cloudflare.com/agents for all the docs. github.com/cloudflareagents. You can file issues. We love that's just it. Like, have your clanker talk to our clankers. And looking forward to seeing what you build.