Building durable Agents with Workflow DevKit & AI SDK - Peter Wielander, Vercel

Thank you all for coming. Hello, hello.

Got you.

Uh, I don't know about you, but my ride agents, I like focusing on the capabilities and the features, and I like not thinking about all of the extra effort that goes into getting something that works locally into production.

And something that's very useful for that is a workflow pattern, and that's why we developed the Workflow DevKit, which is what we're talking about today. Presumably, if you're here, you've had similar issues. And today we are going to turn an agent, a coding agent, into a workflow-supported coding agent throughout this session.

So, I'm going to skip this slide. So we have an open source example ready to go. This is on the Vercel/examples repository, so you can clone that and check out the Vibe Coding Platform app inside. We're going to be using this app for today's demo.

And after we're done, we get first-class observability built in, and also durability, reliability. We get a lot of extra features like resumability, and the workflow dev kit makes it very easy to add human-in-the-loop workflows and similar things. So if you think about our general agent loop that we've all seen before, we mostly have calls back and forth from an LLM to our tool calls and our backend code,right, which would include MCP servers, human approval, any kind of async tasks.

And the usual way to go about this is to wire up some queues and a database, especially if you are doing long-running agents,right, that might run for hours, and you want to scale, and you're running on serverless, for example.

You want something, some kind of reliability layer in between, which usually is filled by queues. And then you'll also need to add a lot of error and retry code. You'll need to store all of the messages that people are sending and also between states.

And you probably also need to add some kind of observability layer in between. All of those things we are going to do today using only a single library, which is the Workflow Development Kit. It's open source, and it runs with any of your TypeScript frontends or backends, and can also run on any cloud.

We're going to be deploying to Vercel today, but this could just as easily run with a Linux or any of your cloud stacks, or Supabase, or any of your custom stacks.

Allright, so who here has heard of the workflow pattern or has used a workflow library before? Show of hands. Allright. That's less than half. I'm going to quickly explain what a workflow pattern is to make it clear what we're doing, and then in about two minutes we're going to go into the code.

So a workflow pattern is essentially a sort of orchestration layer that separates your code into steps that can run in isolation and can be retried and have their data persisted, and an orchestration layer that we call a workflow.

Different platforms have different names for that. And in our case here, the workflow part would be whatever the loop is that calls the reason, the LLM calls, and then goes back to the tool calls, and then back to the LLM calls.

And the steps would be our actual tool calls and our LLM calls.

Right, so looking at the agenda for today, we're going to be jumping into the code. We're going to add Workflow Development Kit, which is going to be quite fast, and then we have a lot of time to talk about cool additional features that it adds, like resumable streams out of the box, how to suspend and resume at any point, and how to add webhooks for human-in-the-loop processes.

At the end, there's going to be ample time for Q&A, but there is a reason that you're here in the workshop and not looking at this online, which is that you can ask questions. Please do so at any point.

Feel free to raise your hand or just shout out the question, and we'll get to it. Allright, so as I said, we're working off the Vercel/example repository, and we're going to be working off of the conf/one-base branch. Why this branch?

I've stripped a bunch of the excess code from the example to make sure that we can focus on the most important parts. And every checkpoint from this workshop will have its own branch, so if you're not coding along directly, you can also check out the steps step by step and then check the diffs and see what changed between.

Allright, so

I have already run npm dev locally on this platform just to show you what it looks like. I'm going to run a simple query. So this is a coding agent,right? It's like a code editor but without the code editing.

And it can take a prompt, generate some files, and it'll eventually show you an iframe with the finished app that is deployed. So it's mostly UI with a few simple tool calls that we'll look at in a second.

And the file system and output runs over Vercel sandbox, but you could just as easily run this locally.

Looking at the code, I'm going to go and check out our actual branch.

Looking at the code, we have one endpoint that accepts our chat messages,right? And it does some regular sort of model ID checking with us to see whether the model is supported. And in the end, it's going to simply create an agent.

Oh, yes.

What was that branch one more time?

The branch was

conf/one-base.

And we just cloned that.

You just cloned that, yeah. And you can see we'll move on to these conf/2, etc. Just look for the numbers and you'll find all the checkpoints.

Yes, so our main endpoint just accepts some messages and calls the AI SDK agent, which is essentially the same thing as a stream text call. It'll pass some tools, and internally it'll just loop for stream text, tool call, stream text, tool call.

And then it'll stream all of the chunks generated back to the client in a format that is easy for the client to understand. This is all sort of AI SDK regular code that you could replace with a different library if you want.

That is mostly there to support the UI. But again, all of the actual agent stuff is very simple and happens here. So,

oh, let's also take a look at the tools that we have. We have four tools: create sandbox, get sandbox URL. These are very simple. They just wrap vercelsandbox.create and .getURL. And similar, with run command, essentially wraps sandbox.run command, and generate files will generate a file from a simple prompt.

And we're going to take a look at one of these tool calls as an example. We have a prompt that looks somewhat like, you know, a markdown file, sort of what to do, what not to do. And my hotkeys are not working.

And

back to the tool call. We also have an input schema that's a Zod schema for what the AI is supposed to pass. This is all very standard. And then an execute run command, which wraps sandbox.run command with some error handling.

So that's essentially our entire agent code set up. And then in the frontend, we just call use chat from AI SDK to consume the stream and display things in the UI. So let's get started adding workflow to this.

Any questions before I get started?

Cool. Allright. So step one is we're going to run npm install for workflow and add workflow/ai,

which will give us the latest version. Workflow is the main library, and add workflow/ai are some helpers for, some wrappers for AI SDK that work well with the Workflow Development Kit. So now that we have this installed, we are running a Next.js app here, so we're going to extend the compiler to compile our workflow code by doing use workflow, or actually with workflow, which we can import from

workflow/next.

And that'll set up Next.js. Yes?

Verifying question. You are in the example/app/vibe-coding directory?

Yes.

So adding this will let the compiler know to also compile our workflow code separately, which we'll get into more in a second. And then for convenience, we can also add a TypeScript plugin to our tsconfig, same package, and that'll give us some better auto-complete for our workflow code.

And so we talked about a workflow being having an orchestration layer and having a number of steps. What we're going to do first is we're going to write the orchestration layer. In our case, that is essentially just the agent,right?

Does the loop, the calls, steps back and forth. We're going to add a new file. You can call it whatever you want. And we're going to take our

agent call and move it over there. And I'm going to call this our code workflow,

which is going to be all of our workflow code.

And then I'm going to go and auto-complete a bunch of imports.

Thank you, AI. So we're just passing most of the arguments that we would otherwise get from here over there. And

this completes the refactor, essentially having done nothing but pull out some of the workflow code into our file. So this is where it gets interesting. Now that this is a code, this is a separate function, we can use the use workflow directive, which will mark this for our compiler as a workflow function.

So what this does is, under the hood, yes? Question? Oh, sorry. Under the hood, it compiles all of the code related to this function into a separate bundle, and it ensures that there are no imports to anything that would have side effects.

Because the workflow orchestration layer needs to be deterministic, so it can be rerun in a deterministic fashion and does no worries about state pollution. So now that we have this, we would need to now mark our LLM calls as steps.

And because the LLM calls are happening inside the agent, this is a little bit harder to do here. And so we ended up writing a durable agent class, which is essentially the same thing as agent with a use step marker in the actual LLM calls that it does under the hood.

So now that we have this set up, we're going to await the actual streaming. And let's see if there's anything we need to do. Checking for errors. Oh, yes. We need a stream to write to. So previously, we could just write to the stream that the API handler gave us.

Now we're going to have to create a new stream to write to. We export a getWritable function from workflow, which gets a stream implicitly associated with the workflow to write to. And we're going to get into that a little bit more in a second.

But for now, we'll just pass that to our agent, and we're going to see if this isright type.

Presumably, not.

And then finally, back in our actual workflow, we need to call our workflow in a way that the framework understands, which for us is a call to start

with the arguments being passed separately, which is essentially telling it to start a new workflow

on this function. And start can be imported from workflow/api. So now we essentially have the workflow fully hooked up, and a lot of this was just pulling out some of the codes and adding a directive.

Preneer is volunteered to help anyone who's following along and has some debugging questions. Just reach out to them.

I'm on the team as well, so let me know if you guys are following along. I'll be around to help.

And finally, this start call returns a run instance that has the stream that we just ended up writing to that we can return to the UI. So this completes our workflow definition. And now we also said that we would need to mark things as steps.

The durable agent class already marked the LLM calls as steps, but our toolsright now are not marked as steps. Thankfully, this is very easy.

In the execute function for each of these tools, you can just write use step, and that will let the compiler know that this is a separate chunk of code to execute in a separate instance,right? If this is deployed to production, this would run in a separate serverless instance, and the inputs and outputs would be cached if it already ran, and it would be retried if it failed.

So I'm going to go and go through the other tool calls and also add use step to these. Thankfully, we only have four of them.

And that should complete our transformation. So now we can go and run npm dev, see if this works as expected. We're going to reload our page.

And it seems like nothing changed. Let us actually run a query.

And we can see that it's still streaming as expected. So for us, developing locally,right, all we had to do is pull out a function and then add some directives. But now, if I deploy this to any adapter, again, Vercel or an AWS adapter, or maybe you have your own, this will run in isolation with durability and all of those good things.

And something that's really nice for local development also is that if I go and

if I go and I'm going to go into the same folder here, and I'm going to run npx workflow web, which is this CLI call to start a local web UI to inspect our runs. And you can see that our run is currently still running.

And every step, everything that is marked as a step will have a span here, and you can inspect the inputs, the outputs, and any associated event. And we can see that our workflow just completed, I think. And yeah, this gets built in.

Oh, yes.

Just for clarification, every time you're prompting your Vibe coder, that is one instance of the workflow that runs to completion. So then, so each one is, yeah.

Exactly, yeah. And you could model this in any way you want. You can also model your entire, like an entire user sort of session as one workflow and have the workflow sort of do a loop, wait for the next query.

And then, again, you know, we can run code for weeks if we need to, essentially. And I'm going to go into some tools for that in a second. So now that we have this set up, you can see that on theright side, we do not get any sort of helpful feedback.

But if I visit this link, you can see that our app has likely been created correctly. Or it failed to capture some errors.

And either way, we're not getting any output on theright side. So the reason this is happening is that we are streaming the agent output to the client, but our tools aren't actually doing any stream callsright now. So what we could do is, similarly, in our tool calls, we could get the writable, which will get the same writable instance as any other, as the workflow itself.

There is an infinite amount of streams you can create and consume in a workflow. And you can also, you can tag them with a certain name and then fetch them from there. But this will get the default instance.

And once we have a writable, we can actually connect to the writable

by getting the writer. And now we can write any kind of information

to the UI to be consumed. I think we want something like data creates samples, I think, is what I hooked up in the UI. And then

we'll call IDs. We want a sandbox ID. I'll put it here. So this is me just writing a data package that our UI knows how to consume. So now that I did this, and if I reload the app and start this again, we'll see that at least the sandbox create call

presumably gets filled in correctly at the start.

You said that there are different things that you can create in a workflow. What do you mean by that?

Right, yes. So

a stream,

the workflow, so the adapter they use for workflows in local development,right, this would just be a file. In production, this might be a Redis instance, supports the workflow calling it to create a new stream, for example, in Redis,right, and then passing that stream back.

And so anytime you call getWritable, it'll create a stream, for example, again, in Redis with the ID of that workflow, and it'll pass that back. So any step can attach to that, and any client can attach to that.

And in local host, this would be written to a file and read from a file.

You're setting up, sorry, you're setting upright now the ability to stream to a sandbox and do that.

Right. So previously, we had an AI handler that took some messages, called the agent, and then streamed back messages from that API handler. Now we have an API handler that starts a workflow, and it'll pass back the stream that this workflow creates.

What this allows us to do also, I think that that was not working correctly. I'm going to restart the server just to see if that's the case.

Anything else up there so far? Getting this set up to where Peter is. Needs any help with getting things set up?

Seems good. Good point you made. Something this allows us to do is that the stream is not bound to the API handler. This means that at any point, we can resume this stream. If you lose connection to your API handler, and then the user reconnects, this stream still exists, and we could reconnect to the stream to resume the session.

This is also part of the durability aspect, where everything you do in a workflow, you can resume at any point.

I'm going to restart this query and hope that it works this time. Yeah. So now that I hooked up this data package, you can see this special UI handling for creating a sandbox works. But even after it's done, it's not showing up that it's done.

This is because we're only writing the initial loading state package. So I could go through all of our tools, and I could add more packets and just make the UI richer. But I'm going to go and check out a different branch, which is the conf/free-sleep branch, which is the next step, which already has these actually, I'll go for the workflow one.

Sorry, conf/to-workflow, which already has all of these writer.write calls populated. There's no difference otherwise. So now that all of our tools have these write calls, the stream would, again, presumably look the same as it did when we started out in this app.

Allright. So now that we have streams working again, everything is working as expected, and we have more observability, and we can deploy this with durability. I talked about resumable streams before. We're going to see if we can get this stream to resume so we have durable sessions.

So the only thing we'd need to do to make that work is to go to our API endpoint. And when we get the run instance, we're also going to return

the workflow ID

as additional information. So I can return run.runId, for example. This is just, again, any way you do this is fine. I'm adding it as a header here because we're already returning a stream. But any way you pass the ID to the UI is something that the UI can then use to resume the stream from.

So what we do from here is the UI should be able to decide whether it has a run ID and whether it should resume a stream. So we're going to go and create a new endpoint. Let's call it

id4, type slash existing ID. And then we're going to make a folder stream. And we're going to add a route handler. So this is just Next.js configuration for adding an API route add slash chat slash id slash stream.

And we're going to

autocomplete with AI.

What we're essentially doing is we get the ID from the params, and then all we're going to do is call getRun in the workflow API, which gets us a run instance. And then we can return the same stream that we return in the other endpoints, just without calling the actual agent, only doing the stream.

And

I think that should be good. We're also taking a start index, which is very helpful from the AI. We can get a readable stream from a certain start point. I think this is why it's autocompleted. So if you're trying to resume a stream midway, you can pass which chunk you were on when you initially left off.

So now that this is done, I'm going to comment out these things we don't currently need.

We need the UI to spot this conditional of whether to resume or whether to start a new chat. So I'm going to go to our chat front end, and I'm going to go pull in some code from a different branch for simplicity, which is the.

It's on the for-streams branch, which I'm going to just show for completion.

We do a useChat call already in the UI to consume the stream. And all we added now is a transport layer, which is this big block here, that has some middleware for the stream that says that if I'm trying to start this call, I'm going to check first whether we have an existing run ID.

And if so, I'm instead going to do a reconnect by calling this different API endpoint instead. I'm sort of hand-waving over this a little bit because it's client-side handling for traditionals. If there's more questions about this, please feel free to ask.

Allright. So that gives us resumable streams. And I'm also going to demo what if we wanted to deploy this and see it in production. So I'm going to call Vercel, and then we can check out a production preview example.

In the meantime, the next thing we're going to do is we talk about

expense and resumability. The workflows, because they run, the way it runs is that every step runs on its own serverless instance in production. And the actual workflow orchestration layer is only called very briefly to facilitate the step runs.

What this allows us to do is to have a workflow suspend for any amount of time. A workflow could wait for a week and not consume any resources. This is built into the workflow development kit

in a way where we can, inside a workflow, anything tagged with useWorkflow, we can simply call sleep three days, for example.

And that would, I should also wait this, that will pause the workflow for three days and then resume from we left off. If someone was trying to reconnect to a stream, for example,right, this was sleep an hour, the stream would just reconnect again to the same endpoint, and things would resume from there.

So we don't lose anything by losing the instance that runs the code because we can always restart it, resume from where we left off. And this is useful for AI agents because we can talk into a tool call.

We can have the UI, the AI agent, have a tool call that says sleep any amount of time, and then use it to make an agent that essentially uses a cron job where it says, every day, read my emails and do this thing,right?

So that would be sleep one day. Yes.

When the kind of agent goes down, that means all the state goes down?

Yes. So when it sleeps, no, when it sleeps for three days.

No, then it kind of paused, but when that would be killed for some reason, where does the state go of that agent?

Right. So the way it works is that any step call is cached. So when an input goes to a step call, we register that as an event, and we run the step. And if the step completes, we cache the output and say, this step has been run to completion,right?

So if it was something like this where we run the agent first,right, let's say we run the agent and we run a bunch of steps, the state of the workflow function at this point in time would be saved, and all of the outputs from all of the step calls would be saved.

And at the time where we restart the workflow from this specific line of code, it'll rehydrate the entire state, and it'll just go from here.

And this happens efficiently so that, again, we don't have to replay any of the code above it in a way that does any actual resource consumption.

Yeah. So we can use this to make an agent that has, it's essentially a cron job, again. And we can use it to make agents that run for weeks or interact with any of your sort of information over a very long time horizon.

And

while I've been talking, we have deployed our current app to Vercel. So I can check out this preview branch, for example. You can see that this is now live online and working just as it usually does. And

yes, it works perfectly. And if I then, again, I can

use the UI to inspect this at any point. If I call workflow inspect web, or just workflow web, with the dash back in Vercel and preview parameters, for example, that'll just let it know where our deployment is to be found.

And then they'll spun up the same UI, and now we can check on this run that's running in production. And you can see we're getting the same kind of information here.

Yeah. So this is sort of, oh, I'm not going to cancel the run. I could cancel the run. Let's cancel it.

This is to show that the way it works locally is the exact same way that it works in production from a conceptual standpoint, which is the UX we are aiming for. Allright. I talked a little bit about sleep and suspend.

Let us go and write this sleep tool call. It's going to be very simple. I'm going to go and copy the, I mean, I already checked it out here, but I'm going to copy this and write it from scratch.

We're going to write a sleep

tool call. I'm just going to call it sleep.ts. And we're going to turn down the input schema to be something like timeout milliseconds and the actual run command to be

none of this. And instead, just call sleep.

Because sleep is already a step that we export from workflow library, we don't need to call, we don't need to mark this function as useStep. But this will now, let's see if this is, oh, this should be a number.

There you go.

Can you say that again? Why don't you need the useStep?

Oh, so this is already a step that we export from workflow. It's going to be the observability will also show it as a step, which we'll see in a second. And

this should just work, assuming the prompt is good, which we're going to modify to say something like, let's see,

also this.

Yeah. Do this. Only use this tool if the user directs you to do so.

Allright. And get a double quote here. There we go. And so now that the sleep call is set up, that should be all that we need to do. We'll call it run sleep command and

sleep tool. And we're going to add this to our tools list.

And

I think I confused our compiler a little bit, or at least TypeScript.

Yes, it's worked. Great. Okay. Now we have the tool. And we also want the UI to sort of display when it's sleeping. So I'm going to add, I'm going to add another function to log sleep.

The reason we're doing this is we cannot write to a stream directly from a workflow because then it wouldn't be deterministic anymore because every run of the workflow would write to the stream again. Yeah.

So I'm trying to run the project. I had to create a Vercel API key for the AI gateway. Did that. I did that. Getting an error that says

headers missing from the request. Do you have the OIDC option enabled in the project settings? You skipped this in the beginning, but.

Oh, yes.

Project.

Yes. Because this code uses Vercel sandbox, you would need to log into Vercel to use this. My mistake. This should be running locally if you don't use the sandbox, which I will have a branch that doesn't use the sandbox for after the talk.

I

mean, at this point, I'll just do it afterwards. It's fine. So here, I'm just going to add another call to writable. And we're going to call, we're going to, let's see, we're going to need

tool call ID. And so now this is just going to write to the stream, and that should allow us to show it in the UI correctly. Let me see if I figured the UI to correctly interpret this packet.

There is no data sleep type, which I think probably wait. Yes. Allright. So now that I have this, I can go start our app again. And

once it loads, we can try out the second prompt here, which is sleep for 30 seconds and then return a reference. Just to show that it's going to correctly interpret the sleep call and then

sleep. It's not showing the data packet here, sadly. But we can go to the web UI, and we can show it has been, it's engaging in the sleep call. And this is going to return after 30 seconds.

Allright. So that's sleep. And there's one final thing, one final feature that I want to show you, which is webhooks and the ability to resume from webhooks easily. Implementing webhooks is usually quite difficult or a headache. And in our case, I'm going to check out the conf/5-webhooks branch and show you that we can, in the same fashion as we do sleep, we can add a new tool that I'll just show you.

But the actual tool call is

just a log call. And then we create a webhook, which is a function we export from the workflow. And we can then log the webhook URL to the client or anywhere else and await webhook. And this will suspend for as long as necessary for someone to click on this URL.

And then let's see if we can stop it running. And I can show you this running, hopefully.

Reload this. And

wait for human approval

before starting. Then call a Pokémon index.

And let's see if it picks up on this correctly.

I've been changing branches, so I might need to restart my server.

And the way this works under the hood is that, again, we'd be creating a URL, and we're going to sleep

the workflow until a call comes into that endpoint. And this comes with, I'm going to run this query. This comes with a lot of extra features, like I could also do respond with if I wanted to, this is a full API request handler.

I could respond with a request object. I can treat this as a, again, API endpoint. I could also check the body against a Zod schema, for example, and then only resume once that matches. So this gives you full control.

But the nice thing is it does hook up the URL internally. And you can see that it passed, waiting for a human to click on this link. And if you're running in localhost, it's a localhost link. If you're running in production, it will be whatever your deployment URL is.

Yes. Just about both sleep and human approval. Workflow is purely steps, and the steps always run to completion,right? So sleep is a step. It's not like the suspension of some sort of, it's not a suspension of the execution.

It's like it's a step.

No, it is. So we model it as a step for the observability and for how you call it. But it is an internal feature that completely suspends the workflow and all steps. Nothing is running while we sleep. You can also do sleep and another step, and you can promise that raise them if you want.

It works as a step call in that sense that it's an execution that takes a certain amount of time. And you can use await syntax to model that. But again, it completely suspends unless there is anything else running at the time.

And the same for the webhook. It's modeled as a step for the observability, but it completely suspends unless you have other code running at the time.

So just from my understanding, if you have an agent running with a workflow and keeps running, you connect to it again, let's say through another session, and you would call sleep in this session. Does that, like the previous one, just like whatever it was doing, just goes down?

So if you have two sessions, so let's say we have a coding session,right? And they've already built an app. And then it's sleeping for a week,right? And then we reconnect to the stream. Is that the?

No, no. The thing is, let's say I kick off a workflow, and it's calculating the numbers of pi. It just keeps on,right? But I connect to the same sandbox, and then I call sleep. Will it stop calculating pi?

So the way you would do this in a workflow is, again, let's see how we would code this.

Do you have a sandbox there? Is it sleep in the sandbox?

Well, you can connect to the sandbox. You connect again to the sandbox. And then some thread calls sleep. Does the whole sandbox go? So the sandbox is Vercel sandbox, which is a sort of just imagine it as an EC2 instance.

So this is just a helper for us to spin up an instance to run this coding agent, like run the code in order to store the files. If you met this differently, you wouldn't have to use Vercel sandbox.

And the sleep call doesn't happen as a bash call, for example.

Two different sleeps.

Right.

Oh, like an orchestration thing. And then when you're actually in the sandbox, you call sleep in a sandbox here.

Oh, yeah. So there are two different things. Right. So there is sleep that you could call from a terminal in the sandbox as a terminal command, or there's sleep from the workflow, which suspends the workflow.

Yeah. So we have these features for webhooks,right? And you can see that after I clicked on the URL, it resumed and then coded me a PocketX.

That is all of the features we are going through in the session. And I think we have ample time for Q&A, another 20 minutes at least.

Please, go for it.

How would I spin up Claude code sessions with this?

A Claude code session remotely, or are you asking?

Kind of run it and kick it off as an agent doing certain stuff. Is that possible? And then kind of orchestrate that as agents?

That is possible. So Claude code is, if you're talking about the app, like a terminal app,right, Claude code, then that doesn't use a lot of the workflow features internally. So it's hard to isolate that or know where the orchestration layer is.

You could write your own version of Claude code or take the Claude code source code and add workflow and use step for the calls. And that would then run as a workflow in the cloud.

There's no way to say, like, OK, I have my steps, spin up Claude workflow, kind of Claude code, type this command, and wait for anything. That would be a Vercel workflow. But how would I actually drop it, like Claude code in?

Is there one command tool that you could, when you're asking?

If you're calling Claude code in a, so made as a confusion of like where this is running,right? For a coding agent here, if the coding agent runs make the,right, for creating a folder, that make the command runs in a step, but it runs against a, like in a sandbox layer, sandbox being a VM.

And so this VM state is not managed by the workflow itself. So if you call Claude code on the VM, that's essentially treated like an SSH session. But if you run any agents or steps within the workflow,right, those steps are going to be resumable and observable through the workflow.

Yeah.

And then another question, how do I control what my agent has access to from going out to the internet doing stuff?

This would be whatever you're already doing for the agent,right? If you, in the end, you're going to be doing tool calls and stream calls to the LLM provider,right? That is in your code presumably already. And whatever you're already using to control permissions there, like your tool calls, for example,right?

If your tool call allows you to delete a resource in S3, for example, then you, as writing the prompt for that tool call, can write whatever code you want in the usual way.

It's my job to implement it, but it's not that it has some wrappers, but yeah, all in the sandbox is common.

Workflows is a general orchestration layer for durable execution and doesn't necessarily provide a sandbox for running code or running third-party code or running agent code or making files. That's something that Vercel sandbox is good for, because every sandbox instantiation is a new VM that only lasts for as long as your session lasts.

Yes.

Yeah. So if I'm writing workflows and I'm creating a lot of agent workflows through my product,

how does that, does that get queued up on your system? How does that get run? Is there rate limiting or currency controls that we can use?

Yes. So this goes into sort of

some of the returns patterns. All of this is going to be supported, and for the most part, it's supportedright now, which is that if you're deploying, for example, to Vercel,right? As usual, if you're doing Next.js, every deploy is a separate live URL,right?

If you call it, it spawns up a serverless instance. And so your workflows are bound to that deployment. So if you have something very nice that you get here is if you deploy an agent and it runs for a week, but you deploy five times during this week, those new deploys are going to be isolated from the original workflow, and the original workflow is going to run to completion.

And then any new workflow will run on the new deployments. And we'll also allow upgrading between those. So if you have a workflow that runs for a year,right, because it's like every month, give me a summary of so and so,right?

But you have new code, and you want the workflow to take its current state and use the new code for the workflow, that's going to be an upgrade button in the UI that checks for compatibility between your old workflow and the new workflow by checking all of the step signatures and all of the existing events.

And then you can upgrade the workflow. Or you can currently already cancel and rerun with the new workflow.

Is there a timeout for these workflow steps?

A time? Oh, yes. So if you're doing serverless,right, and whatever platform you're on, whether it be like Lambda or something else or Vercel, your serverless functions are going to have timeouts. The nice thing is that every step runs in its own serverless function.

So the timeouts only apply to the steps. So if one individual step you have runs the risk of running more than five minutes, maybe 15 minutes, depending on platform, then you can split it into two steps. Or if it runs the timeout,right, it'll fail, it'll retry, maybe the step will be faster.

And you will see in the UI that, oh, this step is being retried after 15 minutes a bunch of times,right, presumably because it's failing. And then you can go and split it into two steps, upgrade the workflow, and it'll just continue from there.

Yeah. The other point was around queuing workflows. Like I trigger the agent a bunch of times. Does it get queued? How does that process when you're saying?

Right. You can model this in different ways. Right now, again, we're doing this from like an API route where every call to this API route will create a new workflow that is mostly, again, the only interactable output you have is a stream in this case.

So it'll do things. It'll write to the stream. If nobody looks at the stream, we don't know that the workflow is running. You can kick off 10 of these,right? And they're going to be running in the background. And there is essentially no limit to how many you can create because they all run in serverless functions.

So you can scale for as much compute as there is in your provider, which is a lot of compute. And

you can also list active runs,right? There is an API here for doing craft interfacing with your runs. Look at all of the runs that are running, look at which version they're on, what step they're on, cancel them. I don't know if that answers your question.

Oh, concurrency, yes. You can also, soright now it's infinite concurrency, but very soon we'll add per step or per workflow concurrency where you can say this workflow is only supposed to run at most 10 times at the same time, and any extra queue addition gets, like any extra start gets queued so that it'll wait for those 10 to reduce and then slice that in.

You can also use that to have a free tier, for example, on your product, where there's 10 instances running for your free tier at any one point, and some people that come in later will wait for the queue.

But your pro tier has infinite concurrency.

Can I roll back steps in a way that, let's say I have 10 steps, but in step seven, I think like, OK, let's go back to step three. Would that be possible to kind of reset the state of the workflow?

You can technically do this. We don't currently support it, but it would be very easy to implement because we have every step, again, the inputs and outputs are cached, and we can enter the workflow at any point and sort of play from there.

So we'd need to expose this in the UI as a function to resume from step so and so. But yes, that would be possible.

More likely, what do you want to do? Do you control the workflow? Do you want to log it there and use it to write this JavaScript and call? I might have to do the step and change state or do something.

Let the event log just keep growing. Maybe my second question. So if you go through the steps, you said, OK, we're passing out input and outputs kind of across, and that's kind of what gets cached. Is there like a way to attach metadata, or does it always have to be in kind of the input outputs of the function?

You can also attach metadata. We'll have a tagging API soon where you can add arbitrary tags to the workflow at any point in the workflow run. And you can use those tags also to maybe decide whether to fit early or deduplicate your runs.

Yeah.

Yes.

About the deployment, are we tied to Vercel, or is it possible to use?

As I mentioned before, so there's two aspects to this. There's the front-end side of the framework. The docs are on use workflow.dev. You can see for the front-end sides, which is also sort of the

API layer it might work with, we currently support all of these platforms and more coming soon. And then there's separately the deployment target,right? Like for Next.js, you can deploy to anythingright now,right? This would work with anything you can deploy Next.js to, for example, or any of these other frameworks.

And we have implementations, first-party implementations for a Postgres example that uses Postgres as the durability layer. And as we'll be building this out and the community comes in, we'll have support for essentially any backend because underneath the TypeScript framework connects to any storage or queue layer.

So anything that provides a storage, a database, or a queue can be used as a backend for this.

Any related question for the observability? We also have providers to Datadog and the other stuff.

Yeah. So we have multiple things. We have an API that you can use to access data directly. And we also have open source UI components that you can use to display it. And then you can export this to Datadog if you want.

Yeah. And either the laptop is probably.

You talked about a screen a little bit and talked about how it's essentially like a cron job. Is there more scheduling and cron controls within the workflow?

So because it's just TypeScript, if you're in a workflow, you can do something like, let's say, we call sleep, for example,right? This would just be the zoom in one day. But what you can also do is this is just a promise, or you can treat it as just a promise.

So you can do

while true, sleep one day, and then

do your code and it'll run once a day. If you want it to run once a day at 2:00 a.m., you could say how much time

time to 1:00 a.m. tomorrow. Thank you, AI. And then done. And you could also wake up every hour, do some checks whether you actually want to run the rest of the code, not go back to sleep. Anything you can do with cron, you can do here.

And if you want, again, concurrency controls something or any kind of other deterministic controls, you have control flow in TypeScript here. You can check external APIs, for example, which you have to wrap in a step, but you can make fetch calls if you want to actually check data and then determine from there.

If you want to do an agent that runs every once in a while, every day, you could have a scheduling wrapper, scheduling workflow that runs as another agent workflow. Is that?

Also, yeah, you can start workflows from workflows, or you could do this,right, where you sleep a day and then call your agent. And then depending on the stream you want to write to,right, this is all writing the same stream.

Presumably, you don't want that. Maybe you could also get writable allows you to do namespace,right?

And you can get a new writable here. And then every time it runs, you can have a new stream that has a deterministic name, and you can choose which stream to connect to. Yeah.

Is there cancellation logic? Like, say I have something waiting for a long time, and then I decide to not have that be a thing. How can I just stop an existing sleep from waiting?

Right. So you can cancel your workflows from the observability UI or from the API or the CLI. All of those avenues you can call cancel. Or you can also say, well, I don't even know if I want to sleep one day and resume.

What you can do is do a, let me remove this part here. You can do something like

await promise.race. And you can do the sleep one day, and you can do some other human approval. Maybe wake up earlier if a human clicks a button than the one day. There you go.

Yes.

If you have multiple agents running, what would be your advised way of having them communicate with each other?

It

sort of depends on what kind of communication you're looking for.

Firing 10 things off and they're working, but I want to share their response.

So in steps, you have access to all code APIs or Node.js APIs, fetch, et cetera. You can have a database,right? If you want to automate over your own data source, you can have a database. If you want to have multiple agents, you can use

some of the same streams to write to and share a stream.

Yeah.

And I guess it's up to us ultimately with our steps that they're like, I demp it in. And if they have side effects when they fail halfway, that it's well-behaved. That's at your orchestration layer. That's up to us.

Right. For the workflow layer, we guarantee that there's no side effects because if you try to import some code that does side effects, anything, it'll just say like, can't compile, don't do it.

I knew that was true for workflows, but for steps.

For steps, it can have side effects. That's sort of the point.

So it's up to us. If it fails, we need to make sure that it's transactional and it's rerunable and I demp it.

There are some error controls you can add here where if a step fails, it'll usually fail with an error that tells the workflow orchestration layer that it can retry it. You can also catch this error and say, well, if it's this kind of error, don't retry it.

Instead, signal to the human to do something or try this other avenue. Yeah.

Let's see if there's something else.

Do you have one of the branches that has the complete code for what you just did?

Yes. So they all build on top of each other. So the conf/5-webhooks branch has the human approval tool call, the sleep tool call, resumable streams, and the using workflow.

Is there anything besides?

I will

see how I can post one for general access.

Can you just tweet it out?

Yes, we'll do that.

The workflows are in beta, and so inboxes.

Yes. Yeah.

OK, yeah.

Yes, I forgot to mention this. Important. Workflow DevBucket is in beta, and it's going general access in January, I believe. And we have a GitHub.

A lot of v01.

So we're using it in production, especially the durable agent stuff.

Internally, we have, I think, more than 1 million workflows have been run on a day. Yeah.

So yeah, it's mostly just like getting stuff or trying to get the API to be stable and a bunch of issues. But one of the things I love is the fact that we actually have more issues. But yeah.

If there's any feature that you need or that you really want to see, we have an RFC section on GitHub discussions for upcoming features, things that will ship by GA or shortly afterwards. And then open issues,right, where you can add any issue, and presumably we'll be able to solve it soon or someone in the community.

Again, this is all of the adapters that help workflow developers run on any kind of cloud backend or your homebrew backend. All of those adapters are also open source. So you can see exactly what's happening, and you can connect it to your own backend and own data source just looking at that code.

And we'll be happy to help you.

Can you talk about?

Yeah. What would you like to know?

The roadmap or.

Right. So for versioning, I talked a little bit about the ability to upgrade runs across versions,right? Versioning is going to be very simple where we have a cloud interface for all of the versions that you have created, which for most people will be a deployment,right?

If you deploy your code, your CI deploys your code to a predevelopment or production. Every deployment will be one version. And you can list those versions at any point using the workflow API. And the run will know which version it's running on, and you can call run.upgrade to see if it's compatible with a new version to upgrade it to that version.

And I don't know, any more things?

Where will you see version?

No. Yeah. So every deployment gets its own URL, not just in Vercel, but presumably in your, if you look at AWS Lambda, for example,right? Every deployment has its own URLs. So the webhooks would apply to its own URLs, which means that you don't need to worry about versioning except for tagging a version when you first pick the deploy.

And then whatever you think is you want to be your main version is the one you route to via your public API.

Yeah. I think, sorry, I'm kind of, obviously, I used to work at Temporal. I have a lot of experience in this stuff. I think a lot of people, originally, this is like isolation of. But sometimes you want to sort of fix in place things that have been going for a while.

Yeah, migrations almost like agent migrations to a new version.

Right. Yeah.

So this is the same as upgrading in that sense,right? But if you have a bunch of runs that are all on a certain version and you have shipped new code and you want all of those runs to be upgraded to the new version or migrated,right?

In the UI, you'll be able to select however many runs you want or through the CLI, you want to be able to get a list and then say, for these 20 IDs, I want to upgrade the run to this version.

It'll do an internal check of, can I resume these workflows from a certain point? Can I migrate them in place sort of because the step signatures overlap? Or if not, it'll offer you the option to cancel all of the existing runs and rerun them on a new version of the same input.

If you write your code in a way that's compatible, again, there's going to be different options for in-place migrations.

How would you detect that? Just by code parts not being changed?

So because we are essentially a compiler plugin, we can get full AST compatibility. And we are saving this AST, the inputs and outputs signatures, to a manifest that we're uploading for the versions. And so for every version, we can tell what are the signatures for every step and for the workflow itself and all the other things that are happening in between.

Another thing here is the workflow function itself. So I guess we played a whole bunch of times during the execution. We do anyway during the execution. So when you want to upgrade and. All the times we played is the preplay or event log that it's kind of like a dry run.

If you take the entire event log, take a new version of the code that you've run on the event log against it, you can just check it and figure out if it's still kind of the version you're upgrading to.

Sure. There's a lot of variations between kind of I do a step, all the previous steps stay the same, or this one got changed. So it's like if everything's done automatically, it feels like, OK, I could go down immediately with all my agents or up.

There's two ways you can do versioning and durability. And I think the thing with ingest is you build for a platform where you assume that your code is always going to evolve in the same place. So what we've seen is you end up with, you start your first version of your workflow to ship.

But as you start making updates to, you start adding new versions, your code now has all the full stuff in there that you have no guarantees of what version of workflow is running on the actual code that you're running.

The default assumption is that my code could be running an event log anyway. And you end up with. Starts great, and then you have to do all the mental modeling on that.

But it was the same with serverless, like killing existing instances.

What's nice for Vercel is because Vercel has already had atomic deployments forever. It's a natural step to go say, cool, we're just going to assume that you push a new version of the workflow. We've hidden everything in the workflow.

And so you don't have to worry about that mental model that code. But instead, when it's time to go upgrade an RG, push a bug, you might think, I can go loop the entire event log, replay it, or even choose, I could theoretically choose exactly how much of that needs to get replayed.

There's a lot of stuff that you can do on top of this once you have the system. But what's nice for me is that's a hard UX and challenge for us to build. When done well, you're not going to have to fight it.

Hopefully, very rarely.

I agree. You're going to talk a lot more. Yeah.

I don't know if we're close to done.

I think we're close to done. We'll be sticking around for more questions.

I guess, OK, so let's have a third party. I think the other part is observability. I poked around in Vercel. I don't see much of a dashboard. I expect that obviously, you're going to build one. And then I also want to import it into my data dog.

See where I put it on a data dog, but what's this observability service?

Open telemetry spans, which we'll be able to emit. We'll add some context to the spans by default, presumably. So if you just type your spans to the data dog, it'll already have a lot of information on the steps and event log.

And you can also emit your own telemetry, obviously.

So is that the plan, or is it you have a first party?

The plan is that we will first party support adding all of these sort of step and event log related context. We'll presumably exploit a helper to add some of this information to the spans. And then all the information you want to tag in there is up to you.

Can I attach secrets to a workflow in a way that when I need to update them, they kind of like.

Yeah. So for one,right now, you can inspect all of the input and output data,right? And it's obviously for you as someone with access to the API, which someone consuming the workflow or starting the workflow through a web API wouldn't usually have.

No, they're not.

But.

Thank you for using the steps.

Yeah. So the workflows run in the same deployment as it would usually do and has access to the process environment,right? So you can inject environment variables the way you would usually do. And as long as you don't log them, which again, presumably, you wouldn't do anyway, it's the same way as an API endpoint.

And then if you want your data to be secret,right? Right now, we expose it in observability if you have access. But we also will allow in the future to do endpoint encryption for any data. Allright. Then we'll close the session, but we'll be around a little bit more for questions if you want to look over code.