The Demo I Wish I'd Had: OpenAI's Agents SDK... serverless! - Brook Riggio

What does full-stack AI engineering even look like today, in the serverless environment? Let's talk about it. My name is Brook Riggio, I'm a technology partner with Thrive.com, and I'm excited to share with you some things I've learned about running production AI engineering workloads in a full-stack and serverless environment.

So let's dive in. The question of 2025 is: how do we deploy ZeroOps-resilient, agent-powered, user-ready apps today? How do we make it happen to get agentic workflows into the hands of users as AI engineers? This is what our job is, this is our goal.

Let's think about how we do this in a modern way. There are, of course, so many options. We are overwhelmed with different pieces of the infrastructure, which basically is going to need to look like this: some type of client app, an agent framework, an orchestration layer, all running somehow serverlessly in the cloud so we don't have to fiddle with all the details of managing ops.

Of course, each one of these options themselves has so many options. You could be running things in Remix, Astro, Vue, Svelte, Next.js, and so many more. For agent frameworks, there's probably a new one announced every day. Now we're probably hearing more and more about new agent frameworks like LangChain, Vercel's AI SDK, Flowwise Agents, OpenAI's Agents SDK as well, and so many more.

There are also dozens of orchestration layers that we could be playing with: Temporal, AWS Step Functions, Langsmith, Ingest, and so many more. And of course, so many different serverless environments to deploy to: Lambda, API Gateway plus Bedrock on AWS, Google has Vertex, Azure has AI Studio, Vercel, and dozens of other serverless cloud environments for us to deploy to.

So many options, and probably in combination at least 42 million combinations of these things. I don't even know what's going to be the best, but I'm going to tell you about my favorite. Because the question AI engineers are asking is: how does it all work together?

How do we get theright pieces of theright layers to actually function? And if you run this through an LLM, you're going to get lots of different answers. You're going to get some code that maybe might kind of work.

Even if you pick your favorite stack, there's not really likely to be a bunch of reference code in the training data. And even leveraging original docs can prove very difficult. I put this talk together because I struggled with this very thing, trying to make it happen over and over again in a production environment.

And I went through a bunch of different combinations and landed on my favorite, which is what I want to tell you about today. As maybe I gave it away in the talk description, but my favorite is indeed a combination of using Next.js with its built-in streaming, first-class server actions, file-based and nested routing in Next.js 15, and a deep integration with Vercel hosting makes this my choice for client apps.

For agent frameworks, from the first day that OpenAI announced their Agents SDK, I was all in on it. It's got native tool calling, one-shot multi-agent calls, built-in tracing, and eval hooks, so you can really get a lot of observability out of this thing.

And of course, strong backing from OpenAI. They're supporting this, it's not going away, and they allow you to interchange models so you're not stuck in their ecosystem even if you're using their open-source Agents

SDK framework. So overall, then, for orchestration, my choice is Ingest. Ingest uses events to trigger things. You don't have to manage JSON documents of state machines or anything like that. It works entirely on demand. There's no worrying about keeping your servers warmed up.

It's got automatic retry, step-level observability. You can see what's happening step by step and where the errors are, and a really nice one-click integration with Vercel. So you can probably guess what I'm going to say: for serverless, let's go with Vercel.

Vercel's got preview deployments with every push. This makes continuous deployment no longer a difficult concern, but an actual just integrated part of your workflow. You can have your whole team reviewing your latest changes on preview branches, and even as you're getting that merged into main, Vercel deploys it all for you so you can see it functioning.

It's an incredible experience. If you haven't tried it yet, I'm here to tell you you need to try it. They also have a really strong edge network, automatic CDNs, a bunch of integrated database options from Postgres to Redis to blob storage.

And of course, since Vercel is the company that's behind Next.js, there's a really strong alignment with developing the ideal cloud platform for the Next.js infrastructure. Between Vercel and Next.js and the whole suite of tools that you can find in Next Forge, there's a really strong ecosystem of pretty much everything you need to make things happen in a client-based app.

And with these other parts, you get to integrate your AI workflowsright into the client app. So here's the beauty of it: this set of tools actually works. Now, the LLMs still don't know how to piece this all together, but that's what I'm here to tell you.

Maybe they can start training on my repo and they'll get there, but for now, you're going to need to understand how to get this implemented so you can guide your vibe coding session in theright way. Here's what the architecture looks like.

Where your user is going to connect to your Next.js app, that is going to be maybe checking the database: is work done? Is there something cached? Do I need to do something new? We can trigger a new workflow with an Ingest service, sending an event to Ingest.

Ingest is your orchestration layer. That manages your connection to Python serverless functions where your AI agents are running. The OpenAI Agents SDK is Python-onlyright now, but that's okay because when you deploy your Python functions to Vercel, Vercel recognizes it as Python and automatically hosts it for you.

Those Python functions can handle all your inference, talking to OpenAI to get the work done, and return results to the orchestration layer, which can, again, inform your front end, your client app, that the work is done, caching things in the database as needed.

So this is the essence of the architecture, but really, I'm sure what you want to see is the code. So I built an example app that uses AI agents to create a newsletter, and I want to show you how it works so that you can get a sense of how all of these pieces work together.

So let's take a look. Here's the README. This code, there's a link to this code in the notes below. It's up on GitHub, and you'll get to explore it there. But let me run you through it so you can see what's happening.

So first of all, we're highlighting that we are focusing on serverless scalability, the ability to support long-running jobs so your agent tasks don't just crash out or blow out the time limits available for cloud functions. That lets you really pay attention to costs.

You can adjust, you can pay only for what you're actually using and not keeping servers idle. All of these tools work together for a really nice local developer experience. I'll get to show you a little bit of thatright now.

And of course, you have full type safety with Pydantic and Python and TypeScript across Next.js. So how do you make it happen? Well, once you've cloned this repo, all you need to do is fire up three terminals: one to run your Python agents, one for Next.js, and one for your Ingest dev server.

This is a really nice feature of this orchestration layer. They give you a specific development environment to manage it. So let's see what it looks like. We'll just go like this, fire up three terminals here.

Get Next.js going.

And we get NPM going. Allright. With those running, we can pop over and take a look at our app. Like I said earlier, we've got a README, and let's see what the app looks like. Here it is, up and running.

This is the homepage of my Next.js app. As you can see, running on localhost:3000. And it provides a place to enter some topics, comma-separated list, as many topics as you want, and it will attempt to generate a newsletter that looks at what's going on at the intersection of those topics.

Let's see what's going on with AI engineering in Seattle. That's where I'm based. And we can generate a newsletter that will look up the latest on what's happening at the intersection of these topics. So we fire this off, and it creates a page that's starting to just poll against the database to see if the database has the answers for us yet.

And we're using Vercel's blob storage for our data store. You can use anything you like, but that's easy to plug in. And in the meantime, we can look at the Ingest server and see it running. It looks like one of our agent calls has already completed, and it got back some data.

And there's the second agent call that's fired off. It has saved it to the blob storage and did a little finalization. So here you can see our orchestration layer has broken the whole task down into individual steps. Each one of those steps can scale up and down to the full length of what Vercel allows for a cloud function to run.

You can configure that for up to a minute, I think, on the free plan, and even higher on pro plan, so you get plenty of bandwidth to play around with individualized tasks when you break it out that way.

And then overall, as the page refreshes, it finds the results in the database and presents back to us our newsletter about what's going on in Seattle and with AI transformation of engineering. All kinds of interesting things about the Seattle AI community and its own take on what we're talking about with engineering.

Allright. So there is the results of our page, and formatted very nicely as a newsletter. And we can go back and generate more as needed. So let's take a quick look through the code to get a sense of how all of this works.

So we were looking at the README, and now we can see the project structure here that lays out everything in our app. Ingest itself has an endpoint that the orchestration server connects to and shows us what's happening. We get all the detail and observability we need about what's going on step by step in our codebase because it's servedright out of our app.

We have then the newsletter API endpoints that Ingest calls, and that in turn invokes the Python agents, which are over here at a top-level API directory. When Vercel finds this top-level API directory, it looks in and serves each file as its own independent function.

So we get this as a cloud function. Otherwise, the workflow is defined in an Ingest folder that's within our source code folder. And we can have as many different types of workflows as we want, each one triggering off of its own event.

So there you have it. Those are the pieces that work together to make this happen. Ultimately, it's made up, once again, of AI agents that are powered by FastAPI, a really quick and easy way to put them together.

Right? So here's a look at our AI agents. We have one that's doing research. We have one that's formatting. And each of those has its own endpoint in the FastAI app, which can spin it up really quickly. All Vercel needs to know to get this going is this one line of code at the bottom.

It fires it off as an independent file. And you'll note here that this repo does not even have a Vercel.json config file. Vercel automatically picks this up when you have things structured this way. That's something I went in circles around with the LLM on in numerous tries to get this whole stack deployed correctly.

So we've got the AI agents, and I'll show you two also here, the Ingest workflow,

which is found here in our Ingest functions. We define them here. And I really like to structure these so we keep each step defined at the top, and each step is essentially just doing one thing of calling out to a function that we define below.

And of course, in JavaScript, we can define those functions later on in our code, but this keeps our overall high-level orchestration very clear what's going on. We're essentially invoking each step and then doing some type checking on it.

Step one, call the agent. Step two, format the newsletter with another agent. And step three, save this to our blob storage so the front end can find it and load it up. So that's what Ingest provides as very nice, lightweight orchestration.

One thing to remember is that every step.run will be invoked in order, but code that's outside of step.run that you have, for example, up here, may well be executed multiple times as it comes back to this function to figure out exactly which step to run.

So keep an eye out for duplication of execution outside of those step functions. But inside the step functions, you can count on them to reliably run in sequence and passing the results to a variable that you can then send into

your next step. Right? So here we have the results of the initial research agent. After type checking, we pass that into formatting our newsletter. We take the results of that formatted content after type checking and pass that into saving to the blob.

And then, of course, that's our final result, which Ingest shows usright there in that Ingest dashboard. So that's it. All of this fits within a Next.js repository, deployed to Vercel. It fits together very well and allows for all the scalability you would expect from a serverless environment, all the resilience that you need from a robust orchestration layer, and most significantly, all the agentic power that you can get out of OpenAI with the Agents SDK.

So this is my favorite combination, and it's working well. It's working well. It deploys really nicely onto Vercel. You can see here it is running on a Vercel URL, and Vercel shows us the full logs of how things get built.

If you're building this, here's a place to check. Under the initial deployment, you can see your API agents folder gets deployed as a Python function. It's running Python 3.12 in Vercel's current Python environment, all laid out clearly for you here.

Full build logs are available if you need to do any troubleshooting, but ultimately, it just is as simple as actually running the Vercel commandright from your terminal to deploy this code to Vercel.

I'll show you that too. We have one more shell to pop open. The Vercel CLI command lets you sign in, connect to your environment variables. You can actually pull down your environment variables from production and run them locally.

For this project, you need an OpenAI API key, of course. You need a Vercel blob storage token to use for storing your data. And that's about it. Those two environment variables are all you need to get going. And you can fire off your build processright here.

That gives you everything you need to make it happen. So that's a tour of the app. That code is up on GitHub. Feel free to check it out. Make some contributions. It's open source. It's ready for you. I'd love to know what you want to do with it.

If you build on top of it, take it and replace the agents with your own workflows, then link back to it, and we'll build a whole section in there with examples of what other people are doing with that same basic structure.

So that's what I've got for you. Thank you so much for your time. I hope you find this helpful, and I can't wait to see what you do with full-stack agents in the cloud. Thanks so much.