Conquering Agent Chaos — Rick Blalock, Agentuity

My name's Rick Blalock, I'm from Agentuity, and, uh, you're all wearing the logo around your necks, so stop by our booth, please. But today I want to talk about conquering agent chaos, but really I'm talking about deployment, running and deploying agents.

And just to give you an example, I was, uh, I was at University of Florida—I'm from South Florida—and I was at University of Florida about a month and a half ago, and I was talking to a bunch of professors and their students on projects that they were deploying.

Almost all of them were working on some type of agents in their class projects. And I asked all of them, "What was the number one problem?" And they all said, including the professors, "Deploying agents." And they—similar problems that I had before we started Agentuity, which was like, you build this thing, you deploy it on serverless—in their case they were using AWS and Lambda—and then they, you know, timeouts,right?

"Oh, my agent runs for 15 minutes," or "My agent runs for 30 minutes." We have an internal agent at Agentuty that runs for 40 minutes a day, something like that. So they were running into that. And then they weren't allowed to use VMs or EC2 for their projects, for obvious reasons.

A bunch of students running at EC2s. And a bunch of other things: they were running into the gateway issues, you know, wiring up agents, talking to agents, and things like that. So it's a very, very common thing. And the other one is, you know, the whole web is built, like, for a stateless type concept,right?

Agents are not stateless necessarily; they're stateful in a lot of cases. And so those are the deployment headaches that I was hearing over and over with the students. And that map to a problem I had about a year ago building a qualitative research agent set of agents—same thing—I built it on serverless, not thinking, and then I realized, "Ah crap, I got to re-architect this whole thing," because it's doing synthesis and it takes a while.

Very common headaches. Has anybody experienced that headache before? I'm curious, yeah? Deploying agents. So that led us to talk about, like, well, what do agents need to be successful? So they need to run as long as they need to run.

And they need to be able to pause and stop and resume. They also have—decoupled from your code—they need to have different inputs and outputs. And, um, actually multiple, various ones,right? I think we understand that, hopefully. And then I put this up here: introspection, self-observability, and self-reflection.

They need those two. And we have that in the product. We have, like, OTel tracing things. Hopefully you've all seen that, some of those types of things where you're doing introspection and traces across. But I put self-observability and self-reflection here because this is important.

So the observability things that we're used to are human-based, where we see, like, these traces and spans. And the agents need to understand that as well. There's a difference there. There's a nuanced difference there. And of course it needs memory.

It needs to get better: evolution, code execution. So that's what agents need to be successful. And so when I was, again, talking to those UF students, personal experience, we need those things. And that's what we started to build at Agentuity.

And so I just want to give you a demo. I'm trying to decide, should I do a live demo or should I do the slides just in case? I don't know.

Live? Everybody says live? You all just want to watch the train wreck. Allright, we'll try it live. So I'll—wow.

It's all live.

Live, wow. Allright, we'll do that. So I'm going to—let's do this. Let's do this. Let's blow this baby up. Can you see this okay?

Yeah.

Allright, good. I was told I had to do it in light mode because that's what you do, so. Allright, so we have a CLI. So if you're going to start, you run Agentuty create to create a project. When you run it, drum roll.

Okay, in this case I'm logged into a bunch of organizations, so I'm going to do our Agentuty demo one. We have three runtimes that we support: Bun, Python, with UV. You notice we put UV. We're trying to bias people towards UV.

Please don't use pip. And then Node.js, of course. So those are our three runtimes. I'm going to pick Bun. And then you can pick several templates as a starting place. Now we're framework agnostic. You notice there's MOSTR here.

MOSTR is pretty good if you're going down the TypeScript route. I'm going to pick Vercel, AISDK, and Grok. A lot of our stuff internally, we have 50 agents or so, and we don't use a framework. But we're framework agnostic.

So you can bring CrewAI and then deploy another one in LangChain, another one in Pydantic, and then Vercel, and then they can all talk to each other. Internal networking talk to each other and that kind of thing. We'll show you that here in a minute.

I'm trying to leave enough time so we can have questions and answers. So I'm going to pick Vercel, AISDK with Grok. Honestly, that's my favorite combo with Bun because of the speed. It's amazing. And then I'm going to name a project.

Let's just call that project Hello. A project is a grouping of agents. It's also—you can think of it as like a one-to-one to a repo because you can hook it up to GitHub. An agent, again, multiple agents in a project.

You get your default one here, so let's just call this Agent1 Hello. Very creative name. Now, one of the things we give you: agents are a first-class citizen. Again, back to deployment problems. Agents are a first-class citizen, infrastructure citizen, in Agentuity.

So that means we handle all the routing and all that to that agent, and you can decide how you want to protect it. So you can protect it via the whole project. Each agent in your project can have an API key.

I'm going to do noneright now because it's a demo. There's more to that, but I'll skip over thatright now. Then you can hook it up to GitHub so that if you merge domain, it'll automatically deploy your project. And that's it.

So then we create this. I'll show you the code here in a second. I'm going to show you dev mode real quick, and then we'll do the code. Allright, so rocking on. So let's do project Hello. And before I show you the code, we have now this Vercel, AISDK, Bun, you know, Grok template,right?

So we have this command that you can run any of your agents with, Agentuty dev. If you have multiple projects, you can run them, and then they'll have ports assigned to them. And so now we're running this locally, as you would expect.

Very what you're used to, I'm sure. But one of the things I want to show you is—so this is the routing to the agent. We only have one agent, so that's the agent ID. But we have this public routing, so we do tunneling for you.

So if you wanted to hit it through another service or something while you're deving, you can do this publicly too. Allright, but let me just show you the dev mode real quick. So click this. Oh, of course I got logged out.

Ugh.

Who expired the token? Okay, so we give you, like, this kind of simulator for your agent. Some agents don't take just text,right? So you can pick JSON, HTML, you can PDFs. Actually, we do emails and things like that too.

That's not in this dev mode. But this is where you can interact with your agent, test it out. And then when you do that, like, I'm just going to run it a few times.

What?

I'm sorry?

You want me to say something?

Should I say something? Oh, okay. When you do that, you can get your logs. Of course, you can see it in the terminal too. You can inspect the logs, get more information. And then the sessions are back to the memorability thing.

So we have the human version of this here, where you can look and you can see, like, oh, this hit our AI gateway, which we have an AI gateway, so you don't have to set up keys for any of these models or any of these services.

And you can get the cost for that call, what the prompt was, what the response was, and all that. And this works in production too. It's a mirror image of production. So that's the dev mode. I don't want to spend too much time on it because we don't have much time, but that's—so put a pin in that.

Now I'm going to kill that and you're like, well, Rick, what does the code look like? And we're trying not to be too opinionated here, but we do have a few conventions. Let's do that. Let's do that. So really the conventions we have in a project are: you have a YAML file, surprise, and that's to help configure your agent when it's deployed.

And then there's an agents folder. Python, it works the same way. And you have an entry point. And so this is the simple, simple template that we created here. And the entry point, in this case in JavaScript, would be you export a default function, and that's your request handler.

If you guys have used Next.js or you've used Knative or whatever, everybody should be familiar with that, hopefully. But what we do is we handle all that routing that comes into your agent, and we put it on that request object.

And so that could be an email. You can add an email to your agent. You can add a phone number to your agent. You can send a JSON, a PDF, whatever. We have all those things that we route to it.

You can stream it. And then also you have things that are like first-class citizens, like CTX, GET agent, and then you can pass in an ID or a name. So we have another agent. Maybe it's a Pydantic agent over here.

And we can actually get it. We'll get an ephemeral token so it's allowed to talk to it for as long as it runs. And then you can execute it, send it back, you can stream it back. There's a bunch of different things you can do here.

And so that's on this context object. So there's—I'm not going to spend a lot of time on thisright now, but there's a lot of, like, infrastructure we give you in this really simple handler, which is pretty cool. Allright, and that's it.

That's the only convention. So again, like, you can imagine if you're using MOSTR, and we have a bunch of examples and templates that you can go look. MOSTR, you create MOSTR template or models the way you would normally do it, and you can drop them inright here.

Same thing in Python. There's a run function in Python, and that's what you have to declare. And then you can drop in a crew or a couple crews and then have it call Pydantic to another agent. And that's it.

That's the code in a nutshell. Now let's deploy it really quick. YOLO.

Allright, and it should—the internet's fast. It should upload pretty quick. There, boom. Allright, so we took that agent, we wrapped it in a container, a specialized container that's, in this case, it's a Bun runtime, and now it's live.

So if I click on that and jump to it, you can see this is when it was deployed. There's our little happy Bun. And let me show you something really quick that you haven't seen yet. So in a project, again, you have multiple agents.

I only have one agent that you saw me create. And then this is where you can get cost breakdown by project, by agent, by run. You saw the session spans. So we try to surface that up in all different ways to help understand how much an agent is costing.

But this is where—remember when I said you can decouple inputs and outputs from your agent if you want to? And this is where I can hit plus here and I can say, hey, I want to give my agent an email.

I hit save, and now I can email my agent. So there's the email addressright there. And I can do that with SMS. I can do that with APIs. I can do a cron job. Wakes up, does a bunch of things.

And that's inputs and outputs. So it's a nice way to build an agent, and then if somebody wants to use it, they can hit deploy now from your GitHub repo, drop it in, and like, okay, let me wire this email up.

Let me wire this up. Later, we're going to get Slack and Discord and all that stuff too. So that's inputs and outputs. Actually, let me just go ahead and try to hit thisright here. So this is our—remember we didn't secure it, so it's public.

So actually, if you took a picture and hit this, it probably would work. Don't do that.

Let's just—I don't know if this is going to work. Hello.

Allright, so I hit the webhook. It created this session. There's the session ID. Now the agent's off running, and I should start getting logs here. I got three minutes.

There it is. Oh, it's an error because I probably—oh, okay. So this is in your YAML file. You can change the memory, and I needed to change the memory footprint for this, so it didn't run. It's like, nope, that's something we're working on too.

So in a nutshell, that's what you get. Of course, you can wire up some integrations and—actually, I think I just lost internet. Okay, there we go. There we go. You can wire up some integrations, GitHub, and a bunch of other stuff that's coming.

So to kind of wrap up, this was like the thing that we wanted to build, not just for everybody, but for us, because we have now 50, 60 agents internally built on this, and it's like, finally, we're getting really good velocity, and I'm really excited about it.

So this is our first product in our Agent Native Cloud suite that we plan on working on. This summer, we're working on infrastructure agents that watch our logs and surface things that are coming up for developers and us.

And we kind of—we were joking last night, we should call it Agent Duty, not Pager Duty. Like, the agents are going to do that stuff. Right, exactly.

Same like it.

So yeah, so there's a bunch of stuff coming, but that's where we're atright now. Stop by the booth. We've got one minute and 50 seconds, so if there's any questions, we've got about a minute and 50 seconds. Any questions?

Jokes, stories, anecdotes? Try to leave just enough. Yes.

One thing that I've been hearing about recently is the idea of adding tool calls into reasoning, like into the reasoning tokens. Is that something you've been thinking about at all recently?

Yeah. And I mean, yeah, for sure. Yeah.

Now, for us, like I said, from a product perspective, there's a thing we always ask ourselves, which is, what does an agent need to be successful? And some of that is what—like, we want to offer that as a services layer.

From an infrastructure services perspective, where a lot of that stuff you just get out of the box, and then you can hook it up to a Vercel AISDK tool thing, or you can hook it up to something else.

We want to do some of those kind of service level things. But I mean, like on a tactical level, yeah, I think it's—I mean, I need them actually for a CRM agent that we're working on. So yeah.

Cool.

Allright, thanks everybody for coming.