Agentic Workflows on Vertex AI: Rukma Sen

I'm Rukma. I work at Google Cloud on our Vertex AI product, and towards the end of the talk, for those of you who don't know what that is, I will discuss it just a little bit more. But where I want to start today is with agents.

So this slide, you're like, understatement much,right? You're like, yeah, you'reright. That's why we're here at this conference, because generative AI is transforming how we interact with technology. And if any of you are wondering, hey, is the rest of this person's talk filled with such groundbreaking insights?

Maybe. Maybe not. Stick around and find out. I kid. I kid. The interesting thing about this statement that I want to think about is, what is the interface of that interaction? Where do all of these, all of us, whether we're developers, employees, parents, students, interface with AI?

I would posit that for the vast majority of many of our use cases that we actually want to accomplish, that interface of interaction with generative AI is going to be an agent of some kind. So the power of generative AI, as I'm sure I don't need to belabor this point to you guys, is immense.

But it can be kind of intimidating and is inaccessible to many people. Perhaps many people who are not in the roomright now with us, but we can think about these personas, people we want to help, people we want to build for,right?

And that's kind of where I think agents come in and where they're really powerful. They're the bridge between the models and everyday users. So they help you go from speaking model language to speaking natural language. Funny joke. No, no, no.

I'm very sad. You guys, give me a laugh.

No.

No. Allright. Allright, allright, allright. I'll try. I'll try. I'll try.

We'll make it there. We'll make it there. What I think is actually really cool, though, is that for actually all of us in this room, we speak both languages. So we're going to be the ones developing these agents,right?

So we're going to be designing how they interact with people, what kinds of limits and frameworks we're putting around them to make sure that we're being ethical, we're being helpful, we're being humane, we're being safe. And that, I think, is kind of magical.

Think back to the days before the internet existed,right? What was the human interaction interface with technology? It was machines. It was things like appliances in the home. And then the internet came about. And the whole way human beings and technology interact completely changed.

We're all looking at our screens. We use gestures like swiping and zooming and scrolling. Think about how cool it could be if you were the one building the next interface, the next kind of boundary of interaction between human beings and technology.

I'm wearing a little necklace that says Wizard in Training because I think this is actually kind of magical. That one got a better reaction. OK, we like wizards in this room.

Given that, though, to quote my favorite Spider person, "With great power comes great responsibility." We all know who to attribute this to, Uncle Ben in every version of every Spider-Man ever. I promised Spider-Man to someone in this room.

I did say there was Spider-Man coming up in my talk, and I'm hoping I delivered on that promise. He's here. He's here. But the point Spider-Man is making, I think, is actually serious and something we should be thinking about.

So with the power to really shape how people are interacting with AI does come responsibility. We must ensure that these interactions are, like I said, safe, humane, and helpful. When you think about, like, what is this responsibility, I would say there are several kind of sources.

But some I would just highlight for everybody to think about are first ethical considerations. What are our moral obligations to protect users who are using these technologies that have really great unlimited powers in some ways? How can we build guardrails that protect people, that keeps them safe, prevents kind of the spread of misinformation, and make it really clear when, let's say, an agent is producing something that's generated versus when it's producing something that should be taken as a true fact?

We should also think a good bit, I think, about safety, cybersecurity, data privacy. Where are we storing the data that we reason over with these models? How are we thinking about making sure that we're safeguarding people's privacy with the rise of a lot of things like wearables and kind of just a lot of what I like to think about as, like, unobtrusive compute, where it's just out there in the world?

These become, I think, even more important things to think about. So great. I talked a lot about agents and how we should think about making them. But let's talk really quickly about what an agent is. Now, real talk.

The reason this talk was supposed to be open models is because we did have a last-minute schedule shift. And fully true story, before I knew I was going to deliver this talk and I was, you know, one week ago registering for this and they asked, hey, what is it you really want to learn?

I said, what's an agent, really? So I'm still actually really curious about this. But this talk is not actually going to focus on kind of the philosophies and ontologies of agents. If you want to chat with me about it, please drop by the Google Cloud booth.

I would be happy to discuss this with you. Can we appreciate that I got a Spider-Man reference and ontology in the same talk? I'm very proud of myself.

OK, so given that, we're just going to move forward with a working definition. And what is a working definition? This is probably the kind of broadest, most overarching definition you can think about. For our purposes, an AI agent simply is a system that's designed to achieve specific goals by interacting with its environment.

So let's break that kind of down into what its key components are. So at the heart of every AI agent is a powerful model. Often, this is based on large language models,right? This is the model that's responsible for reasoning over what are the goals of this agent, kind of determining what the next best plan of action is, and then guiding its behavior.

Think about it as your agent's brain or executive center, if you will. Then let's think tools. So an AI agent doesn't just think. It also acts. And I think this is actually a key piece of the definition, where you can separate it from something where the primary function is just thinking or reasoning or generating.

With an AI agent, you do want to have an action included. So this is where tools come in. Tools are, if the model was the brain, tools are your AI agent's hands. This is where you get to interact.

You can do things like fetch data from the internet, more complex action, calling external APIs to do things like, say, book flights, process payments, et cetera. And then orchestration is the glue that kind of holds everything together. It maintains memory and state, which is really important.

It keeps sort of track of the goals. And if in this analogy of brain and hands, orchestration is really the nervous system tying it all kind of together. So these three components work together, kind of allowing the AI agent to function autonomously and accomplish tasks.

That being said, I really quickly want to say that there are different types of AI agents. And some of these, you could say, have existed for a very long time, way before generative AI really boomed in the marketplace.

So there are deterministic agents, generative agents, and obviously kind of hybrid agents. Deterministic agents are basically following a fixed set of rules or algorithms to make decisions. So given a specific input, that type of agent is always consistently going to return the same output.

So I'm sure you can tell this is quite different from when you're, say, prompting with a generative agent. An example, a very simple example of this could be a calculator. When you give it the input of 2 plus 2, it will always return 4, unless something's deeply wrong and you're in a mirror dimension.

Let's hope not. Generative agents, on the other hand, are more creative. They kind of will work best in use cases where you want to be creative. You want to combine rules in ways that they haven't been combined together before.

And they are capable of a much wider range of diverse outputs, kind of based on the input they receive. So a simple example of a generative agent is a chatbot designed to answer kind of customer questions, a customer service chatbot.

When asked about kind of a product, it will generate hopefully helpful and informative answers based on whatever data source it has about your company's products, et cetera. And hybrid agents combine sort of the strengths of the two. An example of this could be like a financial advisor that uses deterministic agents to analyze the market and predict theright places to invest, but then uses a generative agent to actually communicate this or go out and talk about this strategy to customers.

OK, so this is, I think, where things get really interesting. So given the different types of agents, you can actually architect them quite differently across the spectrum. So from single agent to multi-agent architecture, I think, increases the kind of sophistication and complexity that your agent is capable of very, very quickly.

So just to kind of very quickly go over the single agent one, this is not, I think, hopefully new to most people. This is where a single model is just responsible for everything: reasoning, planning, acting, super straightforward architecture to implement.

You just provide it with instructions and a set of tools to kind of achieve a goal,right?

So what is the problem here? Great. Like, you know, great. Tell it what to do. It's going to do it. It's going to return the output. Well, have you ever tried a prompt like, count how many instances of the letter A are in the word banana?

And the model will say 4. And then you say, hey, can you check that? And then it will say 2. And then you say, hey, can you check that? And then it'll say 1. So in cases where you're trying to deploy a production-ready app, something like this can really be a problem.

So now we get to a much more powerful way to design agents, which is multi-agent architecture. So just like complex human systems, like, let's say, a company you work at, have people specialized in different roles working together to achieve a common goal.

That's what multi-agent architecture does. As an example of this is a customer service system. So let's say there's three levels of agent. Level one, you have a dispatcher agent. The job of this agent is simply to triage everything that comes in, assess the customer's issue, and determine where to route it.

So it triages. Second-level agents, subject matter experts. These agents are trained in specific subject matters, maybe specific product lines or specific regions, if that's how your company functions. And when they are assigned a case by that first agent, they have the expertise to respond.

And then finally, as a level three check, you also have a supervisor agent that quality checks the work against a predefined data set. That agent has the ability to go in and solve some issues, for example. Fun story.

I created a multi-agent kind of architecture once, and the supervisor agent was supposed to return the sentence, "This is not good enough. Please try again if it wasn't happy with the output." And it just kept doing that. It did not like anything my first agent did until I went back and recreated the whole thing.

OK, so as agents are becoming more and more common across industries, we're largely kind of seeing development in four types. And I just wanted to show you really quickly what a set of use cases for agents could look like.

So with customer, I already talked, for example, through what it would look like for a customer support agent, but also things like e-commerce, being able to support B2B, supporting travel if you are a travel vendor, for example. There's also internal-facing employee agents, HR, things like enrollment, benefits questions, those things.

Sales, of course, as I'm sure you can see, would be a great opportunity. Payable, supply chain. So those are kind of thinking about who the agent is targeted to. And then knowledge agents are specialized agents in terms of what exactly is their subject matter of expertise.

So you could have an agent that's specifically very good at answering legal questions, for example. And then finally, we are also seeing through the use of multimodal use cases a huge uptick in voice agents, especially in scenarios like, say, a fast food drive-through.

So I'm sure you can imagine where a voice agent would come in here. You go in, you make that order using your voice, and the agent basically transcribes that and sends it through to the ordering system so that the person at the delivery window can go ahead and serve you.

OK, so we're more than halfway through this talk. So quick moment. So we looked at why we should care about agent design. Then we kind of peeked under the hood really quickly to talk about what the kind of components of agents are.

Then we thought through architecture a little bit and kind of looked at what the top use cases are. So just before wrapping up, the last thing I'm going to do, so you can see my shirt, I'm going to talk about tooling and specifically Google Cloud's developer platform, Vertex AI.

So Google Cloud's developer platform, Vertex AI, offers essentially a full lifecycle AI development platform. So whatever it is you want to do, whether it's things I didn't talk about today, like calling models and fine-tuning them, or it is stuff like I talked about today, such as building agents, we offer you a spectrum of ways to enable that, whether that's super low code, even no code in some cases, all the way up to very high customization, high code methods to do it.

Vertex offers you access to 150-plus models, obviously all of our first-party Google Cloud models. But we also have all of Anthropic's models on there, LLaMA 2 and LLaMA 3, as well as a whole bunch of open-source models. We try to make it easy to prototype so you can get APIs for all of this and start experimenting, start building without having to go through a whole bunch of setup.

We also want to make it very simple to kind of be able to deploy and have peace of mind that your security and all those enterprise concerns I was talking about earlier when it comes to things like data privacy, et cetera, are taken care of.

So we back all of this with Google Cloud-level enterprise readiness, security, things like compute orchestration so you're not ending up paying too much for something if you don't have to, and all of that.

I wanted to quickly flash Model Garden for you, since this is the piece of Vertex AI I did not cover in today's talk. But Model Garden is where you can go in, pick your model, fine-tune it. We have a couple of model eval workflows that you can run to try to match the model to your specific use case as well.

And then finally, Agent Builder, as I said, all the way from no code to kind of full code ways to build those cool, exciting agents that I was just telling you about. The last thought I want to leave you with is this.

We're building for builders. Vertex AI is designed with developers first in mind. And all the choices we make as we build this, from training and quick start resources all the way through to deployment, is for you. So we love feedback.

Please stop by our booth. Tell us if you've used the product, what you love, what you hate. We would love to learn from all of you. With that, I will ask you to please do me a giant favor and take a quick survey to tell us how we did.

And Mayveen, my colleague in the green skirt there, will give you a cute Vertex AI branded water bottle if you show her you completed the survey.

Oh.

That's it. Thank you, guys.