Building Agents with Amazon Nova Act and MCP - Du'An Lightfoot, Amazon (Full Workshop)

Building agents with Amazon Nova Act and MCP.

I'm excited today because we're going to build intelligent, autonomous AI systems that can help you build, scale, and improve your applications and business. My name is Du'An Lightfoot, and I'm joined by—

Oh, you're going to swap it.

Yeah, swap.

Yeah.

There we go.

There you go.

My name is Du'An Lightfoot, and I'm joined by—

Hey, I'm Benjiro Byami. I'm a SOURCERS architect here at AWS.

Now, this is the AI Engineer World Fair, and I've been in tech over 15 years. Andright now is the most exciting time for me in my entire career. And one of the reasons for this excitement is agents. How many of youright now are building agentic systems?

I love it. So when we talk about agentic AI, I think it's important that we level set from an AWS perspective. There are three key terms we need to think about. First, the ability to plan. An agent gets a prompt, it gets an objective, and it determines the actions that need to be taken.

So it creates the plan. And then it takes actions on those actions by using things like tools. Now, the last piece, the third piece, and probably the most interesting, is the reasoning, where the agent is able to evaluate the results and determine if it needs to update the plan and take additional actions until the objective is complete.

This is an agent. Now, when we actually break down the architecture, I think it's important to take a look at this because we have the user input, we have the agentic system, we have the possibility of some type of human in the loop, and then we have the generated response.

Now, when we dive a little deeper, there are some components of this agentic system. We have the LLM. We have a knowledge base with external information that we may want to provide. We have guardrails to say to the model, don't do this, or to ground the model with the truth from our knowledge base to say, OK, is this actual relevant information?

Is this accurate to the information we're receiving from the knowledge base? And then we have access to additional tools, memory, or we may need to talk to additional agents or LLMs like Amazon Nova Act through something like MCP.

And we have the ability to design our own flows for these systems. Now, the most interesting piece that I think a lot of us are probably focused on when we're building these systems is around the continuous evaluation framework.

Like, how do we know if we're using theright LLM? How do we know if our prompt is consistent, accurate, or even optimized for the performance we're expecting? And then how do we even judge our system? How do we rate that and determine that it's actually solving the problems that we need or intend?

Now, once we have this, we need to log this information and then have some type of subject matter expert determine how can we improve this system. And this is the iterative approach. So we're always trying to improve and optimize our agentic system.

Now, continuing on with this story, now there are some use cases that we should be building these systems for. Like, if it's complex tasks and we don't know which tools should be used, how many tools should be used, and we want the model to leverage its reasoning capabilities, well, this is a great use case for an agentic system.

But if it's something that is just one step, our traditional if-this-then-that approach is probably the best solution,right? We don't always need to provide some type of agentic system for something that can be done with a traditional solution.

Now, when we talk about agents on AWS, there are three approaches and perspectives we should think about. First is going to be the specialized using something like Amazon Q. How many of you have used Amazon Q?

There's Amazon Q in the console to help solve your problems on AWS. In the console, there's Amazon Q Developer inside of your IDE. Andright now, one that I think I'm most excited about is Amazon Q CLI Agent. How many of you have used that?

Whoo!

For me, if you are into increasing your productivity, using a CLI agent has helped me tremendously. From editing a video, it can do that. Summarizing a document. Reading my entire code base. Like today, for one of my demos, I had some code and I was trying to figure out why wasn't it working.

I said, analyze this code and tell me what you see. Let me know the APIs that it's calling. Well, I looked at the APIs. Well, it didn't match my APIs and the API gateway. So when the code was deployed, it wasn't deployed with theright APIs.

So the agent was able to help me save a ton of time by just analyzing the code and telling me what it saw because I'd never seen the code before,right? So that's what these tools are able to help us do.

The next is fully managed. If you're using Amazon Bedrock, you're able to leverage Amazon Bedrock Agents to build and manage agents inside of AWS. And today, what we're going to be focused on is the DIY, the do-it-yourself approach by using Strands Agents.

This allows you to not just leverage Amazon Bedrock, but also leverage models through other providers using Light LLM. Now, when we talk about Strands Agents, Strands Agent was announced about a month ago. I want to say something about a month ago.

This is open source, extremely lightweight. So if you use other agent frameworks, it's like that. But the implementation is you'll see in the code how easy it is to build an agentic system or an agent itself in a few lines of code and already get started.

I built a multi-agent solution in about under 50 lines of code.

And so when we break down Strands Agents, there are three components. We have a prompt, we have an LLM, and we have tools. So you create a function called, let's say, a Get Weather tool,right? You define your agent, you give it a prompt, and it's already implemented.

And you'll see in the code as Benjiro goes through it here in a moment.

Now, taking it a step further, as Danielle presented today on Amazon Nova Act, these models are able to do some really cool things. And this is another thing that I'm excited about. Amazon Nova Act is a research preview model, and the capabilities of this allow you to use a prompt or give instructions and take complex tasks and do things like browse the internet to find research or to research or to search on Amazon.com to find the top list of widgets,right?

And then return them and then add them to your cart. So you'll see how we can leverage this, not just using the SDK for Amazon Nova Act, but also by leveraging MCP. Which leads us into the last piece, which I think when we're talking about agents, I don't think we would be here today as fast as we have moved if it wasn't for MCP.

How many of you are leveraging MCP today? Model Context Protocol. How many of you have built your own MCP servers?

I built several. I got two that I use all the time. One, how many of you use Obsidian? OK, so for my documentation, I built an Obsidian MCP server. This allows me to save all my documents, reference all my documents, and just my entire workflow is streamlined because of this MCP server I useright there.

But I also use one for my bookmarks. I built a bookmark manager because every Friday I'm restarting my computer and I lose my bookmarks. I save them and I forget about them. But now I can just say, save this bookmark.

It gives it a description, gives it a title, gives it a date. And I can even add notes so I can remember where this bookmark is. So now when I open up Q CLI, I can say, hey, I'm looking on a topic.

I'm looking for some information on MCP. Can you tell me all the bookmarks that I have? Then it'll find it. Can you tell me the ones I saved last week? And so this is the power that we have today.

But with that being said, I think it's time that we all start building. Benjiro's going to take over. But if you open your laptops and log on to this link, this is going to take you to a workshop environment where you have access to an Amazon account where Benjiro is going to walk you through building out today's workshop.

I thank you for your time.

Cool. Allright, so this is going to be a hands-on workshop. So we've provisioned an AWS account for everybody here. So you don't have to install anything on your computer. Everything is going to be done through the browser. And I always say the hardest part of the workshop is just getting started.

So some of my colleagues are also here. So raise your hand, AWS folks that are here to support. So we're going to take some time to just get logged into an environment. We're going to set up a VS Code server, enable models, get the Nova Act API key.

So again, this is the hardest part of the workshop, just getting started. So let's take some time to just get into the environment. And I'll follow along as well.

And this is, again, everything is you don't have to install anything on your computer. You don't have to use your own AWS account. Everything is provisioned for you. But while that's loading, I'm going to briefly walk through the three modules of the workshop.

So the workshop is really about how you can use Nova Act. So the first module is just getting started with Nova Act. We're going to make an API call for that. The second part of the module is going to make an MCP server that can leverage Nova Act.

And then finally, we're going to use the Strands Agent to hook everything together. So that's kind of the three steps we'll go through at this workshop. And all the code is available via the link on GitHub so you can try it out on your own as well.

But yeah, trying to get started here. If you can't follow along, I'm going to be doing up here, so don't worry too much. And again, all the code is available so you can try it out offline. OK, so the first things first, if you're following along, make sure to click this Open AWS Console button.

Again, we've provisioned the AWS account. Don't log into your own AWS account. Don't try to create a new one. Everything is previsioned here already. So I'm going to click that button to open up your AWS account. So logged into my AWS account.

So the first thing we do in the AWS account, we're going to enable Amazon Bedrock models. So Amazon Bedrock, think of it as a serverless API to access different foundation models that you can build lots of generative AI applications in it.

So it has capabilities like knowledge bases, guardrails. You can build agents on top of it for anything you need to build AI agents or generative AI applications. Amazon Bedrock has capabilities for that. But for this workshop, we're just going to enable specific models.

So I'm going to enable specific models. You can click the Amazon models. And then we'll use the Claude 3.5 IQ and 3.5 Sonnet. So those are the ones we're going to use for this workshop. And I'm just going to request access there.

And again, all the instructions are also in this workshop as well. So you can follow along, but I'm just going to go through it just for the sake of time. And then the next part, once we get the model access, there's a VS Code server that has everything set up already.

So I'm just going to go in there.

And if the URL and password is there, you can log into your VS Code server with everything installed.

And I'm also going to log into Amazon Q. So Amazon Q is our IDE extension to help you write code.

If you have time, you can sign up through a builder ID completely free. You don't need an AWS account. You don't need to put in your credit card. You can just log in through there. I already have an account, so it just speeds it up.

But it puts a nice little AI agent there. It can ask questions, update code, et cetera. So I'll show you some examples. I'll just go through some of the code.

So who's gotten to this point? Setting up all the models, workshops. Once you get all this done, then that's when the real fun begins. So just getting a pulse if I need to slow down or slow down a bit.

OK, I'll wait a bit. Again, raise your hand if you're stuck anywhere. Questions? We have agents that can come around and support you. So I'm going to just pause for a little bit.

Any general questions while we're waiting?

Do we have SLA?

Oh, yeah. So this workshop, again, all the code is available online. This workshop is available as well. So you can also look through that. There's a website called workshops.aws.

And when you go there, you can do something like Nova Act. And then it's the only workshop that shows up. So you can always go to workshops.aws. Just search Nova Act and this workshop will show up. So you can see all the instructions, all the code, and run this on your own.

OK, and then last thing, because we're going to use Nova Act, we actually need to get a Nova Act API key. So if you go to novaamazon.com, this is a website that you can use the Amazon Nova model.

So you can do chatting, generating images, speaking with Nova, generate videos. But then also, this is where the Act API key is generated. So if you're following along and you want to generate your key, again, it's free to log in.

You can use your amazon.com, like when you order something on amazon.com account to log into this. And then you can just generate a key here and then be able to access that.

Oops. OK, so I'm going to walk through what module one is. Before I get in, has anybody got in here? Just quick pulse check. If not, I'll continue. I know the Wi-Fi is slow, so it might be hard.

So I'll just continue on. But yeah, the first one, we got to see how Nova Act works, how the actual code looks like, generated the key, need to export the key, and then kind of running the first script, which is actually going to open amazon.com.

And we're actually going to look for the first copy maker. So let me see how that code looks like. Let's go here. Make this bigger. Oops.

So very simple code. With Nova Act, again, it's all in Python SDK. So I just say what page to go to. So go to amazon.com. I say, I want you to search for a coffee maker. I say, select the first result.

And I say, get the title of that product page. So very simple. If you've ever done kind of web automation before of something like Selenium or Playwright, you probably have to look for this diff tag, look at this H1 tag, grab this information.

A lot of manual processes of actually inspecting the actual website. Here, I'm just saying, click the search bar, find something. I don't have to specify, click this tag, do that. So it makes it much more easier to engage with the website as a natural human would instead of looking through divs and trying to find this P tag specifically.

So this is a great way to just use Nova Actright out of the box. So I'm going to run this so you can see.

Examples. Allright.

Allright, so added my key. Going to what happens when I run this file.

Give it a second.

Oops, it failed. Allright, let's start over.

Python 3. Start one.

Ah, OK, no.

I got to run it with this. Let's start that over.

Yeah, question?

So just explain why is Benjiro running that command? It's running fxvfb. It's a frame buffer where it runs your X11 system. What happens there? Nova Act actually goes and clicks a mouse on a browser. That's why it needs to be run like that.

Otherwise, it has no GUI. So this is just kind of a way to emulate a graphical user interface on this Linux box.

Thank you, Darko. Yeah, since we're running everything in the cloud on a browser, I'm saying, open a browser again, but it's already in a browser. So that's why it crashed. So I had to put that frame buffer command.

And yeah, the workshop kind of walks through why we did that. But you can see what is going on when Nova Act says, I'm going to search for a coffee maker. I'm at the Amazon homepage. My task is to search for this.

So it's understanding what it's doing. I see the search bar has coffee maker. I'm at the search bar here. And now it actually puts the actual log of the actual HTML file. So it's taking screenshots. You can see what it looks like.

It got the first results. I'm on the coffee maker page. It selected it. And now it got the title. Now it says, what's the title of this product page?

Allright, got this Black Decker 12 coffee maker. My task is to return the title of the product page. Product title, it got that. And it ended the session. And then it also creates a video log that I can actually look at to see what it did for everything it did in this webm file.

So yeah, question.

So does it reason about what it's looking at in terms of pixels or does it analyze the HTML?

Yeah, repeat that. We'll give you a microphone.

Thank you. So the question was, does it reason about the page in terms of pixels or in terms of text?

Yeah, so it's actually looking through the actual page itself. So you see in this video, it looks at the page. I can see what's in the page. So it's a large thing with model training. So it can actually see the actual the page is doing.

So it's not looking at the H1 tag or whatnot. It understands the context of that particular page. It can see that's a search box. OK, I'm going to go click through that search box. So yes, it understands the pixel level of what's on that actual page.

So this is kind of the video. It's hard to see. Make it bigger. Sped up.

So it opens the page. It goes to the is able to type in coffee maker there. It gets that information, clicks the button. So even if all the ads and everything, the video, it can understand the task, clicks that, and it gets the information back.

And that was a couple of lines of code. So you can extrapolate to other types of workflows you can do for searching through things.

Sorry, I have another question.

Yeah.

So what I have experience with these kind of frameworks is that when you run this on a server environment, services like Cloudflare will block the access and maybe do a CAPTCHA challenge. How do we solve that using Q?

Yeah, so with so using Amazon Nova Act, so it doesn't do CAPTCHAs. It doesn't do the that nature. So it's meant for workflows you understand. But yes, it's not going to bypass CAPTCHAs and other things of that nature as well.

So it's made for going to amazon.com or looking through a booking site. But if it's something that requires a human or it wants a, you can't bypass that.

How would you productionize?

You wouldn't use Nova Act for that use case. If you need to pass a CAPTCHA or something else, you use another technology. This is not meant to overtake humans. It's more like I'm helping them augment things. But if there's a CAPTCHA involved, then you have to use a different technology for that.

It's also in preview. It's also in preview.

Yes, this is a research preview as well. So that's a very good use case. Leave feedback on the Nova website.

Is human-in-loop possible at all with it yet?

Well, with this one, it's not because I'm writing all the code here. But again, this is Python code. So I could probably put in something here, like ask something, make an API call here. So this is a Python code.

So you might be able to create some type of workflow that might augment, like wait for a human response or whatnot.

Because the browser is happening in headless mode. But could you make it work with a browser and a human is also seeing at the same time?

Yes, yeah.

So it can pause and wait for somebody to put in a password or credentials or do a CAPTCHA. And then once it receives that works, continue on the workflow?

You could do, yeah. Becauseright now, I ran it in headless mode. But yes, it can also run it would open up the browser. If I ran this on my MacBook, it would open up a Chrome browser and go through that session.

Also, if you're running it and you want it to bypass something that has two-factor, if you're already logged into say amazon.com and then you run a code, it's going to use your credentials in that browser session to continue on to perform that task.

So that's something that you can do as well.

Cool.

So let me oops. And then one other thing, you can also do multi-parallel execution. So my next example is actually I'm trying to find multiple monitors and I want to compare them all at once. So I'll show you how that code looks like.

So I can check for the monitor, extract the information. I'm setting I'm defining what I want. So again, I'm saying I want to find the price, the rating, the size. Go to amazon.com. I set it headless mode this time, so I don't need to do the frame buffer.

I start multiple threads. It looks for each monitor simultaneously because each of these are individual tasks or can parallelize them instead of waiting it to go through. I define the list of monitors I want to go through, start the thread, and then it starts executing and finds the results of the monitors.

So I can run that in the background.

So it's starting the three parallel threads. And it's so again, it's running in headless mode. So it's going to be able to do this in the background where we can see kind of what the model is thinking, how it navigates through the web page.

Do you have a question?

Yep.

Allright. I have tried to use Nova in the past April. And it worked for the first time. But once I did it again, it triggered the CAPTCHA. Is this something that has been already resolved or is this happening?

Because I think the website and it was Amazon in this case, it was detecting it was a bot. And is there like an llms.txt or robots.txt that can declare that?

So Nova Act, there is a GitHub repo. So you could go there and just grab that. But it's working now. I'm running it. This is this live code I'm doingright now. I just exported my API key, started running it.

So you can try it in the workshop. But yeah, I mean, it's ready to go. We're buildingright now.

And you can kind of see that it's going on in the background, what it's doing. I've looked at this monitor, the Dell monitor. I'm at the Amazon homepage. It's like it's going through, looking through the search result. It's saving things.

So you can see it's running in parallel. It got the information for one of the first ones. So it's going. I just set that up. And it can execute that. So if you have some type of, I don't know, daily news thing, you need to go to the website and get news or something and have a report, Nova Act, there's no API for that.

This is one way you can encodify how to do that kind of search and get the information.

Any question?

Yeah, I'm wondering, so how successful is this in terms of more ambiguous tasks? Because I ran the Amazon demo and that worked. But I'm wondering, could I just add Google there?

Sure.

And how vague and how much does it know when it's navigating through? I was thinking, if I wanted to return a pair of sunglasses that broke, would I be able to just say, start in Google and then find this company's website, find a way to engage support, open a ticket?

How much sort of how vague can you be and how smart is it currently? Would that.

Yeah, I mean, the more instruction you give, obviously the better. But it's able to understand how to navigate a website. That's what the model is trained on. So if you say, go to this sunglasses website, it probably wasn't trained on a specific sunglass website.

But it can understand that button to support. This button is click a ticket. So it understands kind of the general knowledge of how to navigate the website. But if there's something very intricate about that website, you're going to have to encode it in the text.

Make sure you click button X first or whatever. So it understands how to navigate websites.

Got it. And does it understand when it's failed?

Yeah, sometimes. I've seen it sometimes get stuck in a loop. It's like, oh, no, I keep scrolling. I keep scrolling. I keep scrolling. It doesn't know when to stop. So again, this is in research preview. So things are getting better.

The model is getting updated behind the scenes. But it's not AGI.

Got it. And one last question. How is it in terms of navigating distrustful parts of the internet? I mean, there's a lot on the internet that we see and we know is not to be trusted or it's something not to be followed.

How have you sort of worked around that problem?

Yeah, again, it is a model in the background. So it's going to understand if you're doing some type of thing, it's not going to want to click that or it might be there are safeguards in place. So that's built into the model.

But again, it is in research preview. You still have to explicitly say what buttons to press for certain actions. But again, the model, it is an LLM trained. It's going to be able to understand the nuances and say, if it can't take this action or can't do that, that could happen.

But I haven't seen that use case. But if you keep pushing it, maybe you'll find those things.

Well, the thing I had in my mind is if you go to a site where you have to download a link, sometimes there's an ad that says download a link. And you know that that's just an ad trying to get your attention.

Of course.

Would the model know or is that some present problem?

Yeah, yeah. For example, in the amazon.com, it shows an ad for something. When I said find the first thing, it was able to scroll past that ad and click something. So the model understands the task you give it.

So yes, it can understand that.

Thank you.

Allright, so this just finished. Yeah, that's really quick. It showed it was able to find all the models, give me the size, the rating, the price for each of the monitors. So again, it executed that on parallel. It got me the nice information.

And that's kind of the idea of it can do parallel execution in the background. So you don't have to wait for it and don't see it actually clicking through the task. And you get your information. Allright, one more question.

Then we'll move on to the MCP part.

So Nova is specifically meant to be used with the browser, correct?

So Nova Act. So Amazon Nova is a family of models on Amazon. So if you go to this website, novaamazon.com, you see there are different foundation models like Nova Pro, Premier Lite, Micro. These are like the text understanding models, so like your typical LLM calls.

There's also an image model called Nova Canvas that can generate images. There's a video reel called Nova Reel where it can generate videos. And then there's also a speech model, speech to speech, called Nova Sonic. So Nova is a foundation of foundation models by Amazon to do all these types of tasks.

And Act is just another one for browser automation.

Are there plans to expand this beyond the browser so that we can someday take actions in Slack or IDE or anything outside of the browser?

Maybe. Some of the team is here. So maybe talk with them later.

Allright, thank you.

Allright, so I'm going to move on to the MCP part.

Bad job.

Yep.

Nova Act is only available in the US?

Yes,right now, Nova Act is only available in the US. It's in preview. So it's just getting started. So if you log in from a different account, like address, like UK or something, it won't work. So it only works in the US at the moment, yes.

Allright, one more question over there. And then I'm going to move on.

If you look at the three monitors, I got the same results as you did. But I guess you got a different price for the Samsung Odyssey. Why do you think that might be?

Oh.

Your amazon.com is different. I don't know, yeah, because it is opening up a different browser. So it could have clicked something differently, yeah.

It could be an employee discount.

Yeah, that'sright.

You could actually look at the video preview or video playback to see what the results were,right?

Yeah.

One more.

Oh, one more. OK, one more quick.

Quick one. Are there plans to support persisting browsing data, such as cookies, in the cloud browser?

Soright now, it's opening up its own browser. But you can also set your own Chromium profile and open up that browser. So everything you have saved there, like if you want to log into your stuff, you can set your own custom browser.

But by default, it opens up a new, completely new browser without anything saved.

Allright, so I want to show I actually made an MCP server for Nova Act. So a module tool is going through MCP. And I can kind of show you what I did for the MCP server. In fact, we can use Amazon Queue here.

So I'm going to ask it, can you tell me about the Nova

Act MCP server?

What it does

and oops.

So tell me about the Nova Act MCP server. So you can see it's going through, integrates Nova Act browser and MCP. It has the browse session tool, browser action, execute parallel tasks, take screenshots, close browser, list results. So I created these different aspects of the MCP server.

So I could use something like Claude Desktop or Cursor or Amazon Queue CLI to just say, open amazon.com and find information for me. So it's portable. It understands. So I don't have to actually write code. I can say, so go to amazon.com and find me the first coffee maker.

It'll actually write all that code I did in the initial one to do that on the multi-monitor. So I wrote a bunch of code to do this. If I just said, get me these three monitors and get the price, it would actually write all the Nova Act code it needs to do that using the MCP server.

So that's kind of the power of MCP, that I just describe a task. And then it will encode the actual browse action things it needs to. And then I also made an MCP client that can actually interpret that.

So oops.

It connects to the MCP server. It runs the code. And it's able to use query Bedrock. I am using a model. So I'm using Claude 3.5 Sonic here because it's an MCP client. It needs to have an LLM behind that.

And then it's able to understand which tools to use, run the code, and open up the browser and whatnot. So let me just run the example here.

So module two.

So we are open the file, just did that. We ask Amazon Queue to explain the file to us. And now we're actually going to run it. So Python 3. And then I can open this up.

MCP examples.

OK, so let's be adventurous. So somebody give me a query to try. Anyone has an idea? I'm going to just ask it and it'll do something. So someone give me an idea of what to run on Nova Act.

Fix Wi-Fi.

How would you fix the can you find a website to fix website? Can you find let's see, website to fix Wi-Fi. Use headless mode.

That's better on. But let's see.

Allright, it goes to google.com. How to fix Wi-Fi problems, troubleshooting guide in the box and press Enter. Return a list of the website, title, descriptions. Allright, it's going through that. So it opened google.com. How to fix Wi-Fi problems.

I see an empty search bar where I can type queries for search information. I should type how to fix Wi-Fi. So you can see it's understanding what to do. Oh, it hit a recapture page. So OK, the search results are not available, blah, blah.

So see, it looks like it got stuck on a recapture page. So this is like a headless agent. So if someone asks a question about going to pass captures, whatnot, you see that it got stuck doing that. It looks like it's stuck in a loop now.

So it sees the capture again. So I should just click the clip button to skip the capture window. The capture is still open. So it's probably going to be stuck here unless I close it. So you can see there are limitations.

It's not going to pass captures and whatnot. But that was a good query to show that it oh, did it fill it? It's still open. So it's going to be stuck here. So I'm just going to close it out.

But you can see it can't pass everything. It can't navigate the website. So something like that will not work. So that was a great test example to show.

If I use the baked-in one, find that coffee maker under $50, it'll be able to go through that and use headless mode. But any questions on that, seeing how the MCP server is working? I didn't have to write code.

I just said, do something. It actually wrote the code to do it for me.

Question?

Question over here.

Are you able to break into the browser and finish the capture and let it keep going?

Yeah, yeah. So a question about if I can actually go into the browser and do it myself. Yeah, if I ran this locally on my machine, it will actually be able to it'll open up the browser. And I can actually click the button.

And it'll continue doing that. Right now, I'm running it within the browser. So I'm doing everything in headless mode. So we can't interact with that.

So you can see it's able to find search under $50. It can actually look at the website. It's found search results on amazon.com. So yeah, for that use case where we're not passing captures, it's able to continue and find the information there.

Can it actually order something?

So a question about can it actually order something. If I use my own browser session and logged in to my amazon.com account and said, yes, order this for me, click through, it'll be able to understand that thing. But I would have to put in I would have to use my own browser session.

So I wouldn't want it to log in by myself. So yeah.

Another question?

If you give Nova Act the authentication for Amazon, for example, you give it your login details, then can it log in and complete that action for you?

Yeah, if I say, this is my username. This is my password. Enter that into that field. It'll be able to understand this is a sign-in button. And I have this information. But again, this is all Python code. So yeah, you can encode it.

You can make it an environment variable. So it won't read it directly. So a lot of ways to do that.

Does it also understand 2FA? Let's say it asks you to go to your Gmail. And will it then open the Gmail website, check the email if you're logged in again on your session, and then input it? Or is it?

Well, if there's no capture, like we just saw the capture, yeah, so there's no something blocking. But yeah, again, Nova Act is free to use. There's a lot of creativity in this room. So I think we should have a Nova Act hackathon.

I think that would be do something crazy with Nova Act.

Allright, so.

One more question.

Yep, one more.

Can I book a flight when my price alert is less than $100? It's like a continuously check.

You could use something else for that. But yeah, I mean, Nova Act can open up that website. It can just have a query every day. Open Google Flights and look at the quickest thing. And if something is below this threshold, send me an email.

So again, this is all a Python script. So you can set up something that triggers once a day, like in a lambda function. And so yes, totally possible. So Nova Act is very flexible. And because it can run in headless mode, you don't need to have that UI.

So that's really what makes it helpful for interacting with websites that don't have a native API.

Thanks. Yeah, this is pretty cool. I'm a little bit confused. We have the Nova SDK API key. And we were also doing some stuff in Bedrock.

Ah, yeah.

How does this actually work?

Yeah, yeah, yeah. So the Nova API key is separate. But for this MCP client I did, it actually needs a large thing with model to understand what's still happening. So if I go to Claude, oops. I actually said I'm actually using Claude Sonnet 3.5 for my MCP server.

So that's how because I just asked it, find that website for me. How does it know that without any of the code doing that? So it's using a large language model underneath the hood to actually find that information.

So that's where we use Bedrock for. Trying to find it in the code, but Sonnet. Yeah, I set the model ID. So you're an AI assistant helping. You have tools. You're using Claude 3.5 Sonnet. You're making an API call to Bedrock whenever something happens.

So that's where the LLM we're using. But Nova Act is separate from that. So I'm using Claude Desktop. It's running an LLM inside of that to be able to understand that for the MCP server.

A question here.

A question.

The question is, does it integrate with browser plugins as well? Could it integrate with LastPass? If you have the LastPass plugin, fill in the credentials through LastPass and then continue?

I haven't tried that. But again, you can set up to use your own browser. So if you do that and it does integrate it, it might be able to do that and click through that. But I have not tested that.

But something to try out.

Thank you.

The biggest problem you would face is two-factor. Even if you gave it a password, like if you're using something like Google Authenticator or something, that would be the biggest problem with capture. But other than that, if you provide it an environmental variable or if you give it instructions on how to access LastPass in the browser, it should be able to do it.

Allright, and one more question. And we'll go on to the last module.

So clearly, there are a lot of different agent architectures you could use. And I can imagine using this as you have a coordinator agent set up somewhere that's running in the overall app. And then when something pops up and says, hey, you need to go and look this up online, go and check it, it should so my question is, how modular I mean, it's just Python.

So it should be pretty modular,right? Is that the way in which you're imagining the architecture to be? It's just if I was coding a coordinator agent in LangChain or LangGraph, for example, it would then call your sub-agent and run its stuff and then get a text-based output that I throw into my message queue.

That's how it all integrates together. Is thatright?

Yeah, that's one way you can do it. So Nova Act, again, it's Python. So it could be a tool. It could be an API call. And the next module we're actually going to show you how to actually make an agent from that.

So good tee upright here. So Duan talked about the Strands at the beginning. So Strands is a new agentic framework launched by AWS. So let me open up the link. It's easy as this pip install Strands. And the first agent is like agent equals that.

So it's a model-first way of interacting with agents. If you used a lot of agent frameworks in the past, there's a lot of bootstrapping and making sure everything is correct. But that was necessary for kind of the older models.

If you think back to LLaMA 2, for example, how far our models have evolved since then. But now, we can pass a lot of the bootstrapping we did previously. The agent can figure that out. So we don't need all these very heavy ways and make sure everything's typed and everything so whatnot.

So here's a very simple example of how I actually spun up and also, it has MCP native support. So in this example, I actually have two MCP servers. I have the AWS documentation and AWS diagrams MCP server. So if you go to this AWS Labs MCP, these are their official AWS MCP servers.

And there's a bunch of different ones from a cost analysis, Nova Canvas, diagramming, cloud formation, lots of different ones here. So again, it's all on GitHub, AWS Labs MCP. But the example I do here is I'm actually I made a solutions architect agent.

Your role is to help customers understand building on AWS. And I define these two MCP servers here. I give it a prompt. And I say, this agent has all the tools in the MCP server. It has a Bedrock model.

I'm using Claude Haiku here. And what's cool about Strands, it can also use Lite LLM and Ollama. So it has access to lots of different things where you can run it locally. And of course, it has access to Amazon Bedrock.

So that's what we're using here. So all those three things makes the agent, the tools, the model, and the system prompt. And then I can say, get the documentation for AWS Lambda and then create a diagram of a website that uses Lambda.

So let me run this code.

CD Strands.

OK, so it uses UE to install the MCP server locally. A lot of people, I don't know where does MCP run. This is running locally. But there are other ways to run it, like in a Lambda function and whatnot.

But for just testing it out, it pulls down the MCP server locally and runs it. And you can see it's already executing. So let's make this a bit bigger.

So it says, OK, I'm going to help you with that. First, I'm going to search the AWS Lambda documentation, read the documentation, then I'll create a diagram illustrating a static site. So you can see it does a POST request to do the search.

So the MCP server defines where everything is. I don't have to feed it in the well-architected framework. The AWS documentation is always updated. So it just knows call the search function. It got the Lambda welcome file. It put that in.

It's able to generate the diagram. It generates the diagram. It tells us what is going on, how the workflow looks like. It tells me it saved the diagram to this location. So I can open it up, generate a diagrams.

Oops. And oh, it's very small. Let me see if I can make this bigger.

There we go. So it was able to generate the diagram for me. So all through about

40 lines of code, I have two MCP servers. I have my prompt. And it's able to understand that, get that, and just generate something for me with that. So that's very easy to get started with Strands of building agentic workflows.

I know agent means a lot of different things to different people. But you have tools, the model, the system prompt do some type of action. And Strands makes it extremely easy to do that. If I use other frameworks, it could be a lot more code to do something like that, especially integrating MCP natively like that.

I'm going to pause here for any Strands questions.

I

know Bedrock already had its kind of agents SDK. So is Strands replacing that? Or is this now the is this replacing that? Or is this supposed to complement that? Is this the preferred way of creating agents with models in Bedrock?

Yeah, when it comes to preferred way, it always comes down to your use case. So the Bedrock agent has a lot more, I guess, opinionated ways to do things. You can do it through the console. It has built-in supportright there in AWS.

While Strands is more of an open-source framework. So you can download the code. You can use other models through that, like Lite LLM or Ollama. If you use Bedrock agents, you can't run that offline. So there's different use cases, different developer tooling.

I mean, me as a software engineer, I like code first doing things. So it does depend on your use case, what you're trying to do in your experience.

Can you show the code real quick?

Yeah, yeah, this is the code. Yeah.

Just show the agent. So this is an open-source framework. If you go where it says agent and it says model,right now, we're using a Bedrock model. But you can use another model with Lite LLM. So you don't need AWS at all in that instance.

You can use Ollama. You can use OpenAI. You can use.

Right.

Anthropic. So it doesn't matter.

Yeah, so there's documentation in Anthropic, Lite LLM, a lot of different model providers, Ollama, OpenAI. So it's an open-source framework. So you can use it whatever you want. But yeah, that's the idea with Strands, open-source model agent development kit.

One question.

Suppose I want to build a text-to-square agent. And I have, say, 15 tools already built in that I want this agent to be able to use. If I use this framework, how can I make sure that the agent knows when to use theright tool and the sequence?

Yeah, great question. So this example, I have a weather agent. So one thing you said you already have tools. What I like about Strands a lot is I can write a Python function I already have. And that's put this tool decorator.

And that's it. You don't have to put anything else. It understands this is what you need to do. And then when I'm going to that agent, I have these tools. And it has put in the native tools we're going to be using.

HTTP request is a standard tool in the Strands framework. So in this example, I'm asking, what is the weather in Seattle? And then also, how many words are in this response? There's this open API, api-weather.gov, where you don't need an API key.

And it can find the information for you. So I'm going to just update this. San Francisco. And I just showed it wrong. But it should figure it out, weather example, weather word count. And I was very specific, find the weather first and then how many words are in the response.

So it's able to use that tool. It gets the forecast. And then it knows to use that word count tool next. So we're passing a lot of the information to the model. The models are very smart now. We don't have to say, do this, do this, do this.

Let the agent figure it out. That's kind of the goal of the agent. You give it the context and the tools necessary. It figures out the best way to solve the problem.

But then wouldn't it be prone to hallucination when you give it 20 tools? Because we've tried that with AWS Bedrock, no, the similar things. When you bind

more, I think more than 10 tools, it's going to.

Sure, there's always a balance. But again, the models are much better. Try using Claude for Sonnet. Is it hallucinating as much? These newer models are much better at understanding the concept and understanding what tools when. The older models, sure, they get confused.

There's so many things. But I'm very confident on these newer models. They can understand your use case and what tools are available and figure out the best way to solve the problem.

So then with this framework, there wouldn't be a way for you to orchestrate a customized flow, but more like you give the control to the agent?

You could. If you want to have specific, do this specific way, there are different ways in Strands with something called workflow mode, where you actually say, this is the workflow I want to do. Research results, analyze things, write a final report.

If you have to do something very sequential, a Strands has that. I won't have time to go through all the different ways to do multi-agent collaboration and whatnot. But for that specific flow, I wanted to do XYZ first.

The workflow way can do that. So yes.

Then is it possible, say, I don't have a predefined workflow, but I know it needs to figure out theright workflow?

Then that's what I just did there. I just gave it a sentence and figured it out.

I see. OK, perfect. Thank you.

You can have a touch on it. But Claude 4 has something called interleave thinking. I believe that's what it's called, where it can handle multiple tools processing much better than most models today. So if you're passing in 20 tools, it's able to work through the agentic loop to really figure out which tool to run.

And it's also able to run parallel tool calls. So rather than just say, OK, here's the objective. Let me run this tool. It can say, here's the objective. Let me run this tool, this tool, this tool, and this tool.

It then processes the results and determines what needs to happen next. So I would try Claude 4, which like Benjamin mentioned.

And then last example, really quick. Again, Strands, I made my Nova Act MCP server. And it can actually run that. I define, this is the MCP server. Use the Nova Act MCP. Use Claude. So same type of thing.

I can have another agent use Nova Act as well. So Strands make it very easy to build these agentic workflows. So I really enjoyed the developer experience of using Strands. And I already have the MCP server. We see the same exact example before.

So once you have the MCP server, it's very easy to just plug it into different architectures. And Strands makes it very easy to accept that.

But yeah, those were the three modules, really, about how to use Strands, MCP, then Amazon Nova Act. Again, Strands is open source. You can download it, pip install Strands. If you just type strandsagents.com, it'll take you to the documentation.

Again, also Nova Act, novaamazon.com. It's free to log in. And then I think that's all the time we have. But we do have a survey. And you can get AWS credit code by filling out this survey.

Benjamin, I have a question about that workflow thing in Strands. When you create these individual agents, can you define which tools are passed on to each agent?

Yeah, yeah, that's a great question, Eric, about different agents. We're running out of time. But I'll quickly show. I have a multi-agent example, I believe.

Oh.

I think you had it in the docs.

Yeah, yeah, it's in the docs, yeah.

Yeah.

Yeah, each of these is a different agent. So this is an agent. You can have a different system prompt. You can have different tools. So you're just defining the agent. And then yeah, you can have different tools, different whatever there, different models.

And then the workflow would just call that. So yes, completely customizable. So that's the good thing about Strands. It's very easy to customize and build scalable solutions like that.

Thank you.

And then again, here's the survey. You can get AWS credits for filling out this thing. Tell us how we did, what you liked, what you want to learn more. And now go build.

Any other questions while we wait? I think we have a minute.

Thanks for the presentation. So as these systems develop, I think that it's reasonable to assume that they would emerge as an increasingly effective vehicle for committing fraud online at scale, which would push businesses to implement more things like CAPTCHA, which kind of decreases the surface area that tools like this would be applicable.

So what is the long-term strategy for that?

Well, you already saw we failed with CAPTCHA today. We're not trying to bash CAPTCHA. We're not trying to break things. Responsible AI is very important to Amazon. So no, we're not trying to let this tool commit fraud. You have to have an API key.

So it could be monitored. So use cases like that will be shut down.

Are we done?

I think we're done, yeah. So thank you all.

I think it's finished.

Oh, we can keep going? We have more time? Oh, the clock ran out. So I thought we were kicked out. Allright, well, more questions then, I guess. Yeah, another question.

So regarding Nova Act, let's say that I have a headless browser in the cloud. Is there a way to connect Nova Act to my custom browser instance in the cloud?

Yeah, yeah, yeah, there's a way to put your own browser instance. So yeah, Nova Act supports that. So totally possible, yeah.

Thanks.

Let me go to the Nova Act GitHub page.

And there's some examples there.

I think,right? Because it says you start at 1:00. And then you have 120 minutes.

I know. I just told him. But I think what happened that time he was over.

Oh, yeah. No, you guys can keep rocking it.

OK.

Yeah.

Yeah, so yeah, there's a way to set up your own user agent for Nova Act. So definitely possible.

So we still got time. So if people keep building or if they've got questions, I think I don't know what happened with the time. But we still got time.

Allright, well, apparently, I have still more time. So I don't know if anyone actually got into the workshop. So we can still

build some stuff. Or I can try some other examples.

Try to make Nova Act.

I tried to make a Streamlit app with Nova Act. So we can try if that works.

Oops.

So one example I tried, I tried to make a Streamlit app that looked for the top five PlayStation games on GameFAQs and then create an image, like a nice graph for me. But it can fail. So I think that's one of the issues there.

I think it failed at one of the steps there.

Let's see. Oh, Nova Act got an error. So it couldn't navigate GameFAQs.com. So it does fail at some of the things. So that's, again, research preview. You have to be more specific in how it goes through things. But yeah, let me show you where the code is, just so you can have an example.

Let me pull up the code.

Oh.

Do you want some data handles more?

Yeah, let me try. Let me set up my local machine so we can see how it works, yeah.

For Nova Act, how much does that.

Oh, yeah, go for it.

How much does Nova Act depend on semantic HTML and good web design to actually work?

I mean, it understands the actual page. So it can click through those things. But if the page doesn't have a search box or button, it'll not be able to navigate. So as long as it can see the page, understand where to click, and then click those correct buttons.

Maybe a follow-up. Is there any efforts to do experimental engagement on the page? So if it comes on a page that it's not familiar with, maybe it would try and act like a human would to click on things or try things out.

Depending on what you put in that prompt. Because again, you're creating that workflow, what it should do. So if you say, explore this website and find things, it'll try to click through that. But again, it's up to kind of what that initial prompt is that you have for it.

Yeah, when you use Nova Act, you're kind of giving it step-by-step instructions when you're using the SDK. So that way, if you kind of know it's an obscure website, you can give it those instructions that it needs to perform.

Rather than the MCP server, it's using natural language to infer what needs to be done. So it's not specific instructions coming from you, unless you provide it.

Yeah, so I'm going to run it locally on my machine just to show an example. Let's see. Nova Act examples.

Oh, let me hide my key for a second because they're just being recorded.

Python Git coffee.

Thank you for coming.

Allright, so I'm just running it locally on my machine. So without headless mode, so you can see it opens up the browser.

It's able to type Coffee Maker.

So what we're looking at now is not in headless mode. This is actually Nova Act actually performing a task in the browser.

Yeah, a lot of questions about how does it work. And we can try more complicated examples that I just wanted to show. It can work on your machine.

And then you can see the log. I'm just looking for and if I change the page while just doing something, it's going to mess up. So I'm going to click the page and see what it does. So someone asked about click things of that nature.

What's it going to do now?

So see, it crashed now because I changed a different page. It didn't know what to do. So examples, you can interact when it's going through the motion as well. And then I believe I have an. I can show you the MCP server.

I set up a Claude instance. Oops.

And then I have my Nova Act MCP servers there. So I'm able to actually, I click this. You can see all the tools it has available. So I can ask it to navigate a website. So

anyone having a complex example can see the MCP server. I know some people have been asking some complex examples. So go ahead and give me one.

Yeah, you got one?

One question I had is, can Nova support drag and drop functionality?

We can try it. Do you have a specific website that has drag and drop?

I guess

Draw.io.

Let's go to draw.io and make a cool diagram. Use Nova Act. And let's see what happens.

Allright, so I go to draw.io. Allright, I open the page.

Do I have to accept something? Nope, it's going. Oops. Allright, I opened draw.io. Let's see.

Make this smaller.

Wait for a page to load. Look at my initial setup for attempting selections. Allright, it's going.

Oh, it crashed. What happened? Oh, do I have to allow? Allow always. Oh, it took a screenshot.

I need to continue the browser session to see what's available. Let's look at the screenshot. Allright, it's opening the app again. It's going to draw.io.

Yeah, if I keep clicking away, it clicks back to the browser session. So I need two monitors.

Let's see. Is it going to figure out how to use draw.io?

Wait for a page. Take a screenshot. Look for template options. Click on blank paper. Allright, so it's kind of I didn't give it any specific instructions. I just said make something cool. So maybe that's too hard to interpret for this website.

Maybe I have to say, click the square button and then drag the square to the center or something. I might have to have been more explicit for that.

It seems to have frozen. Allright, it's clicking something. Allright, click new.

Oh, OK, it's doing stuff.

Again, it's not like super real time. It's going. It's not like instantaneously. But it is clicking through the buttons, clicking through stuff. Allright, did it do anything? Oh, it declined. So it looks like it failed. So yeah, it looks like Claude failed that one.

So I won't blame Nova for that. But that's the last idea. So thanks for trying to do something hard.

Allright, another question back there? Oh, yeah.

Can we try another one?

Yeah, let's try another one, sure.

Can we do

on Google Maps, find the top three rated coffee shops within a mile radius of this hotel?

Top three coffee shops

near the Marriott Marquis in San Francisco.

It'll figure it out.

Allright, open Maps Google. Search Marriott Marquis, San Francisco. Wait for results to load. So it has a plan. It's going to do something. So let's see. It opened Google Maps.

Allright, type Marriott Marquis, San Francisco. So it's able to type that.

OK, it searched. It found the Marriott Marquis.

So there's a coffee button. Let's see if it clicks that. I'm curious.

Looks like it's frozen. Give it a couple more seconds.

What did it click? It got this 15 minutes. I was trying to type in that box. OK.

Allright, it's typing coffee shops. Allright, allright, it's going.

Allright, so allright, it opened the coffee shops. And let's see if it can get those top three. There's a 4.8, 4.7, and another 4.7. Let's see if it can get that.

Did it crash? I think it did it. But I think I'm going to blame Claude. Claude desktop crashed. Might need a different MCP client.

Yeah,

I think, yeah, I think Claude desktop doesn't like doing that. But again, because it's an MCP server, I can open up a different MCP client. So I can open Cursor, for example, and ask it questions through that.

Cursor, then.

Let me close this. And then you see it has the MCP tools. Oops, it has this up. Let me just open up a new one. I can do the same thing and use Nova Act.

And then it's calling the MCP tool again. So that's the beauty of MCP. I already have this server. I can just use a different client. It can understand all the information it needs to and do the exact same command.

So it's going to do the same thing. Cursor might be smarter than Claude code. But yeah, it's able to do the exact same type of thing.

Another question over here?

Yeah, I just got a question on the Nova Act model.

Yeah.

That model, is that running in the cloud?

Yes, so Nova Act question was, where is Nova Act running? And yes, it's running in the cloud. So yeah, you get that API key, and it's doing the call behind the scenes in the AWS cloud, yeah.

So then what does it upload to the cloud?

Well, it's asking the questions. And go to Google Maps. And then it's able to understand that. And it's actually clicking those buttons and doing the actions. So the actual intent of what you're trying to do in this specific action.

OK, and if I was using it locally, there's a lot of space for my.

You couldn't use Nova Act locally. It has to be connected to the internet to use it.

OK, but if I, for example, though, if I wanted it to look at my Gmail.

Oh, yes.

Would it then be able to have access to see.

Ah, yeah, I see what you're saying. Yeah, yeah. I mean, it is, it's an API endpoint. It's been passed to AWS. So only passing information that you feel like it's not going to be without training the data or taking any of that nature.

But it's going to the AWS cloud and processing what to click on this button locally on your browser.

OK, cool. OK, thank you.

So looks like it's not, yeah, see, now it's even searched in the rating. It actually knows which rating to press.

So the Nova Act is going to just give the plan,right?

Right, yeah, yeah. Well, Nova Act is executing. In this MCP server example, I say, find the top three coffee shops in Marriott near the Marriott Marquis. And then I'm passing that information to the LLM to understand that plan.

And then it uses Nova Act to interact with the browser. Because Cursor or Claude code or Amazon Q, they can't interact with the specific

website by itself. It uses Nova Act to do that.

Right, but given a question, though, how does it come up with the plan?

Oh, the MCP server, the client. So I picked the model. In the example, we had the MCP client. We showed the model. I didn't use the Claude 3.5. That's coming up with the plan. Same thing here. I asked, help me find the top three coffee shops near the Marriott Marquis.

The model that Cursor is using is coming up with that plan. And then I'm using the Nova Act MCP server.

To act on it.

Exactly. So this is the plan. Search for Marriott Marquis. Click the Marriott Marquis. Search for the things. And you see all this information Nova Act returned. And it actually returned this time. So I think the problem was with Claude desktop.

But it got the top three coffee shops there.

Right, what are all the tools that Nova Act can do today?

So the MCP server is what I wrote. But the idea between Nova Act, it can interface with the web browser. That's the tool. The browser is the tool. And it can anything on the website it can actually click through, go through the example, et cetera.

I see. Got it.

You got the repo? You got an architecture that shows the MCP just so they can see it?

Yeah, so I mentioned there's an official AWS MCP server. So this AWS Labs MCP. And there are a lot of different MCP servers here. For the one, the Nova Act one, I created my own one. You can go back to the Nova Act examples.

Or where did the, ah, here. When I used Amazon Q to explain the MCP server for what's going on, what tools, the browser session, performing an action on the browser. So this is a good thing to talk about.

So can you dive deeper on the browser action function? And we can see, because this is how it's actually acting. So

Amazon Q, browser action, is designed to perform actions.

It has this, what's cool about it, it just does, oops, let's go into the code. It performs a single action in the Nova Act browser. So it's executing that action. It stores this act.act is like what Nova says, click the search bar, do this.

The MCP client understands how to use this act.act that passes the correct action. So we saw the example here. One of the actions was like, go to Google Maps, or click this button, or do that search. That's how it's able, these actions.

And then the Nova Act MCP server is translating that to actually click that button. So the MCP server provides all the interfaces it necessarily needs. So then these MCP clients can interact and do actions and do things. And Nova Act is just the model in the background that's able to click those buttons.

Extending this question, so your MCP server, so Claude or Cursor running locally,right, it's calling your MCP server that's also running locally. And your MCP server is the one that spun up, I guess, the Chromium instance,right?

Yeah.

Is your MCP server taking screenshots of what you see in Chromium and shipping them to Nova Act?

The screenshots are locally. And then based on that, you can see it's actually getting all the information, the final page information. So it's not storing your screenshot data and sending that. Everything that is running locally, it's clicking those buttons based on what's on the browser session.

Got it, but is any of the information in Chromium, does any of that need to be sent in any form to Nova Act?

No, no, everything running, yes, running locally. That was the distinction.

OK, perfect, thank you.

And let me open up the,

where was I looking? Ah.

So one of the things about making MCP servers is you have to provide a lot of context. So for Nova Act, like I say, when writing actions for Nova Act, be descriptive of what to do. Click the hamburger menu icon.

Go to order history. Don't find my order. So the more concise and descriptive of what you want to do, it's better. Search for hotels in Houston. Search by average customer. So the better specific it is, that's how the MCP client is able to make those great requests and find the information.

So type coffee maker, search box, enter. So the more descriptive you are with Nova Act, the better results you're going to be. And I encoded that all into this MCP server so the clients can leverage that.

So I think that's probably one of the hardest things about making the MCP server is just making sure you provide enough context of when to use the tool, how to use the tool, the inputs and outputs. But once you solve all that, it's very easy to just plug and play the different MCP clients like we've done here.

Question, yeah.

With the same example of the MCP client, how do you make sure, or how does the model make sure that Nova Act executed the action successfully?

Right, so what Nova Act is doing is passing back the log of everything it's doing. So you know what steps it did. So the starting page, the results, the action, result ID. So it's keeping a log of everything it did.

So it's able to get that JSON and understand what the ID, what the result is. So you can see what it's doing. So it can move on to the next step.

Yep, let me make this bigger.

Another question?

Sorry, a quick question.

Yeah.

Is this able to do automated UI testing because of this?

Well, with Nova Act, you can define what you want it to do. So you're going to have to define, go to this button, click this. Does this work? So you can define that workflow. So I mentioned before, back in the day, if I'm writing Selenium code, I have to click this H1 tag, do this.

Now you can just write in natural language. Click this button, click that button. So yes, it can handle that use case specifically of opening the browser or checking these things. But you have to, this Nova Act, search for coffee maker.

You specifically have to write what buttons to press.

Yeah, thank you.

Let's see. I guess if you have time, I can show some multi-agent collaboration with Strands. That could be something cool. I think I have a repo for that. So

it should be, go to the AWS Labs page. Where is that? How's it working?

And then Claude, cool.

OK, I'm just going to copy this code and put it into our environment.

So in this example, I'm actually going to show how Strands does multi-agent collaboration. So one way, I'm actually going to create a PowerPoint presentation based on a cloud migration request. I want to move my infrastructure on-premise to the cloud.

Give me a presentation of how I would do that. And so for this, I created three different agents. I created a cost analysis agent. So I have a system prompt there, a solutions architect agent to map out what you're going to be doing.

And then each of these tools is an actual agent. So this cost analysis process has the docs MCP server, the cost analysis MCP server. It has its own prompt. The presentation agent has its own system prompt. It has a tool from the PowerPoint MCP server that I'm using.

And there's an architecture agent that also has its own specific tools, system prompt, et cetera. So different agents for different things in the workflow. And then I have this orchestrator agent, what I call the migration orchestration agent. It has a prompt.

I tell it what tools it has access to. And then the cool thing with Strands is I make this orchestrator agent. And then the tools, or just other agents in that. So it knows when to call this agent for this particular tool, when to do that.

And I say, I want to migrate my workload. So write tools to find that. So I made a fictional company called ShopEasy eCommerce. They have on-premise Java, MySQL database. They want to zero down proper migration, all these constraints in there.

And I wanted to make a migration plan and a PowerPoint presentation that I can present to my executives of how this would work. And I just designed, and the orchestrator agent will find out what to do. I don't specifically say, do this one first, do that first.

We'll let the agent figure that out. So let me run that, Strands. And it should be multi-agent.

Right, so Cloud Partition, Agent as Tools. Allright, again, so all the MCP servers running locally. It downloads it using the UX. It starts with the architecture design first, generates a diagram.

I'm going to use WAF. It takes some time. It might fail, but it will just update itself. Making another judgment.

Allright, I think it couldn't generate the diagram there. But it's saying, allright, this is what the diagram should have. This is what we're going to be doing.

Now it's going to do a cost analysis, cost analysis on base of the things we did there. So this workflow takes maybe a couple of minutes to run. But you can see it's calling all these agents different things.

It's understanding what to do, what actions to take first. It's finding pricing for EKS because it has a cost analysis tool. It knows where to find that information. So it has the up-to-date pricing all the time, funding for Aurora for its database.

So it's able to understand all that information and get the real-time up-to-date information just because we have that pricing MCP server from the AWS Labs example.

Does region matter for this?

Pricing.

US West, too.

This is it, oh, cost analysis.

Just pre.

Yeah, cost analysis MCP server, documentation, all the stuff you need for finding theright price on AWS. It has all that information. And the agent was able to just use that once going to generate a report.

So we can compare.

Yeah, got it.

So it's still running. Again, this does take a while because I'm asking a very complex question, a lot of things going. So it does take a couple of minutes to run through all that. It gets its monthly spend predictions, monthly savings, et cetera.

So it's able to understand all the information and get all up-to-date information based on the plan we provided.

And the last thing, now it wants to create an executive presentation. So it downloaded the PowerPoint MCP server. And now it's going to make a PowerPoint presentation based on that.

Going to be a role for the next year tool again?

Yeah, so adding the title slide. So add a placeholder. So. Generating PowerPoints is a very popular use case. And there's an MCP server that can go ahead and just do that, add bullet points, et cetera. So give it a couple of a minute or two.