The Adversarial Path to the Personal Assistant: Sumit Agarwal

Good afternoon, everyone. God, I am so in love with generative AI. I don't think I've been this in love with anything since my 20s, when I was a puppy. Uh, I love what Swix said at the last conference.

He said, "We're too late to explore the world, too early to explore space, but we're so lucky to be alive because we're the generation that gets to deliver on the promise of AI." And we're all going to make that promise come true for the world in a lot of different ways.

I want to talk to you about one way that we at Ario are going to make that promise come true, and it is by giving people back what I think is arguably the most precious commodity in the world: time.

Our goal at Ario is very simple. It's to give every single person that wants it one hour a day back in their lives. One real hour of things that they don't have to do. Because there's so much that we have to do that isn't fun that still has to get done.

I love this quote that someone put out. It's an author named Joanna. "I don't want AI to write poetry. I want it to do my laundry so that I can do the wonderful and beautiful creative things." We love it so much that we got her permission to stick it on our homepage.

This, I think, is what captures the real potential of what Gen AI can do for people. We're going to deliver on this promise in a way that you've all heard about before, but there are going to be some riffs that we talk about throughout the, uh, throughout the session.

We're building an AI-powered personal assistant. Not an executive assistant for busy executives that have lots of money, but a personal assistant that helps ordinary people in their lives. Now, you've heard a lot about that technology and that capability.

I'm going to tell you how ours is a little bit different. We're announcing today that we've raised $16 million to make our version of this come true. I'm going to tell you why. Thank you so much. I'll tell you why investors are so excited about that.

Our system is very simple. The foundation of the system is something we call adversarial ETL. The ability to get users' data, every Google search, every Facebook post, every Doordash order, every Strava run, every Instagram, everything that they do online, even though the large systems that have that data don't actually want you to get it out.

I don't know how many of you have actually gotten a few hundred meg of your data out of those systems, but I bet that the percentage is less than 1%. So we start with adversarial ETL as the foundation of our system.

Aggregate all of that user data on behalf of a user. Then we build an agent architecture on top of that. The LLM, the RAG, data management. We package all of that up in use cases that are simple and make sense to users, that add value in their lives.

So I'm going to talk about as many of those components as I can in the time that we have remaining. And before we dive in, I want to tell you that I'm as tired as you are of rosy visions of the future that are ultimately mirages and fall flat on demo day.

So everything that I'm going to show you today works. Mostly. If you try it a couple of times, it should work for you. To that end, you can go to our iOS app in the App Store. You can take a photo of this.

It'll be online. The URL is heyario.com. And you can download it now. You can download it tonight, and you can play with these capabilities. So let's start with the very first and most important thing: getting the data out and understanding the user.

I'm going to show you a few examples of what we do and how we orchestrate this data from data sources like Google Calendar, Amazon, Doordash, and a few others. Two quick notes. Data privacy. 100% of this data belongs to you.

It actually belongs to you whether or not I say it. We simply acknowledge and respect that fact. We take no rights in the data. We never move the data. It's your data. You have full crud. You're technical people.

You can create it. You can read it. You can update it. You can delete it. You have full ownership of the data. Uh, we will operate on it at your instruction to create value for you if you want.

Second, when you use the app, I need to apologize in advance that connecting data sources is hard. There is no easy way to deal with that from an authentication, authorization point of view. It is just painful, but if you suffer through that, there will be some pots of gold at the end of the rainbow that I think are interesting.

With that said, I want to show you the very first thing that we do after a user has downloaded our app and gone through, you know, one, two, three minutes or so of a little bit of friction in connecting their data sources.

We generate for them, and this is my co-founder Mungmung's, uh, application, a data portrait. So that's not a random piece of generative AI artwork. That's a data portrait that represents her and reflects back to her things that she cares about and likes.

She loves photography. She has an SLR in her hands. She does drone photography. She has a young child, their toys, little bikes in the background. She's a big avid hiker with her husband. So all of these facts about who she is and how we reflect back to her things about her come from her data.

Come from the various data sources that we have. I'm going to take you into a little bit of a wall of text, but that's what happens when you click Read More. We show the user all about themselves, what we've learned.

So she has been taking a lot of daycare appointments because she has to put her son in daycare. Uh, she likes going to Japanese restaurants with her husband. Uh, she has been ordering particular things in Doordash from certain restaurants.

And so those kinds of things are now captured in her profile and can inform her engagement with the system. So all of the things that I'm showing you are auto-magically derived. You can edit these profiles. You can edit these forms.

You can correct things that we've gotten wrong. And I'll show you a few things that we've gotten wrong. But you also don't have to do that work. So there's no intake form. There's no essay. There's no 20 questions that just immediately go stale when you're done.

It's an ever-evolving, ever-updating profile and understanding of you. Let's click into this one for a second. There's that daycare thing again. Her son, Hunter, is really into Legos and trains. She took him recently to the California Academy of Art.

On Amazon, she bought for him sand toys and rash guards because they're planning trips to the Santa Cruz or to the beach. Uh, we guessed in here that he goes to kindergarten. He's a little younger than that, so you're a little off on that.

The precision is hard. Uh, and at the bottom, her mom is trying to avoid, uh, high sugar foods because there's a glucose intake medical issue. So these are the kinds of details that you pick up from people based on their data, on their Google searches, on the things that they tell the system explicitly in conversation.

And then you use these things to serve the user much more effectively. I'm not going to take you through every single one of our data sources that would take too much time. We do it with Amazon. We do it with Doordash,right?

We extract an understanding of you to build that profile from each data source that you connect. We do it from your Twitter profile. We understand what you like to read, who you like to follow, who follows you, et cetera, et cetera.

So there's one last component that's not a data source, but it's memory. Based on the conversations that you have with the system, based on explicit things that you tell the system, like, "Hey, would you please remember?" or "I'm interested in XYZ," the system will remember important things about you in a separate section called memory that you can go and edit and update.

Uh, and I've compared this to ChatGPT, and I find that ours is a lot more robust. We've put a lot more time and energy into tuning what needs to be remembered. And we're very excited about what this enables, and I'll show you how it comes into play in a minute.

So now we're going to move to demos. And I'm not quite as brave as Emil. And so all of my demos are the real product, but they're just videos of the product so that I don't have to worry about them not workingright now.

So here's the first one. Let's hit play. This is now a Copilot example in which we're going to run one of the tiles that Copilot likes to promote, and then we're going to run that exact same query in Ario.

So this is the tile that says, "Where's a sunny, warm place I can travel toright now?" Copilot says, "You can go to Belize." Here's why. It's got beautiful beaches. You can go to Mexico. You can go to Key West, Florida.

I would call this response really little more than Google++. It's a natural language interface on a Google query that may as well say, "Top warm places to visit." Like, it's fairly unanchored in anything that's likely to be highly relevant for you.

Those are nice places to visit, but it's just a sort of hodgepodge of nice places to visit. When you run that exact same query in Ario, you get something that's a lot more tuned to your interests and your likes, and the system even tells you why.

You might want to go to Maui because of your love of Asian food and because of the activities there that are relevant to you. You might want to go to San Diego because they have Legoland and you've been buying Lego toys for your son.

You might want to go to Miami because there are certain things that you've been doing lately that match the things you've been doing in the Bay Area, and so on and so forth. So it's early days, but you can see that adding a lot of personal data to that query allows us to give you far more intelligent, far more t-tailored and tuned recommendations.

And we haven't really even begun. Oh, next slide. You can imagine that where this goes next. Find something that matches my schedule. Find something that harmonizes between grandma, my husband, and me. Find something that matches my budget because you know what my budget is.

So as we add more and more data sources, we're able to perform more and more sophisticated queries that go way beyond just saying, "You should go to San Diego because you like Legos." That's fairly easy. I would call that practically a string-matching query.

But how do you get to something that demonstrates an understanding of your life and gives you much better results as a result of that? We'll do one more example. So here's a query that we ran in ChatGPT versus the one that we ran in Ario.

This is one of the promoted tiles there.

Oh, we're having some networking here. "Help me create a personalized morning routine." So this is a failure mode of AI that I call 20 questions. It's not very fun. You ask a question, but really the response is a whole series of questions.

By the time you're done answering all of those questions, you may as well fill out an intake form. You can get that kind of a recommendation just by going anywhere. I mean, you can go to, uh, any number of websites that offer you the ability to sort of create routines if you're willing to answer so many questions.

You run a query like that in Ario, and you get a much different response. So we type that same thing in. "Help me create a personalized morning routine that will boost my productivity." Great.

So you get kind of a play-by-play schedule. 7:00 a.m. Wake up. That's tuned to the time that this user actually wakes up. Hydration and light stretching. That's because there's a record of the kind of exercise this person does.

They actually do a small amount of exercise. Only 20 minutes. Quick workout from 7:10 to 7:30.

We'll keep going.

Shower and get ready. The food recommendations are based on what that user orders for actual food, actual ingredients, based on the things that they like from Doordash. So you can see that adding in all of this personal context really, really takes us from what I like to call Google++ to something that is far more relevant.

Something that feels like it knows you.

I'm going to skip two of the demos in the interest of time. And we're going to get to the part where you guys figure out how you use this. We do the same thing in Gemini versus Ario.

I want to show you one more that we really like. Uh, one of the use cases that we like to work on is busy people, busy parents that are managing complex households and lives. Some of you that have children might be familiar with a schedule like this.

And even if you don't have schedules, or children, rather, everyone has schedules, uh, you might have seen something like this. A lot of information densely packed in a single place. You take a photo of something like this, and you upload it into Ario, and you get all of the information on here published into your calendar.

Now, that's just standard sort of knowledge extraction. You've seen a lot of people talking about that. But we marry that with the context of your life. We know what all the conflicts are on your calendar. We identify those conflicts, and we tell you 24 hours, 72 hours, and 7 days in advance.

So you can kind of set and forget. You put this large amount of information into the system, and now there is an entity that's looking out for you. All of those entries are now scattered across the calendar. I'm going to show you one more little detail here.

This is so typical of schools in particular. They publish the schedule, and they don't even have nailed down the first day of school. So they're going to tell you in email later. So you sort of live in fear of not even being able to being able to be sure of when the first day of school is.

And so you'll see that both of those calendar entries are on the schedule because the most important thing is holding them. You can figure out which one to let go of later. And, oops, in the next version, uh, which is going to roll out in the next four to six weeks, Ario will sit in your inbox and monitor your email for that particular thing.

We can't broadly monitor email yet. That's a little hard. That's a big problem. But we're special casing one category of information at a time. And a big one is where your kids are going to be. Where you need to be.

So things like scheduling, things like the first day of school are actually going to be monitored in the system, and then that incorrect calendar entry will be dynamically removed. You'll be notified, and now you know. So the idea is to take some of this unnecessary headache of keeping track of all this disparate information away from you.

So I want to share, uh, a couple of very, very high-level insights on RAG that we've been generating as we build these tools out. So I've shown you getting some data out. I've shown you orchestrating that data on behalf of the user, and I've shown you some use cases.

So rule number one of building tools and systems with LLMs is avoid the LLM at all costs. In other words, do as much data enrichment, as much processing, as much heavy lifting as you can with old, baked, cheap, reliable tools.

I'll give you one micro example. We were working with Strava data, which has a start time and an end time. Start the run at 1:05 p.m. end the run at 2:00 p.m. How many minutes did I run? Who's fast with math?

55 minutes. But there was no column in the Strava data that said total run time. So we would ask the LLM, "How long did this user run?" And its first response would be, "Oh, my God. It is unknowable how many minutes this user ran.

Like, nobody on Earth can figure this out." And then we would give it an instruction. We'd consume precious real estate in our prompt, and those budgets are small. We'd consume that real estate to say, "Hey, by the way, why don't you try manipulating various columns?"

We wouldn't say, "Subtract column 17 from column 4." That's very prescriptive. We would give it a general instruction, like, "Try manipulating the columns and seeing if the relationship between them might give you a clue as to what the user wants."

And then it would come back and say, "Oh, my God. Thank you for that tip. That was amazing. I figured out that I could subtract end time from start time. And indeed, you've run 55 minutes." But this is a bad use of LLM because that math is trivial.

All kinds of actions that the LLMs can do when given enough subtle instructions and prompting are not appropriately or best done in the LLM. So we've kind of taken as an operating principle inside our company, "Use the LLM last.

Try five ways. Try seven ways. Try nine ways of doing something before you resort to the LLM." There's a time and a place for it to do magic. But subtracting two columns' worth of data is not that place.

Second, search is still king. It's, uh, something that we've talked about for such a long time. You know, we were just talking about Google in the earlier presentation. Um, but everything really becomes a complex search problem as you scale.

And so just taking data and vector embedding it, just having this raw data available to your system doesn't do a good enough job. So one of the things that we've shown you here is that we take raw data, like the Doordash or the Amazon data.

We turn that into profiles. We try to abstract that, and we try to give processed and refined data to the LLM as an input so that it can figure out how to strategize properly about how to answer the question and only go down into the level of raw data when it really needs to.

The more questions you can answer closer to the edge of the system, closer to the LLM, without having to go all the way back to your raw data, the better off you'll be. And lastly, make sure that you present theright data at theright time.

Having a whole lot of data and just making that available in the system doesn't really produce the best results. We spent a lot of energy in trying to actually avoid putting too much data into various parts of the pipeline, uh, which then causes the system to be very complicated and kind of get overwhelmed.

So I know these were very high level. Our experts, like Karen, who's in the audience here, are going to be rolling out blog posts and videos trying to share with you as much as we can about the lessons we've learned along the way in building these things.

Now, I promised you that I would try to leave you with things that you could use on your own. Our mission in life is to unlock the power of user data on behalf of users. I think this is a huge mission.

I intend to spend the next 20 years on it. So you can use tools like Ario Boost without having to buy into our system. So Ario Boost is a browser extension. You can use it in one of two ways.

If you want to use it as part of using the Ario mobile app that you might download from that QR code, click the button on the left that says "Log in with Ario." Then, when you access your data from lots of different data sources, it's immediately available in the mobile app.

But let's say that you don't trust me because you don't know me, which is absolutely how you should feel. You want to use it by clicking theright-hand side button. That button is effectively the offline or developer mode. You just say, "Skip logging in."

So if you skip logging in, then we don't know who you are. There is no online account on your behalf. You can still use this tool to download your data from 15 or 20 different data sources. And I'll just give you a quick visual of what that looks like.

So we're doing this in developer mode. We're skipping the login step. There's no account created for you. And you can just auto-magically enjoy the benefits of the adversarial ETL system that we've built, which is logging into various systems on your behalf, which is using screen scraping when possible, APIs when possible, automating GDPR data downloads when possible, using all the different techniques that we've learned about to download your data.

And this is showing you locally that data. You can perform search on it. This is showing you my Sutter Medical Health logs in on your behalf and pulls those after-visit summaries and downloads them all to your device. So in this mode, all of that data is on your local device.

It has not gone anywhere. You can do whatever you want with it. You can feed it up to a different LLM. You can use Llama. You can build. You and your users can build. You can do whatever you want.

So this is part of our sort of passion for helping everybody learn and enjoy the benefits of personal user data. And that brings us more or less to the end. Uh, thank you so much for your time today.

I really do want to hear from you. That's our email, and we're growing in all areas. Thank you.