Privacy-Preserving Intelligence — Steve Korshakov, Bee (acq. Amazon)

Hello everyone. I hope this talk will be shorter. Uh, I'm from Amazon, and our company was acquired about 8 months ago, and we built the AI wearable, which is on my hand, which is essentially a microphone that records everything and builds your personal agent, personal AI, and on top of that you can extract all the data that you record and plug it to your systems or agents and do whatever you want.

Just to be to get in perspective how confidential, how like private data we're capturing, a single person usually like captures about 10 million tokens per year. So this and even within like a first week of recording, people usually tell extremely sensitive stuff to their friends, to their family.

You can learn virtually everything about the person within just like one week of wearing the Bee device, which is extremely sensitive. I think we're one of the most sensitive capture devices on the market now. So and

because of this we had to encrypt everything, and our mission was to not have access to any of this data and not being able to look at it, anyone at Amazon. And it became a little bit challenging for us at Amazon because Amazon itself provides strong security and privacy guarantees, but if you're Amazon and you're using Amazon stuff, there is like much more serious security stuff you need to do.

First of all, we defined like few core principles what we needed to do for our specific agent. First of all, we believe the agent should be like working all the time, nonstop, for your good. It should be doing stuff on your behalf, and

we should not consume customer resources such as batteries and stuff. So this way we

so this leads us to one specific design of the

of the our system. Current system usually builds the on request response system where you send request to your line from your, say, iPhone, calculate something, and the backend gives you back. Unfortunately, we already see that this is not enough, that we need to

that we need to run stuff continuously, and sometimes for days. We can see we can see this like as a like glimpse into the future how cloud code works. So like just a few months ago it was like more like request response stuff, like change this, change that, and now it can work for like hours for us.

We think the same will happen to all your like personal agents anyway. So because of this we built a stateful runtime with persistent memory that we still don't have access to. It can connect to different tools. It can connect to any like third-party services if we if we program it to.

And we don't require the user device to be online. So it's fully autonomous, but at the same time it's fully controlled by the user.

So encryption system is built

on few on four like core ideas that we need to follow. First of all, the key lives and managed only on customer device. So it's user's iPhone or Android device itself. We don't have the key ourselves. We don't persist it anywhere.

So keys is

stored only on the customer phone. Everything is encrypted. We don't have any opt-out. There's no way to disable it. There's no way to bypass it. At the same time, we to protect ourselves from like internal threats, we do fully transparent and audit of all our workloads.

And we on top of that, we try to minimize the dependencies on the

on what we can trust, really.

So any security system, if you do end-to-end encryption or any kind of encryption, there is a huge problem is key management. So the first step is like I want to tell you how we manage the key. So we start with the the key, as I mentioned before, starts on the phone and at least persist it on the phone.

Then the phone connects to our backend and runs very sophisticated attestation pipeline that verifies both integrity and that the specific workload is inside of public transparency log. We use Sigstore for our transparency log, and anyone can go there and try to look and verify that this workload is genuine.

The method is too complicated to include in this talk, but we will publish details at some point. Once the once attestation was finished, we the client shares with the our main frontend backend, and backend then replicates this key with the similar nodes that runs within our confidential compute.

Because we can't leave the unencrypted data out of our perimeter, all our we run our own inference too. So this puts us a little bit more complicated task than typical AI company. We run like all kind of models, all kind of inference software.

And so

yeah. And we don't replicate code to this inference node. We like replicate only on specific ones, limiting the scope of what we can do. And on top of that, we introduced everywhere where we have the keys in the memory

the forced expiration of seven days. We picked the seven days because we think it's like how much realistically the time horizon for the like something useful can be done for the user. 24 hours will be too low because you can like not open your phone for like 24 hours.

Something will be missed. And like so we picked like about seven days.

Then we need you know, we need to ship some something to the production. And then the biggest question, like how we can, well, not ship something that will compromise anything. Our goal was to build a system that no one outside of Amazon will be able to ship anything unnoticed.

Obviously, the software has bugs, have problems, but we shouldn't be, well, being able to ship anything. So we solved this by two two-tier system, essentially. So there is a dedicated team inside of organization, inside of Amazon, and maybe maybe not even one, I would say, that manages the privacy part of this, the transparency log.

Our team, when we ship the software, we can't influence them. We can't control them. And we hardcode their their signing keys inside of our client apps and our backends. So we can't really

we as the team, we can't do this. And like it's very, very hard and very high level.

Employees need to sign off to any kind of change. So it's like at a big company like Amazon, it's virtually impossible, really. And we do this in two parts because this process is too slow. So we split it in two parts.

So like the first one is to build the base image, which we put some kind of base software for that is needed for our own team, like the tools that measure the build, measure the manifest, measure workloads, and data that we need to put to the node.

And

and then when we want to deploy, we do the process the same similar time. We got the base image, and then we deploy the two transparency log specific manifest that any and this setup helps us to be able to security audit companies and inside, outside, to anyone to, well, we're not doing this public, but like we to like very high-profile audit companies, we work with them all the time.

We can provide any image, any data that was deployed ever. So we can like trace any possible weak spots if we like deploy something wrong, which we do not. Then after self-verification of VM, it issues a certificate that embeds all encryption, all transparency proofs, attestation documents into certificate itself.

We are using private CA because you can't do this in public certificates because it will populate the public transparency log. So we had to use the private one. We probably will introduce the extra proxy that will do a normal TLS with attestation with like lighter mode, but we don't have this yet.

Yeah, that's essentially what we built. Thank you. Any questions?

Any questions?

I think you just shot it.

I'll just shot it.

Yes.

So it's interesting what the process was like going. How in terms of like values and approach to security going to going inside Amazon, were there things that you had to change that had changed prior to like going in there?

Just as far as like processes and sort of values around security, because I know they've got very hard-coded, entrenched ways to do things.

Can you repeat?

Yeah, sorry. There's a lot of words. What was it like from a process standpoint? What had were there things that had to change when you went from being like your own small startup into existing within Amazon?

Oh, what changed with like when we joined Amazon?

Yeah.

Well, the big change is that we before like you run on Amazon, and Amazon gives you like guarantees as a customer that they can see your data. But once you're inside, this changes a lot because you're Amazon. Like so that's why you need like to provide more protection on top of this.

So we need to protect from our internal threats too. So that was a big change. So before that, it was just kind of easier, I would say, to configure everything. I'm not sure I can tell much, honestly.

Do you have anything like re-provisioning your stack from like your because I'm assuming you built on AWS side.

Yes. It's just normal EC2 instances. Yeah.

Yeah. Did you have to rebuild it?

Yeah. Yeah. We almost we're not using like yeah, almost everything we built from scratch. Well, we tried to use like the existing stuff, like but it's more like common, like popular software. And we built we tried to minimize amount of code that we produce.

So it's I calculated before this talk, it's just like about 20k lines on memory-safe language. So it was very small scope that we were able to audit and verify that all this kind of stuff. Most of this code is just verifying attestation, really.

And everything else can be like reused and like very like, you know, it's very trustworthy, I would say, software. So we didn't try to don't invent

yeah, we don't try to invent. Like when I was at Telegram, like we reintroduced like built our own crypto. And it was like questionable way of doing stuff. So I try not to do the same at Amazon, obviously.

Yeah, that's what we do.

Any other questions for Steve? Up in the back.

Yeah. I have a question. So I guess for the tail end, the die-hard security enthusiasts, just thinking about the AI side of things, you're the first person I've heard really talk about things like encryption layer. I don't have a lot of experience with that.

So we know AI agents when they go rogue or have some kind of mistake that delete databases or even the RM flag or RM forward slash lose everything. What do we do? What do agents basically do? Encrypt our hard drive or data?

I've not heard that happening, but I don't see that being very far off. And there could be an AI-based ransomware attack on an individual. And Bee, encryption?

Well, I just prefer them not to put to the computers, to personal one. So we did several experiments how to tame them, not to do bad things. And honestly, I think nothing works except like sandboxing and just not giving them a way to hurt themselves.

It's like, you know, our brains, they can't stop the heart at wheel,right? So otherwise, you know, they will be we have much more problems. So I think the same. We shouldn't give them a way to do any harm.

That's the only way, honestly. And yeah, and put something between if they want to change something. Unfortunately, I think that's the only way. Yeah.

Are you taking action on behalf of them? If it happens today?

I'm surprised they're not representing on this,right? Like they're screaming so much about security, but they didn't come to this one.

I'm not I tried OpenClaw. It's like it was once they started to try to tighten this down, it became much less useful.

So I think their approach is not really that good. So I would love to have a wild agent. That's that's our goal too. But we try to just deploy the sandbox for specific agent, and it will just they just can't do much of the stuff.

Everything else fails, unfortunately.

Cool. Allright. Well, thank you so much, Steve, for the presentation. That was amazing.