Teaching Gemini to Speak YouTube: Adapting LLMs for Video Recommendations to 2B+DAU - Devansh Tandon

There's a lot of attention in terms of how LLMs are going to transform search. Google Search is having a revolution, ChatGPT has a big chat interface, Perplexity is a product that a lot of people use. But I think recommendations is probably a bigger problem that is underhyped because it's kind of transparent to the user.

And I think the application of LLMs to recommendations is going to be a bigger consumer application than search. So in terms of my talk, I just want to introduce the problem of YouTube recommendations and then talk about how we've built large recommender models.

We're adapting Gemini for YouTube, how we built SemanticID and how we're using that, and then end with this recipe of how you might use an LLM to make a recommendation system. To start, why this is important. Who here watches YouTube every day?

It's one of the biggest consumer apps in the world. And a large majority of the watch time on YouTube is driven by the recommendation system. And we serve recommendations across Home, Watch Next, we have a big Shorts product, and even a lot of our search results are personalized in some way.

And so if you think about consumer applications of LLMs, I think in terms of consumer engagement and impact, recommendations is going to be a much bigger application than searches. And this is true of any consumer app with a billion DAU.

The way I think about the recommendation problem is you're trying to learn this function of you get a user and their context as input, and you're trying to give them a bunch of recommendations. At YouTube, we have a bunch of user information like their demographics, their age, their gender, where they're located.

We have a lot of context about them. What are the last 100 videos they watched? How deeply did they engage with them? What did they comment on? Who are they subscribed to? And we use all of that to make video recommendations.

We've tried a lot of different modeling techniques here: multi-headed rankers, embedding models, sequence-to-sequence transformers. There's a long history. And about two years ago, we started thinking, how can we rethink this recommendation system on top of Gemini, which has been making incredible progress in modeling?

How can we adapt that for YouTube?

And so we've built this system, which we call LRM, Large Recommender Model, where we adapt Gemini for recommendations. So we start with this base Gemini checkpoint, and then we are adapting it for YouTube recommendations, teaching it a lot of information about YouTube to get this kind of unified YouTube-specific checkpoint of Gemini, which we call LRM.

Then we can align it for different recommendation-related tasks, like retrieval and ranking, and basically make a small custom version of this model for all of the major recommendation surfaces. And so this is a model that we have launched in production at YouTube for a while in terms of the retrieval system, and we're experimenting a lot on the ranking side.

So I want to start with just kind of explaining how we built this YouTube and Gemini model, and then we'll talk about how we use it for retrieval. The first step of this kind of a model is you have to develop a way to tokenize videos.

So

in terms of an LLM, when you give it an input, it tokenizes that text and then is predicting the next text token. The ideal product we wanted to make was we want to give this model an input of a number of video tokens and then just get video tokens out that would be good recommendations.

We had to build this because even with a million tokens of context, when you want to reason over many videos, you have to compress that video representation in some way. And before we kind of settled on this approach, we tried a bunch of other things, like predicting search queries and retrieving videos through that, or trying to just recommend videos directly.

And those solutions were just not good enough. And so we built SemanticID, which we actually wrote a paper about last year, and it was presented at Rexus. The way that this SemanticID works is you take a video, you extract a number of features out of it, like the title, description, transcript, even the audio and video frame-level data.

You put all of that into a multi-dimensional embedding, and then you quantize it using RQVAE to give every video a token. We've written a pretty detailed paper about this if people are interested. But at a high level, the way I think about this is we're making the atomic units for a new language of YouTube videos.

Once we have these tokens, you can imagine the whole corpus of billions of videos on YouTube gets organized around these semantically meaningful tokens. And so you could imagine the first token representing topics like music, gaming, sports. Within sports, you would have different sports, and then you can get to volleyball.

And so these two volleyball videos would share some tokens in the prefix, but also then have a unique identifier. And this, I think, in itself is an interesting milestone to move away from hash-based tokenization into a semantically meaningful one.

And we use this in production at YouTube. What we then tried to do is this process of what we call continued pre-training, where we're trying to take this model and have it understand both English and this new YouTube language.

And we do this in two big steps. One is around linking text and SID, and then the second step is around having it understand sequences of watches and be able to reason across this video space. And so some of the example training tasks that we're teaching this model, you have this video, it's a tennis highlights video, which has some SemanticID, and you can prompt it and say, hey, this video has title XYZ, and the model starts to learn to output the title.

You could imagine a very similar thing where you could say, has creator or has topics, and so on. And so you're basically trying to connect text and this video token. Then what we can try to do is we have a corpus of all the YouTube engagement data, all the paths that users took through YouTube when they watch videos together.

And you can prompt the model with things like, a user has watched the following videos, A, B, C, D, and you mask some of those videos. And the model starts to learn to predict those masks. And now it's starting to understand what are videos that are watched together and make relationships between videos on the basis of user engagement.

After a bunch of pre-training tasks like this, we get this really interesting model that can reason across English and YouTube videos. And so this is an example from a user's watch history. And we find that this model can now reason across these videos.

So you could prompt it with things like, hey, video one is interesting to tennis fans because it's about Wimbledon. Video two is interesting for F1 because it's about the Spanish Grand Prix. Video three is interesting to math fans because it's about pi.

And then you prompt, video four is going to be interesting too. And the model starts to be able to understand that it's interesting to technology fans because it's about AI. And this is just based on the SemanticID definition of a video.

It doesn't really have a lot of other information to go off of. So I think this in itself is a very interesting checkpoint that is starting to reason across English and YouTube. Once we have this model, we think about how we can use this for different video recommendation tasks at YouTube.

And the first one that we focused on is generative retrieval. And so here you could just construct a prompt for every user and see what this model recommends. And so in this example, you have a user, they would be a 24-year-old woman in the US on Android.

They're watching this highlight video from the Olympics, and they have some watch history of 50 videos they've watched in the past, how they engaged with it. And you can just construct a prompt like we have on theright with this user demographic information, the context video, and have the model decode some video recommendations as SIDs.

We find that this gives really interesting, unique recommendations, especially for our hardest recommendation tasks. So in this example, when you're watching this highlight from the Olympics, the production system before LRM would give you other men's track races. Now with this new model, it's able to find this unique connection between the user demographic and their past watch history and find related women's races that we weren't able to recommend in the past.

And so we find that especially for users where we don't know as much about them, we get very interesting and unique recommendations out of this strategy.

And so we've experimented with this and launched it in a few places at YouTube. The big findings from this is that LRM is a very powerful model, but it's really expensive to serve. It learns very quickly. It's very training data efficient, and it handles our toughest Rex tasks.

But the biggest limitation was that the serving costs are too high, especially for the scale that YouTube operates at with billions of users. And so after we got our first experiments working, we spent a lot of time just reducing the TPU serving cost.

And we got 95% plus cost savings to be able to actually launch this in production.

One other strategy that we used, which I think is kind of interesting, is we tried to turn this into an offline problem where it's the same prompt and the same model. We just remove the personalized aspects of this prompt.

And we wanted to build just an offline recommendations table where if you're watching video A, what are the candidate videos that would be good to watch next? And normally, these unpersonalized recommendation models just don't hold a candle to a personalized recommender.

But because this LRM is trained from a really big checkpoint, it actually gives us some differentiated recommendations. And so in the YouTube context, we can take our corpus of billions of videos, look at the head, which represent a lot of the watch time, do offline inference, make this offline Rex table, and then we can just do a simple lookup to serve some recommendations.

And so this was kind of a complete way around our serving problems.

I want to talk a bit about the challenges for YouTube. And I think in some ways, making an LLM-based recommendation system is harder than training an LLM. One of the big differences is the vocabulary and size of the corpus.

So for Gemini, if you're training an English LLM, your vocabulary is about 100,000 words in the Oxford Dictionary, and they add about 1,000 words every year. At YouTube, if you imagine the library of YouTube, it has billions of videos.

We have 20 billion videos on YouTube with millions added every day.

And the freshness of videos is really important, much more so than LLMs. So if you think about a new word that's added to the English dictionary, word of 2023 was Riz. If your model, Gemini, doesn't know about Riz, it can still answer 99% of questions that people would have.

Maybe it misses some jokes. Maybe it misses some pop culture references. But in the world of YouTube, if Taylor Swift drops a new music video, you have to be able to recommend it within the next minutes or hours.

Otherwise, a lot of users are going to be upset. So even within this large corpus, you have to very quickly understand what are the videos that are important and start recommending them to theright user. And so what we do with this LRM recommender is we have to continuously pre-train it on the order of days and hours, which is very different than classical LLM pre-training like Gemini, which happens maybe once in three to six months.

And so in that way, it's a much harder problem. And then the last part is scale. We have great models in Gemini. Gemini Pro is incredible. But there's no way that you can serve that to billions of daily active users.

And so for YouTube, we had to focus on the smaller, more efficient models like Flash and even smaller checkpoints than that, just to be able to hit the latency and scale requirements that we have.

So I kind of want to summarize the journey that we've been on YouTube in this, what I think of as an LLM and Rexus recipe that you can maybe adapt to your own application. And there's three major steps to this.

The first is you want to find a way to tokenize your content. Just like LLMs tokenize text, you want to make some essence of your content into an atomic token. One way to do that, which we've done, is you find some rich representation, a bunch of features, build an embedding, and then find a way to tokenize or quantize it.

And the outcome of this is like you're making your own domain-specific language. The second step is you then want to adapt the LLM and basically make links between English and your domain language and find training tasks that help you reason across English and these new tokens you've built.

And so the outcome after this step in my mind is it's a bilingual LLM that can speak English and natural language, but it can also speak your domain-specific language. And then once you have this, you can do the third step of prompting it with user information where you can just construct personalized prompts with user demographic, user activity, different actions, and then train task-specific or surface-specific models.

And you have a generative recommendation system on top of an LLM. And this is like a tweet-sized summary of maybe two years of work.

Maybe the last thing that I want to talk about is kind of where I see this going and some possible future directions for LLM and Rexus. I think the stage that we're atright now is that LLMs are just augmenting recommendations.

They bring these magical recommendation experiences. They enhance the quality, but they're largely invisible to users. Like your YouTube feed just got better, but you don't really know whether a Gemini inference happened or not. This is why I think the LLM application of Rexus is very underhyped, because users don't directly know what's happening.

I think we're close to a world, and we're experimenting with this. If you have, like we talked about, a bilingual LLM across English and recommendations, users can then talk to it in natural language. And I think you're going to start to see experiences where users can steer recommendations to their own goals.

The recommender can explain why a candidate was recommended to a user, and users can start to align it towards their own goals expressed in natural language. And I think also the lines between search and recommendation start to blur in this world.

And then maybe a hint of the future is I think you're going to see recommendation and generative content start to come together in the future where we're going to be recommending a personalized version of a piece of content.

And in the future, instead of recommending content, we may even start creating it. And you can get to really interesting N of 1 content that's generated for the user. I think we're a bit away from this, but it's going to come sooner than you expect with all the advances happening in AI.

So yeah, thank you. I'll take any questions.

Thank you, Devansh. We have time for a few questions.

Hi. Great talk. One question generally, how you balance the learning of the SemanticID embeddings within the model versus keeping the general language capability not damaged by learning through, for example, a tokenized user history, which is a very second language, very different from English.

Any high-level takeaway that you can share?

That's a super interesting question.

We've struggled with this a lot. In terms of some of our early applications, we mostly cared just about recommendation quality, in which case we over-indexed on speaking the SemanticID language. And as you over-train on more and more of those examples, actually, the model forgets to speak English.

Maybe it's reasoning in some intermediate layers, which finally end up in SemanticID language. We are trying a bunch of things like with a mixture of experts. Maybe we can have a few experts that retain the text capability while other experts focus on the SemanticID capability.

And so

it's a balance. And I think we're going to shift more towards text as we try to build these interactive experiences where text input from users is going to become more important.

Thank you.

So during this process, did you learn

any good suggestions for cold-starting embeddings on these domain-specific tokens?

Yeah. So one thing is SemanticID training process is entirely unsupervised. We're not telling it's making its own quantization of the video corpus. When you sample to see what the model is doing, we find that it's learning concepts like sports versus movies and entertainment.

But we didn't actually try to teach that explicitly, which I think is very interesting. I think the second aspect is because of SemanticID, we can warm start into a semantically meaningful space. And what we find is performance for videos that were uploaded in the last day or the last week gets much better because we're better understanding this fresh and tail content.

Got it. Thank you.

Hey. Quick question. So when you said you extract frames as part of making the SemanticID, are you just running a video at, let's say, 30 to 30 FPS, making a grid of them, running Siglip or Siglip 2, and inserting that?

We're just trying to sample video frames.

We've tried a few different approaches where maybe we try to sample from key moments in the video. We actually have the engagement data. If you've seen in the YouTube player, it can highlight what are the places where people had the most engagement.

So we try to sample from there. Given the scale, we can't sample a lot of video frames. So we try to intelligently select it. But we do have video frames, and over time, I think we'll get more.

In this way of selecting it, are you able to highlight important things that are based on small objects in a video pretty well? Let's say there's a person in the distance that's of attention of this video.

Hard to say because at the end, all of this video information gets compressed into eight tokens. So it's probably learning something, but it's hard to know exactly what it picked up from that video frame. Yeah. So it's unclear.

Thank you. Yeah.

So yeah. It was a pretty good talk. I want a question regarding pre-training. So did you also feed in a user query and what they watched also as pre-training data? If yes, then did you also use SemanticID for user as well in pre-training, or is just SemanticID only for the videos?

Yeah. So in this case, we have only tokenized videos. And we focused more on sequences of watches rather than search query to what watch originated from that search query. You could imagine some parallel work where you try to tokenize users and build some kind of user token that represents the last 500 watches that they have had and so on.

We've experimented with some stuff there. I think it's less far along. But yeah, I think it's a very interesting research direction to do.

So the pre-training was done on top of existing Gemini pre-trained model,right?

Yeah. We basically take a Gemini checkpoint and then adapt it for this YouTube purpose and get this YouTube and Gemini LRM checkpoint.

OK. Yeah. So it would be cool to see SemanticID of videos to VO3.

Yeah.

Hey. I'm kind of curious. How much improvement do we see compared to the non-LLM or more traditional recommendation system? And when should we use a more traditional one, and when should we use LLM-based recommendation system?

Yeah. I can't really share metrics. I can share everything except code and metrics. And so we've given you as much conceptual steps of what we did. Maybe what I'll say is I think it's been the biggest improvement to recommendation quality we've seen in the last few years.

So I do think it's quite significant.