Netflix's Big Bet: One model to rule recommendations: Yesu Feng, Netflix

Uh, good afternoon. Thank you, Eugene, for the introduction. So today I'm going to share our big bet at Netflix on personalization, namely to use one foundation model to cover all the recommendation use cases.

At Netflix, we have diverse recommendation needs. This is an example homepage of one of the profiles on Netflix. It's a 2D layout, rows and items. Diversity comes at at least three levels. There is first level about rows. We have diverse rows.

We have young rows, for example, rows on comedies, rows on action movies. We have rows about new trending just-release titles. We also have rows about, for example, titles only available on Netflix.

So that's the first dimension. Second dimension is, of course, the items or entities. In addition to traditionally movie and TV shows, now we have games. We have live streaming, and we are going to add more. So our content space is expanding to very heterogeneous content types.

The third level is page. So we have homepage, we have search page, we have a kids' homepage, which is tailored very differently to what kids' interest. Mobile feed is a linear page, it's not a 2D layout, so on and so forth.

So page, different pages are also very diverse. What happened traditionally was that these lead to, naturally, many specialized models that got developed over the years. Some models rank videos, some rank rows, some focus on, for example, shows users have not watched yet.

Some focus on shows users are already engaging. And many of those models were built independently over the years. They may have different objectives but have a lot of overlaps as well.

So naturally, this leads to duplications. Duplications in our label engineering as well as feature engineering. Take the feature engineering as an example. We have this very commonly used factual data about user interaction history. The factual data is the same, but over the years, many features are developed derived out of the same facts data, like counts of different actions, counts of actions within various time windows, or other kinds of slice-and-dice dimensions.

Similarity between the user's history titles against the target titles. Unique, lastly, like just a sequence of unique show IDs to be used as a sequence feature into the model. So this list can go on and on. And a lot of those features, because they are developed independently into each model, they have slight variations but become very but largely very similar.

So it becomes very hard to maintain. So the challenge back then was, is this scalable? Obviously not. If we keep expanding our landscape of content type or business use cases, it's not manageable to spin up new models for each individual use cases.

There's not much leverage. There are some shared components on building the feature label, but still, by and large, each model basically spinned up independently. And that also impacts our innovation velocity in the terms that you don't reuse as much as you can.

Instead, you just spin up new models pretty much from scratch. So this was the situation about four years ago, at the beginning or middle of the pandemic. So the question we asked at that time was, can we centralize the learning of user representation in one place?

So the answer is yes. And we had this key hypothesis about foundation model based on transformer architecture. Concretely, two hypotheses here. One hypothesis is that through scaled-up semi-supervised learning, personalization can be improved. The scaling law also applies to recommendation systems as it applies to LLM.

Second is that by integrating the foundation model into all systems, we can create high leverage. We can simultaneously improve all the downstream canvas-facing models at the same time. So we'll see in the following slides how we validate those hypotheses.

I'll break up the overview into two subsections. First, about data and training, and then later, second, about application and serving. So about data and training. So starting from data, a very interesting aspect of building such foundation model, autoregressive transformer, is that there's a lot of analogy but also differences sometimes between this and LLM.

So we can transfer a lot of learnings and inspirations from LLM development. If we start from the very bottom layer, which is basically data cleaning and tokenization, people who work with LLM understand tokenization decisions have profound impact on your model quality.

So although it's the bottom layer, the decision you make there can percolate through all the downstream layers and manifest as either your model quality problem or model quality plus. So this applies to recommendation foundation model as well. Instead of there are some differences, very importantly, instead of language tokens, which is just one ID, here for if we want to translate the user interaction history or sequence, each of the tokens is an event, an interaction event from the user,right?

But that event has many facets or many fields. So it's not just one ID you can represent. There are a lot of rich information about the event. So how all of those fields can play a role in making the decision of tokenization.

I think that's what we need to consider very carefully. What is the granularity of tokenization and trade-off that versus the context window, for example? And through many iterations, we reach theright I think reach theright abstraction and interfaces that we can use to adjust our tokenization to different use cases.

For example, we can imagine we have a tokenized one version of tokenization used for pre-training. For fine-tuning against a specific application, we apply slightly different tokenization.

So moving up from the tokenization layer, then it becomes the model layers. At high level, from bottom to top, we go through the event representation, embedding layer, transformer layer, and the objective layer. So event representation, as we just briefly touched upon, many information in the event.

But at high level, you can break it down by when, where, and what. When that event happened, that's about time encoding. And where it happened, it's about the physical location, your locale, country, so on and so forth. But also about device, about the canvas, or which row, which page this action happened.

And then what, basically, is about the target entity or the title, which title you interacted with. What is the interaction, how long, and any that kind of information associated with the action. So that's where we need to decide what information we need to keep, what we should drop, so on and so forth.

Moving one layer above, the embedding feature transformation layer, one thing that needs to be pointed out is that for recommendation, we need to combine ID embedding learning with other semantic content information. If you only have ID embedding learned from scratch in the model, then you have a problem with co-star, meaning that titles the model hasn't seen during training.

It doesn't know how to deal with it at inference time. So we need to have semantic content information to be complementary to those ID embeddings. This is not a problem for LLM, but very commonly encountered co-star problem for recommendation system.

Transformer layer, I think there's no need to talk too much into this in terms of architecture choices, optimization, so on and so forth. The only thing that I want to point out is that we are using the hidden state output from this layer as our user representation, which is one of the primary goals of the foundation model is to learn a good long-term user representation.

Then we need to put this into context. Then things to consider are, for example, how stable is our user representation. Given our user profile, user interaction history keeps changing, how do we guarantee or maintain the stability of that representation?

And what kind of aggregation we should use. You can think of broadly aggregate across the time dimension in terms of sequence dimension, or aggregate across layers. You have multiple self-attention layers. How do you aggregate that? And then lastly, do we need to do explicit adaptation of the representation based on our downstream objective to fine-tune it?

So then we move to last, the very top layer, objective loss function. This is also very interesting in the sense that it's much richer than LLM. Because you can see first, we use instead of one sequence, but multiple sequences to represent the output.

Because you can have a sequence of entity IDs. That's your, like, next token prediction, softmax, or sample softmax. But then we have many other facets or fields of each event that can be also used as a target,right? So it could be for things like action type.

It could be some aspect of the entity's metadata, like entity type, young row, language, so on and so forth. And also about your action, like a prediction of the duration or the device where the action happened or the time when the next user play will happen.

So those are all legitimate targets or labels. Depends on your use case. You can use them to do the fine-tuning. Now, instead of so you can cast the problem as a multitask learning problem, multi-head or hierarchical prediction. But you can also use them just as your weights, your rewards, or your mask on the loss function.

So in terms of to adapt the model to zooming into one subcategory of user behavior you want the model to learn. OK. So that's about the model architecture that I want to talk about. So does it scale? The first question, a part of the first hypothesis we want to answer is, does a scaling law apply?

And I think the answer is yes. So this is over the roughly two, two to two and a half years, we were scaling up, and we constantly still see the gain from only on the order of a 10 million profile or a few million profile to now on the order of 1 billion model parameters.

We scale up the data accordingly. Now, we stop here because we can still keep going. But as you may realize, that recommendation system usually have much stringent latency cost requirement. So scaling up more requires us to also distill back.

Yeah. But certainly, I think this is not the end of the scaling law. Before we're wrapping up the data and training discussion, I would like to highlight some of the learnings. I think quite interesting. We borrow from LLM.

This is not an exhaustive list, but I think very interesting to me. The top three. One is multi-token prediction. You may have seen this in the DeepSeek paper, so on and so forth. So you can implementation-wise, you can use multi-head, multi-label, so on, different implementation flavor.

But the goal is really to force the model to be less myopic, more robust to serving time shift. Because you have a time gap between your training and the serving. And also force the model to target long-term user satisfaction and long-term user behavior instead of just focused on next action.

We have observed a very notable metrics improvement by doing that. The second is multi-layer representation, which I touched upon on the profile representation. So this is also translated from LLM side of techniques of layer-wise supervision, self-distillation, or multi-layer output aggregation.

The goal here is really to make a better and more stable user representation. Lastly, this is also should be no surprise, long context window handling from truncated sliding window to sparse attention to progressively training longer and longer sequences to eventually all of the parallelism strategies.

So this is about more efficient training and maximize the learning. OK. So shift gear to talk about the serving and applications. Before the foundation model FM, this is roughly the algo stack we have for personalization. Many data, many features, many models independently developed, each serving multiple or one canvases or applications, we call.

Now, with the foundation model, we consolidate largely the data and representation layer, especially the user representation as well as content representation in the personalization domain. Model layer as well, because model now, each application model now are built on top of FM, so become a singer layer instead of a very standalone, full-fledged model trained from scratch.

So how do the various models utilize the foundation model? There are three main approaches or consumption patterns. The first is foundation model can be integrated as a subgraph within the downstream model. Additionally, the content embeddings learned from the foundation model can be integrated as the embedding lookup layers.

So downstream model is a neural network. It may already have initially some of the sequence transformer tower or graph, and then using a pre-trained foundation model subgraph to directly replace that. Second is that we can push out embeddings.

This is no surprise from both content side and entity embedding as well as member embeddings. The only the main concern here, of course, is how we want to how frequently we want to refresh the member embeddings and how we make sure they are stable and push them to the centralized embedding store.

And this, of course, allow far more wider use cases than just personalization because people, analytics, data scientists can also just fetch those embeddings directly to do the things that they want.

Finally, user can extract the models and fine-tune it for specific applications. Either fine-tune or they need to do distillation to meet the online serving requirement, especially for those with a very strict latency requirement.

To wrap up, I want to show at high level the wins we accumulated over the last one year and a half by incorporating FM into various places. So the blue bar represent how many applications have FM incorporated. The green bar represent the A/B test wings.

Because in any application, we may have multiple A/B tests going on there to have wings. So we see we indeed see high leverage of FM to bring about both A/B test wings as well as infrastructure consolidation.

So I think the big bets are validated. It is a scalable solution in terms of both in terms of a scalable scaled up the model with improved quality as well as make the whole infra consolidated and scale to new applications to be much easier.

High leverage because it's a centralized learning. Innovation velocity also is faster because we allow a lot of newly launched applications to directly fine-tune the foundation model to launch the first experience.

So the current directions. One is that we want to have a universal representation for heterogeneous entities. This is, as you can guess, the semantic ID and along those lines because we want to cover that as Netflix expand into very different, very heterogeneous content types.

Second is generative retrieval for collection recommendation,right? So instead of just recommend a single video, be generative at inference time and serving time because you have to have multi-step decoding. A lot of the consideration about business rules or diversity, for example, can be naturally handled in the decoding process.

Lastly, faster adaptation through prompt tuning. So this is also borrowed from LLM. Can we just train some soft tokens so that at inference time we can directly swap in and out those soft tokens to prompt the FM to behave differently?

So that is also a very promising direction that we are getting into. Allright. That concludes my talk. Thank you for your attention and questions.

Thank you, Yesu. If you have any questions, may I invite you to come to the mic in front while we get our next speakers from Mr. Khat to get set up.

Hi. Thank you for the talk. Since you get billions of users, so except the recommendation system, maybe it can do much more,right? So what's your plot on that? Since I can just ask it to predict who's the next president in the United States.

Thank you.

So I actually don't can you explain a little bit what do you mean by beyond recommendation? Do you mean the other personalization or other things?

Yeah. Yeah. Since you get kind of billion users preference. So actually, that preference is also been leaning to what things they'll buy or who they will vote for the next president. So do you think your foundation model has that capability to expand not only recommendation, what videos they want to look, what others they like, or what's their opinions on anything else?

Thank you.

Yes. So I think we are expanding to different, I think, entity type and also capture users taste from both on and off our platform. I think that's a general trend that we're going to. Yes.

OK.

We're all done.

Great. Thank you. This was really helpful. Question on and you might not be able to share it for IP reasons, but whatever you can. Thoughts on graph models. I didn't hear a lot of that in your talks. Graphs and reinforcement learning, any utilization there?

Any benefits you saw? Any boost in performance and accuracy?

Yeah. That's a very good question. I think we have actually a dedicated team, subteam doing graph model, especially around our knowledge graph to cover the content space both on and off our platform in the whole entertainment entertainment ecosystem.

So we use actually a lot of embeddings, for example, from the graph model to co-start. That's where I see I show those semantic embeddings. That's where it comes from. In terms of reinforcement learning, yes, as well, especially where we consider sparse reward that we have on users from users action are pretty much sparse.

But we want to use them to guide how, for example, we generate the whole collection. That's where we need to consider how to use those reward to guide those process.

Can I take one more question? I'm sorry.

Can I ask a two-part question?

Sure. I will be here and so we can also follow up.

One question. Do you also use these unified representations as embedding features to downstream models? You had a slide how you use the unified model.

Yeah. So the

we so for the embeddings learning within our model, we also expose to downstream to direct consume them. We also have to train our unified embedding. We also have some upstream, like just, for example, the GNN embeddings that those are also consumed to to do that.

Ask more questions.

Last one. Is it fast?

Yes.

Allright. Hello. In your embeddings, are you just using when someone does an action or sorry. For the in these embeddings, are you just using metadata over the video to understand what they like? Or are you actually using, like, frame by frame of the video or second clips?

Not yet. We do have that from some other content group of our organization. But I think the trend will go there. So we are not yet into very granular sub, like, clips level or view level. We have those embeddings, but not quite yet to incorporate.

Yeah.

Thank you.

No problem.

Thank you, Yesu. Please a round of applause for Yesu.