Meta - the parent company of Facebook, Instagram, WhatsApp, Threads and others - operates one of the largest recommendation systems in the world.

In two recently published papers, Meta researchers reveal how generative models can be leveraged to better understand and respond to user intent.

By treating recommendations as a generation problem, new approaches can be adopted to solve it that are richer in content and more efficient than traditional approaches. This approach is important for any application that needs to retrieve documents, products, or other types of objects.

Dense search vs. generative search

The standard way to create a recommendation system is to compute, store, and retrieve Dense representations of documents. For example, in order to recommend items to users, an application must train a model that can compute user requests and embedded representations of a large number of project repositories.

In reasoning, the recommendation system tries to understand the user's intent by looking for one or more item embedded representations that are similar to the user embedded representation. As the number of projects grows, this approach requires more and more storage and computing power, because the embedded representation of each project must be stored, and each recommendation action requires comparing the user embedded representation to the entire project repository.

Generative Retrieval is a newer approach that attempts to understand and recommend user intent by simply predicting the next item in a user interaction sequence, rather than by searching a database.

It works as follows:

The key to making generative retrieval work is to compute "semantic ids" (SIDs), which contain contextual information about each item. Generative retrieval systems like TIGER work in two phases. First, an encoder model is trained to create a unique embedded value for each item based on its description and properties. These embedded values become SIDs and are stored with the project.

In the second stage, a converter model is trained to predict the next SID in the input sequence. The input SID list represents the user's interactions with past projects, and the model's prediction is the SID of the project to be recommended. Generative retrieval reduces the need to store and embed representations across individual items for searching. Therefore, as the list of items grows, its inference and storage costs remain the same. It also enhances the ability to capture deeper semantic relationships in the data and provides other benefits of generating models, such as adjusting the "temperature" to adjust the diversity of recommendations.

Advanced generative search

Although generative retrieval has lower storage and inference costs, it also has some limitations. For example, it tends to overfit items it has seen during training, which means it will have trouble working with items that are added to the catalog after model training. In recommendation systems, this is often referred to as the "cold start problem," which involves new users and new items that have no interaction history.

To address these shortcomings, Meta has developed a hybrid recommendation system called LIGER, which combines the computational and storage efficiency of generative search with the robust embed quality and ranking power of Dense search.

During training, LIGER uses similarity scores and the next marked target to improve the model's recommendation. In reasoning, LIGER selects several candidates based on the generation mechanism, complements them with some cold start items, and then ranks these based on the embedded representation of the generated candidates.

The researchers note that "the convergence of Dense and generative search methods has great potential for advancing recommendation systems," and that as the models develop, "they will become increasingly applicable to practical applications, enabling a more personalized and responsive user experience."

In another paper, the researchers introduce a novel multimodal generative retrieval method called the Multimodal Preference Discriminator (Mender), a technique that enables generative models to capture implicit preferences from users' interactions with different items. Mender is built on a generative retrieval approach based on SIDs and adds components that enrich recommendations with user preferences.

Mender uses a large language model (LLM) to translate user interactions into specific preferences. For example, if a user praises or complains about a particular item in a review, the model will summarize that as a preference for that product category.

The main recommendation model is trained to predict the next semantic ID in the input sequence based on both the user interaction sequence and user preference. This enables the recommendation model to generalize, learn context, and adapt to user preferences without explicit training on those preferences.

"Our contribution paves the way for a new generation of generative retrieval models that are able to leverage organic data to guide recommendations through text user preferences," the researchers wrote.

Impact on enterprise applications

The efficiency provided by generative retrieval systems has an important impact on enterprise applications. These advances translate into immediate practical benefits, including reduced infrastructure costs and faster reasoning. The technology maintains constant storage and inference costs regardless of directory size, which is especially valuable for growing businesses.

These benefits span industries, from e-commerce to enterprise search. Generative search is still in its early stages, and we can expect more applications and frameworks to emerge as it matures.