🤖 AI

Generative Long-term User Interest Modeling for Click-Through Rate Prediction

This paper proposes GenLI, a generative long-term user interest model for CTR prediction that replaces traditional target-centered retrieval with an interest generation module to capture diverse, interaction-aware user interests while achieving $O(1)$ behavior retrieval complexity for improved efficiency and accuracy.

Original authors: Jiangli Shao, Kaifu Zheng, Hao Fang, Huimu Ye, Zhiwei Liu, Bo Zhang, Shu Han, Xingxing Wang

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiangli Shao, Kaifu Zheng, Hao Fang, Huimu Ye, Zhiwei Liu, Bo Zhang, Shu Han, Xingxing Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to guess what a customer wants to order next. You have a massive notebook containing every single thing this customer has ever looked at, clicked, or bought over the last few years. This is the "long-term user behavior" the paper talks about.

The goal is to predict if the customer will click on a specific new item (like a new restaurant or product) to improve the "Click-Through Rate" (CTR)—basically, how often people actually choose what you show them.

Here is the problem with the old way of doing things, and how this new paper, GenLI, fixes it.

The Old Way: The "Target-Centered" Search

Previously, systems worked like a librarian who only looks for books that match the exact title of the book you are currently holding.

The Filter: If you are looking at a picture of a pizza, the system scans your entire history and only pulls out the 10 items that look most like pizza.
The Flaw: This misses the big picture. Maybe you love pizza, but you also love hiking, sci-fi movies, and buying gardening tools. By only looking at "pizza-like" things, the system ignores your other passions. It's like judging a person only by their favorite food and ignoring their personality.
The Speed Problem: To find those 10 pizza-like items, the librarian has to compare your current item against every single item in your history one by one. As your history grows to millions of items, this becomes incredibly slow, like trying to find a needle in a haystack by checking every single piece of hay individually.

The New Way: GenLI (The "Generative" Chef)

The authors propose a new system called GenLI. Instead of just looking for matches, GenLI tries to understand the customer's current mood and interests first, then finds the relevant history. It works in three creative steps:

1. The Interest Generator (IGM): Reading the Room

Instead of asking, "What looks like pizza?", GenLI looks at your recent actions (the last few clicks) and asks, "What kind of person is this user right now?"

It creates three different "mood maps" (distributions) of your interests:
- Explicit Interest: What you clearly clicked on (e.g., "I definitely want pizza").
- Implicit Interest: What the system showed you that you didn't click, but might still like (e.g., "I saw a salad, didn't click, but maybe I'm just not hungry for it yet").
- Relative Interest: The difference between the two (e.g., "I prefer pizza over salad right now").
The Analogy: Think of this as the chef tasting the air in the kitchen. Instead of just grabbing ingredients that look like the current order, the chef senses the customer is "in a spicy mood" or "feeling nostalgic." This happens without looking at the specific target item yet, so it doesn't miss any hidden interests.

2. The Behavior Retrieval (BRM): The Magic Lookup

Once GenLI has these "mood maps," it needs to find the relevant items from your massive history.

The Old Way: Compare the target item to every history item (Slow!).
The GenLI Way: It uses a simple lookup. Because the "mood maps" are pre-calculated, the system just checks a scorecard. It's like having a pre-sorted index where you can instantly jump to the "Spicy" section or the "Nostalgic" section.
The Result: It picks the top items that fit those moods. This is incredibly fast (mathematically, it's $O(1)$ , meaning it takes the same tiny amount of time whether you have 100 items or 10 million).

3. The Interest Fusion (IFM): Blending the Flavors

Finally, the system takes the items it found for each "mood" (the spicy ones, the nostalgic ones, etc.) and mixes them together with the current item you are looking at. It uses a "gating mechanism" (a smart switch) to decide how much of each interest to focus on before making the final prediction.

Why This Matters (The Results)

The paper tested this on real-world data from Meituan (a massive Chinese lifestyle and delivery app) and public datasets like Amazon and Taobao.

Accuracy: GenLI was better at predicting what users would click on than all the previous "state-of-the-art" models. It improved the prediction accuracy significantly because it didn't ignore the user's other interests.
Speed: Because it stopped doing the slow, one-by-one comparisons, it became much faster. This is crucial for online systems that need to make decisions in milliseconds.
Real-World Success: The paper states that GenLI is already deployed in Meituan's advertising system. It is currently serving hundreds of millions of users every day. In a live test (A/B testing), it increased the click-through rate by 0.776% and the revenue per thousand views by 1.567%.

Summary

In short, the old systems were like a librarian who only found books similar to the one you held. GenLI is like a librarian who first understands your current mood and diverse tastes, then instantly pulls the perfect books from the shelves without even needing to check the title of the book you're holding. It's faster, smarter, and makes more money for the platform.

Technical Summary: Generative Long-term User Interest Modeling for Click-Through Rate Prediction (GenLI)

Problem Statement
Click-Through Rate (CTR) prediction is a fundamental task in advertising and recommendation systems, heavily reliant on accurate modeling of long-term user interests derived from massive historical behavior sequences. Current state-of-the-art approaches typically employ a two-stage framework consisting of a General Search Unit (GSU) and an Exact Search Unit (ESU). The GSU retrieves a subset of relevant behaviors based on similarity to the target item, while the ESU aggregates them via attention mechanisms.

The authors identify three critical limitations in existing target-centered GSU approaches:

Incomplete Interest Representation: By focusing solely on behaviors relevant to the specific target item, these models ignore other latent user interests, leading to biased and incomplete interest features.
Computational Bottleneck: The matching-based retrieval process requires calculating pairwise similarity scores between the target item and every historical behavior. As user behavior sequences grow (often to thousands of items), this becomes a time-consuming bottleneck for online services.
Ignored Behavioral Interactions: The pairwise architecture compares the target item against individual behaviors in isolation, neglecting the interaction information and patterns existing among the user's historical behaviors.

Methodology: GenLI Framework
To address these issues, the authors propose GenLI, a generative long-term user interest modeling framework. Unlike traditional retrieval-based methods, GenLI adopts a generative approach that is target-independent. The framework consists of three core modules:

Interest Generation Module (IGM):
- Input: The IGM utilizes a short-term behavior sequence (the most recent $l$ behaviors) rather than the target item.
- Mechanism: It employs Multi-Head Attention (MHA) to process these short-term behaviors. The query vectors consist of the embedding of the latest behavior and a learnable query vector to capture interaction knowledge.
- Output: The module generates multiple discrete interest distributions representing different aspects of real-time user interests:
  - Explicit Interest ( $\mathbf{p}_E$ ): Derived from direct interactions (e.g., clicks), optimized via negative log-likelihood on clicked items.
  - Implicit Interest ( $\mathbf{p}_I$ ): Derived from exposed items (impressions without clicks), representing stable, long-term preferences.
  - Relative Interest ( $\mathbf{p}_R$ ): Defined as the difference between explicit and implicit distributions ( $\mathbf{p}_E - \mathbf{p}_I$ ), capturing the changing trend of user interests and decision-making contrast.
- Key Feature: This process is target-independent, ensuring the generated distributions encompass a broader, more diverse range of user interests.
Behavior Retrieval Module (BRM):
- Mechanism: Instead of calculating pairwise similarities, the BRM performs a simple lookup operation. For each generated interest distribution, historical behaviors are assigned scores based on their unique IDs mapped to the distribution indices (via a modular operation).
- Efficiency: This reduces the time complexity for weighting each behavior to $O(1)$ .
- Selection: The module selects the top- $k$ behaviors for each of the three interest distributions ( $\mathbf{B}_I, \mathbf{B}_E, \mathbf{B}_R$ ), resulting in a total of $K$ retrieved behaviors. This process inherently incorporates interaction information from the generation phase and filters noise.
Interest Fusion Module (IFM):
- Aggregation: The selected behaviors from each distribution are separately aggregated using Multi-Head Attention, where the target item embedding acts as the query.
- Fusion: The resulting interest embeddings are concatenated and processed through a gating mechanism (MLP with sigmoid activation) to produce the final long-term user interest feature vector ( $\mathbf{x}_l$ ).
- Prediction: This feature is concatenated with short-term interests, side information, and the target item embedding to predict CTR via a standard MLP.

Key Contributions

First Generative Approach for Long-term Interests: The authors claim to be the first to introduce generative methods specifically for modeling long-term user interests in CTR prediction tasks, moving away from target-centered retrieval.
End-to-End Framework: GenLI integrates Interest Generation, Behavior Retrieval, and Interest Fusion into a unified, end-to-end trainable framework that can be easily plugged into existing CTR models.
Diversity and Efficiency: By generating target-independent distributions, the model captures diverse and complete user interests. Simultaneously, the lookup-based retrieval significantly reduces computational complexity compared to matching-based methods.

Experimental Results
The authors evaluated GenLI on three datasets: Amazon (Book), TaoBao, and a large-scale industrial dataset from Meituan.

Offline Performance:
- On the Amazon and TaoBao datasets, GenLI achieved State-of-the-Art (SOTA) performance, outperforming strong baselines like TWIN, SIM, and ETA. Specifically, it showed absolute AUC improvements of 0.61% (Amazon) and 1.03% (TaoBao) over the best baseline.
- On the industrial dataset, GenLI improved AUC by 0.22% over TWIN while reducing inference time (4.6ms vs. 7.9ms for TWIN).
- Ablation studies confirmed that all three generated distributions (Explicit, Implicit, Relative) contribute to performance, with Explicit interest having the most direct impact.
Online Performance:
- In an A/B test on the Meituan food delivery platform (handling 30% of traffic), GenLI demonstrated a 0.776% increase in CTR and a 1.567% increase in Revenue Per Mille (RPM) compared to the production baseline.
- The model has been fully deployed in the real-world advertising system, serving hundreds of millions of users daily.

Significance and Claims
The paper claims that GenLI successfully balances accuracy and efficiency, a critical requirement for industrial CTR systems. By shifting from a target-centered retrieval paradigm to a generative, target-independent modeling approach, GenLI overcomes the limitations of incomplete interest capture and high computational latency. The authors emphasize that the ability to model diverse, latent interests without the bottleneck of pairwise matching allows for more robust personalization at scale. The successful deployment and positive online metrics validate the practical application value of generative methods in long-term user interest modeling.