Technical Summary: Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders

Problem Statement

Generative Recommenders (GRs), which utilize Large Language Models (LLMs) and Semantic Indices (SIDs) to replace traditional item IDs, have shown significant potential in transforming recommendation paradigms. However, this study identifies that GRs remain severely susceptible to popularity bias, a long-standing issue where algorithms disproportionately favor popular "head" items at the expense of relevant but sparse "tail" items.

Current state-of-the-art GRs (e.g., LETTER, LC-Rec, ED2) exhibit extreme over-recommendation of head items, creating a "filter bubble" where trending content monopolizes visibility. Existing debiasing methods, such as item re-weighting and result re-ranking, have proven largely ineffective when applied to GRs, failing to achieve Pareto optimality between overall performance, tail performance, and fairness. The fundamental reasons for this failure in GRs remain under-explored, particularly regarding how the unique end-to-end generative framework and SID-based tokenization interact to exacerbate bias.

Methodological Diagnosis

The authors diagnose the root causes of popularity bias in GRs through two primary theoretical analyses:

1. Gradient Starvation in MLE Optimization: Under the standard Maximum Likelihood Estimation (MLE) objective, tail item tokens predominantly act as negative samples in the Softmax denominator. Due to the long-tailed distribution of training data, these tokens receive negligible positive gradient updates. Instead, they are pathologically pushed away from the user preference space, leading to "gradient starvation."
2. Bias Amplification via Undifferentiated Tokenization: Current SID generation methods (e.g., RQ-VAE, RQ-KMeans) treat head and tail items identically, creating unstructured branching points where head and tail tokens compete throughout the generation sequence. The gradient starvation of tail tokens causes head tokens to dominate at every branching point. This results in a geometric suppression of tail item probabilities, where the bias amplifies exponentially ($O(\gamma^z)$) as the generation sequence progresses.

Proposed Solution: Ghost

To address these issues, the paper proposes Ghost, a novel generative recommender system comprising two core components:

1. Skeleton-Founded Tokenization (SKT)

SKT restructures the SID space to eliminate unpredictable competition between head and tail items.

• Mechanism: Head items are first assigned SIDs to form a "skeleton" of the SID system. Tail items then inherit the prefix (the first $L_h$ tokens) of their semantically closest head item.
• Differentiation: Tail items generate only additional tokens ($L_t$) to capture their unique distinctiveness after the shared prefix.
• Effect: This unifies the branching point between head and tail items to a single, predictable step ($L_h + 1$). It prevents the geometric accumulation of bias by ensuring tail items do not compete with head tokens during the initial generation steps.

2. Asymmetric Unlikelihood Optimization (AUO)

AUO addresses the gradient starvation issue by introducing explicit negative supervision.

• Mechanism: For each target tail item, Ghost constructs an "undesired collection" ($\bar{\Omega}$) consisting of popular head items that are semantically similar but possess divergent SIDs.
• Loss Function: The model is trained with an asymmetric unlikelihood loss ($L_{AUO}$) that actively penalizes the generation of tokens in $\bar{\Omega}$.
• Effect: This creates a "rescue force" that redistributes probability mass away from dominant head tokens and toward the target tail tokens, effectively counteracting the suppressive MLE gradients.

The overall objective function combines standard MLE (NLL) and the AUO loss: $L_{All} = L_{NLL} + \alpha \cdot L_{AUO}$.

Key Contributions

1. Diagnosis: The study theoretically identifies gradient starvation in MLE and bias amplification caused by undifferentiated tokenization as the fundamental drivers of popularity bias in GRs.
2. Architecture: The development of Ghost, featuring SKT to structurally limit bias amplification and AUO to provide corrective gradient signals for tail items.
3. Empirical Validation: Extensive experiments across three Amazon datasets (Musical Instruments, Arts/Crafts, Video Games) demonstrate that Ghost substantially alleviates popularity bias while maintaining competitive overall utility.

Experimental Results

Evaluated against SOTA baselines (LETTER, LC-Rec, ED2) and debiasing methods (IFairLRS), Ghost achieved the following:

• Tail Performance: Ghost increased Tail HR and NDCG by an average of 63.91% and 70.66% compared to standard GRs, and by 28.39% and 15.04% compared to existing debiasing methods.
• Fairness: It reduced Mean Group Unfairness (MGU) by 55.76% on average compared to standard GRs.
• Pareto Optimality: Ghost achieved a comprehensive normalized score (CNS) gain of 16.81%, approaching the Pareto optimal frontier where improvements in fairness do not come at the cost of unacceptable drops in overall accuracy.
• Scalability: The framework demonstrated robust performance across LLM backbone scales ranging from 0.6B to 8B parameters, with tail item retrieval improving as model capacity increased.

Significance and Claims

The paper claims that Ghost effectively breaks the "filter bubble" in generative recommendation by addressing the root causes of popularity bias rather than applying superficial post-processing. By theoretically grounding the solution in the optimization dynamics of generative models, the study demonstrates that it is possible to promote fairer, long-tail recommendations with only slight degradation to overall utility.

The authors position this work as a step toward the responsible deployment of LLM-based recommendation technologies, addressing the algorithmic "Matthew Effect" that marginalizes niche content. They acknowledge current limitations, noting that the study is confined to the supervised fine-tuning (SFT) paradigm and static undesired collections, leaving reinforcement learning-based approaches and dynamic collection updates as directions for future research.