Aligning Recommendations with User Popularity Preferences

1. Problem Statement

Popularity Bias in recommender systems is the tendency to disproportionately surface already well-known (popular) items. While often viewed as a system-level issue (e.g., the "rich-get-richer" effect), this paper argues that the core problem is a misalignment between a user's individual preference for popular vs. niche content and the system's output.

• The Gap: Existing mitigation strategies often apply global debiasing (reducing popularity for everyone) or rely on coarse metrics that fail to capture the complexity of individual user preferences (e.g., a user might like both very popular and very niche items, but dislike mid-tier items).
• The Challenge: Measuring this misalignment is difficult because user preferences are multimodal and complex. Standard metrics often fail to capture the direction (too popular vs. too niche), magnitude, and dispersion of the bias.
• Goal: To develop a measurement framework that quantifies user-recommender alignment and a mitigation method that corrects this misalignment per-user without retraining the model.

2. Methodology

The paper introduces two main components: a new measurement framework and a mitigation method called SPREE.

A. Measurement Framework: Popularity Quantile Calibration

The authors critique existing metrics (like Popularity Lift or User Popularity Deviation) for failing to satisfy key validity criteria (ordinality, cardinality, dispersion, scale-invariance, and robustness). They propose a new framework based on Calibration Theory:

1. Concept: Treat the user's historical interaction history as an empirical distribution of popularity preferences, $p(s|u)$, and the recommender's output as a predicted distribution, $q(s|u)$.
2. Popularity Quantile Calibration: A recommender is "calibrated" if the quantiles of the recommendation distribution match the user's historical distribution.
   • Definition: For a quantile level $\tau$, the fraction of items in the user's history with popularity below the $\tau$-quantile of the recommendations should equal $\tau$.
3. Metrics:
   • Popularity Calibration Curve: A plot comparing the theoretical quantile $\tau$ against the empirical fraction $\hat{\tau}$ of user history items falling below that threshold. Deviations from the diagonal indicate bias.
   • Popularity Calibration Error (PCE): A scalar metric aggregating the squared differences between $\tau$ and $\hat{\tau}$ across quantile levels. Lower PCE indicates better alignment.

B. Mitigation Method: SPREE (Steering PopulaRity toward human prEferEnces)

SPREE is an inference-time activation steering method designed to align sequential recommenders (specifically Transformer-based models like SASRec) with individual user preferences.

1. Activation Steering: Based on the linear representation hypothesis, the method identifies a "popularity direction" vector in the model's hidden state space.
   • Vector Identification: The authors create two contrastive sets of artificial sequences: one with high-popularity items ($D^+$) and one with low-popularity items ($D^-$). They compute the mean activation difference to derive a steering vector $\mathbf{v}_{t,\ell}$ that points from popular to niche content.
2. Adaptive Steering Magnitude: Unlike previous methods that apply a uniform shift, SPREE adapts the steering strength and sign based on the specific user's bias.
   • Bias Estimation: A lightweight Lasso regression model ($\hat{f}$) predicts the user's bias $e(u)$ (the signed difference between the recommender's median popularity and the user's historical median) using the model's internal activations.
   • Steering Operation: The residual activation $\mathbf{x}_{t,\ell}$ is modified as:
     $$\mathbf{x}^*_{t,\ell} = \mathbf{x}_{t,\ell} + \lambda \cdot \hat{f}(\mathbf{x}_{t,\ell}) \cdot \mathbf{v}_{t,\ell}$$
     • If the user's recommendations are too popular ($e(u) > 0$), the vector is subtracted (steering toward niche).
     • If too niche ($e(u) < 0$), the vector is added (steering toward popular).
     • If aligned, no steering occurs.

3. Key Contributions

1. Measurement Theory Analysis: A rigorous critique of existing popularity bias metrics, establishing desiderata (ordinality, cardinality, dispersion, etc.) that previous metrics fail to meet.
2. Popularity Quantile Calibration: A novel measurement framework that treats popularity bias as a calibration error between user history and recommendation distributions, introducing the PCE metric and Calibration Curves.
3. SPREE Algorithm: A lightweight, inference-time mitigation method that uses adaptive activation steering. It is the first method to explicitly steer recommendations based on individual user bias estimates rather than global system properties.
4. Empirical Validation: Demonstrated that SPREE improves user-level alignment (lower PCE) while preserving or slightly improving recommendation accuracy (NDCG), outperforming global debiasing methods and other inference-time baselines.

4. Experimental Results

The authors evaluated SPREE on four datasets: Foursquare Tokyo, MovieLens-1M, MovieLens-20M, and RateBeer.

• Alignment vs. Performance Trade-off:
  • SPREE consistently reduced PCE@100 (improved alignment) across all datasets.
  • Unlike Inverse Popularity Ranking (IPR) or PopSteer (which globally reduce popularity and hurt alignment for users who prefer popular content), SPREE maintained high NDCG@100 (recommendation quality).
  • Personalized Popularity (PP) improved alignment but degraded NDCG significantly by over-recommending items the user had already seen.
• Calibration Analysis:
  • Baseline models often exhibited a "negative bias" (recommending items less popular than the user's history).
  • SPREE successfully shifted the calibration curve closer to the identity line (perfect calibration) without overcorrecting.
• Ablation Study:
  • Removing the adaptive bias estimator (using a fixed global steering strength) resulted in a massive increase in PCE (worse alignment) on some datasets, proving that per-user adaptation is critical.
  • The adaptive method maintained a constant global popularity level (zero-sum adjustment), whereas non-adaptive methods simply shifted the global popularity distribution.

5. Significance and Impact

• Paradigm Shift: Moves the focus of popularity bias mitigation from "system-level diversity" to "user-level alignment." It acknowledges that "fairness" or "diversity" is subjective; some users prefer niche content, while others prefer hits.
• Practicality: SPREE requires no retraining of the underlying model, making it highly deployable for large-scale production systems (like Amazon Music) where retraining is costly.
• Ecosystem Benefits: By aligning recommendations with individual tastes, the method can improve user satisfaction (retention) and provide a fairer distribution of exposure for creators, potentially reducing the "rich-get-richer" dynamic without sacrificing relevance.
• Limitations: The method relies on the existence of a linear popularity direction in the embedding space and currently targets the median of the distribution. Future work could extend this to specific distribution tails or other alignment objectives (e.g., genre diversity).