Technical Summary: Bidirectional User Modeling via Profiles (BUMP)

Problem Statement

Personalizing Large Language Models (LLMs) is a critical challenge as these models are deployed across recommendation, search, dialogue, and content generation. Traditional user representation methods, such as collaborative-filtering embeddings, do not interface naturally with instruction-tuned LLMs, which rely on prompts for conditioning. Furthermore, simply concatenating a user's entire interaction history into a prompt is infeasible at scale due to context window limits, latency, and the "lost in the middle" phenomenon where salient signals degrade in long contexts.

Existing approaches to generating natural-language user profiles typically rely on explicit rewards derived from labeled downstream tasks (e.g., click-through rates or specific classification labels). This dependency creates significant bottlenecks:

1. Scalability: Rewards are expensive and sparse, requiring annotated supervision for every target task.
2. Generalization: Profiles optimized for a specific labeled task may not transfer well to other domains or zero-shot scenarios.
3. Bias: Profiles become features engineered for specific classifiers rather than faithful descriptions of the user.

The paper posits that a robust profile generator should be trainable without any downstream task labels, relying solely on raw interaction logs.

Methodology: BUMP

The authors introduce Bidirectional User Modeling via Profiles (BUMP), a self-supervised framework that trains a profile generator using only raw user interaction data. The core intuition is that a high-quality profile must satisfy two complementary properties:

1. Predictive: The profile should predict the user's future (held-out) interactions.
2. Identifiable: The user's interactions should uniquely identify the profile among other users.

Training Framework

The system employs a Group Relative Policy Optimization (GRPO) algorithm to train a policy $\pi_\theta$ (the profile generator) using a small, frozen LLM as a judge ($J$). The training process operates over a batch of users $\mathcal{B}$:

1. Data Partitioning: For each user $u$, the interaction history is split into a visible prefix ($H^{vis}_u$) used for generation and a held-out portion ($H^{ho}_u$) used for supervision.
2. Bidirectional Reward Design: The frozen judge $J$ computes a dense reward based on two ranking directions:
   • Forward Direction (Profile $\to$ History): The generated profile $s_u$ is used as a query to rank the user's held-out items ($H^{ho}_u$) against in-batch items from other users. The reward is a multi-positive NDCG score.
   • Backward Direction (History $\to$ Profile): A held-out item $h^{(p)}_u$ is used as a query to rank the user's generated profile $s_u$ against profiles of other users in the batch. The reward is a single-positive NDCG score.
3. Regularization:
   • Position Debiasing: To mitigate LLM list-ranker position bias, the candidate items are presented in $M$ random orders, and the NDCG scores are averaged.
   • Length Penalty: A soft penalty is applied if the profile exceeds a token threshold $T$ to prevent the model from "reward hacking" by padding profiles with generic boilerplate text.
4. Optimization: The total reward $R(s_u)$ combines the forward and backward NDCG scores minus the length penalty. GRPO updates $\pi_\theta$ to maximize this reward.

BUMP+: Hard-Negative Mining

To address the issue where random in-batch negatives are too easy (topically unrelated), the authors propose BUMP+. This variant uses a frozen embedding model (BGE) to mine harder negatives:

• Forward: Selects held-out items from the user that are topically distinct from their visible history (low similarity to the history centroid).
• Backward: Selects profiles from users whose visible histories are topically similar to the target user, forcing the judge to distinguish between highly similar user profiles.

Key Contributions

1. Novel Self-Supervised Objective: The paper formulates user profile generation as a bidirectional in-batch ranking problem, deriving dense supervision from raw logs without downstream labels.
2. BUMP and BUMP+: The instantiation of this objective using GRPO and a frozen LLM judge, enhanced by hard-negative mining to sharpen the learning signal.
3. Empirical Validation: Demonstrating that a self-supervised 4B parameter model can match or outperform closed-source APIs (e.g., Gemini-3-Pro) and prior methods relying on labeled rewards.

Experimental Results

The framework was evaluated on the LaMP benchmark, covering six tasks across five domains (classification, regression, and generation).

• Performance vs. Task-Supervised Methods: BUMP+ matched or outperformed the "Direct Reward" baseline (which uses labeled downstream signals) on 9 out of 11 metric rows. This confirms that self-supervision can substitute for labeled rewards.
• Performance vs. Closed-Source APIs: BUMP+ outperformed the closed-source Gemini-3-Pro on all 11 metric rows, despite using a significantly smaller (4B) open-weight model.
• Comparison with Raw History: While raw history performed best on generation tasks (where surface-level n-gram overlap is rewarded), BUMP+ significantly outperformed raw history on classification and rating tasks, where topical preferences are more critical than verbatim text.
• Ablation Studies:
  • Removing either the forward or backward direction degraded performance, confirming both are necessary.
  • The "Forward" direction alone was more effective than "Backward" alone, suggesting the forward term drives the primary task-relevant signal while the backward term acts as a regularizer against generic profiles.
  • Hard-negative mining (BUMP+) provided significant gains over random negatives, though excessive mining introduced noise.

Significance and Claims

The paper claims that self-supervised learning can effectively replace labeled downstream rewards for training textual user profiles. By leveraging the inherent structure of user interaction logs through bidirectional ranking, BUMP generates profiles that are:

• Task-Agnostic: Optimized to be a faithful description of the user rather than a feature for a specific classifier, enabling cross-task transfer without retraining.
• Scalable: The supervision budget grows automatically with the user population rather than requiring manual annotation.
• Efficient: The resulting profiles are compact, fixed-size representations that can be precomputed and cached, avoiding the latency and memory costs of processing full interaction histories at inference time.

The authors acknowledge limitations, including potential biases inherent in LLM-as-a-judge (e.g., verbosity, self-preference) and the reliance on embedding similarity as a proxy for confusability in hard-negative mining. However, the results suggest that bidirectional self-supervision is a viable and powerful alternative to traditional supervised profile generation.