Technical Summary: Benchmarking and Improving LLM Robustness for Personalized Generation

Problem Definition

The paper addresses a critical gap in the evaluation of Large Language Models (LLMs) regarding personalization. While existing research and benchmarks (e.g., LaMP, PrefEval) focus primarily on whether a model's response aligns with user preferences, they largely overlook factual correctness. The authors argue that a response can be perfectly personalized yet factually inaccurate, leading to unreliable AI systems.

The core problem is defined as robustness in personalization: the ability of an LLM to maintain factual accuracy while incorporating relevant user preferences and ignoring irrelevant ones. The authors formalize this by defining a model as robust only if it satisfies two criteria simultaneously:

1. Relevance Handling: It maintains factual accuracy when conditioning on relevant user preferences (e.g., "provide a concise answer").
2. Irrelevance Filtering: It ignores semantically unrelated user preferences (e.g., "I am a vegan" when answering a math question) without degrading performance.

The paper posits the research question: "When provided with user preferences, does the model compromise factuality in its response to meet personalization?"

Methodology

1. The PERG Framework and Dataset

To systematically evaluate this trade-off, the authors introduce PERG (Personalized Evaluation of Robustness in Generation), a scalable evaluation pipeline and dataset.

• Source Data: The dataset is constructed from three established factual benchmarks: MMLU (Massive Multitask Language Understanding), TruthfulQA, and CommonsenseQA. These provide questions with objective ground-truth answers.
• Preference Construction:
  • Relevant Preferences: Generated using GPT-4o-mini based on in-context examples, ensuring they pertain to the query (e.g., requesting "concise" or "detailed" explanations) without altering the ground truth.
  • Irrelevant Preferences: Extracted from the PrefEval dataset (Zhao et al., 2025), covering domains like lifestyle, travel, and shopping that are semantically unrelated to factual QA tasks.
• Dataset Scale: The final PERGData contains 7,200 examples, each comprising a query, a ground-truth answer, and associated preferences.

2. Evaluation Metrics

The authors propose four complementary metrics to quantify robustness, where lower values indicate better performance:

• Breakage Rate: Measures how often personalization causes a model to fail on inputs it previously answered correctly without preferences ($1 - \text{Accuracy}_{\text{pref}}$ given $\text{Accuracy}_{\text{no-pref}} = 1$).
• Alignment Failure: Measures how often a model fails to follow user preferences despite answering correctly.
• Robustness Error: The union of breakage and alignment failures, representing the total rate of failure in either factuality or alignment.
• Performance Variation: Measures the divergence in correctness between personalized and non-personalized settings (similar to Jaccard distance).

3. Experimental Setup

• Models: 14 models from 5 families were evaluated, including open-weight models (Mistral-7B, LLaMA-3-8B/70B, Gemma-2-9B/27B, Qwen3, Janus-7B) and commercial APIs (GPT-4.1, GPT-4o-mini, DeepSeek-R1).
• Prompting Methods: Experiments included Zero-Shot, Chain-of-Thought (CoT), In-Context Learning (ICL), and Self-Critic prompting.
• Evaluation: A fine-tuned Mistral-7B model was used as a "Choice Selector" to extract answers from multiple-choice responses, and GPT-4o-mini served as a judge for preference alignment (validated against human annotators with a Cohen's Kappa of 0.85).

4. Proposed Solution: Pref-Aligner

To mitigate robustness failures, the authors propose Pref-Aligner, a two-stage agentic framework that decouples generation from personalization:

• Stage 1 (Generator): An agent generates a response to the user query without considering preferences, ensuring the base content is factually accurate.
• Stage 2 (Aligner): A second agent takes the unconditioned response and the user preferences. It performs lightweight edits only if the preference is relevant (e.g., shortening a response). If the preference is irrelevant, it returns the original response unchanged.
• Rationale: This approach leverages the finding that the semantic distance between an unaligned response and an aligned response is smaller than the distance between a raw query and an aligned response, reducing the risk of hallucination.

Key Results

1. Current Models Lack Robustness

• Breakage: Even the strongest models fail to maintain correctness in personalized settings. GPT-4.1 and LLaMA-3-70B exhibit a breakage rate of 5%, meaning they fail on questions they would have answered correctly without preferences. Smaller models like Mistral-7B and Janus-7B show breakage rates as high as 26%.
• Alignment vs. Factuality: There is a trade-off; models often prioritize alignment over factuality. For instance, Janus-7B has a high alignment failure (16%) but also a high breakage rate.
• Scaling: Robustness improves with model scale. LLaMA-3-70B shows significantly lower error rates than LLaMA-3-8B.
• Prompting Limitations: Standard prompting techniques (CoT, ICL, Self-Critic) yield mixed results and do not consistently improve overall robustness. In some cases, CoT improves breakage rates but worsens alignment failure.

2. Impact of Preference Types

• Irrelevant Preferences: The presence of irrelevant preferences significantly amplifies robustness errors, particularly alignment failures, as models struggle to distinguish between relevant and irrelevant signals.
• Preference Categories: Preferences requiring "Thinking" (step-by-step reasoning) or "Context" often lead to higher breakage rates, particularly in TruthfulQA, suggesting that over-conditioning on reasoning steps can cause models to "overthink" and lose factual accuracy. Conversely, "Clarity" preferences (conciseness) generally result in lower breakage rates.

3. Effectiveness of Pref-Aligner

• Performance Gains: Pref-Aligner consistently reduces robustness error across all tested models.
• Quantitative Improvement: The framework achieves an average 25% improvement in robustness across models.
  • For LLaMA-3-70B, the robustness error dropped from 9.0% (Standard Prompting) to 6.5%.
  • For Gemma-2-9B, the error dropped from 12.6% to 6.8% (a 46% relative reduction).
  • The breakage rate for LLaMA-3-70B in relevant preference settings dropped from 5.6% to 1.1%.
• Generalization: The framework was also tested on free-form generation (MATH dataset), where it reduced breakage by over 60% and alignment failures by over 70%, confirming its transferability beyond multiple-choice questions.

Significance and Claims

The paper claims to be the first to explicitly conceptualize and formally define robustness in the context of LLM personalization, moving beyond simple alignment metrics.

• Critical Gap Identification: The work highlights that current evaluation practices risk overestimating model reliability by ignoring the trade-off between personalization and factuality.
• Practical Utility: The authors provide PERGData and Pref-Aligner as open-source tools to support more reliable, user-aligned LLM deployments.
• Model Selection Insights: The results offer actionable insights for practitioners, suggesting that larger models are more robust but that even top-tier models require architectural or pipeline interventions (like Pref-Aligner) to handle personalization safely.
• Future Directions: The authors modestly note that while their framework is scalable, it currently focuses on explicit user preferences and multiple-choice settings. They suggest future work should explore implicit preference extraction, broader personalization axes (values, demographics), and training strategies that explicitly optimize for robustness.

The paper concludes that while personalization is valuable, current LLMs are not fully robust, and a decoupled approach like Pref-Aligner is a necessary step toward trustworthy, user-adaptive AI.