Technical Summary: PREFER - Personalized Review Summarization with Online Preference Learning

Problem Statement

Product reviews on e-commerce platforms significantly influence purchasing decisions, yet the sheer volume of text often overwhelms users, obscuring information relevant to their specific needs. Current summarization systems typically generate a single, static, generic summary for each product. This approach fails to account for two critical realities: (i) different users prioritize different product attributes (e.g., durability vs. price vs. ease of use), and (ii) user preferences are not stationary; they can evolve over time and vary across products. Existing systems do not adapt to these dynamic, latent preferences, nor do they learn from user interactions to refine future summaries. The core challenge is to formulate personalized review summarization as an online feedback-adaptive decision problem where the system must generate useful summaries while simultaneously learning a user's latent preference vector from lightweight scalar feedback.

Methodology: The PREFER Framework

The authors propose PREFER, a modular framework that operationalizes personalized review summarization through a feedback-adaptive loop. The system balances the exploration-exploitation trade-off between learning user preferences and generating high-quality summaries. The framework consists of four main components:

1. Offline Latent Aspect Discovery

Instead of relying on pre-labeled keywords or tags, PREFER discovers a latent aspect space $\mathcal{A} = \{a_1, \dots, a_K\}$ directly from review text using unsupervised learning.

• Embedding: Review sentences are mapped to a continuous semantic space using a pre-trained embedding model (e.g., sentence-transformers).
• Clustering: Principal Component Analysis (PCA) reduces dimensionality, followed by K-means clustering to identify centroids representing latent aspects.
• Soft Assignment: To handle sentences discussing multiple themes, a soft aspect assignment is used via a distance-based softmax over cluster centroids, yielding a distribution $\phi_i \in \Delta^{K-1}$ for each sentence $i$.

2. Personalized Evidence Selection

Given a user's current preference estimate $\hat{w}_{u,t}$ (a distribution over aspects), the system selects a subset of review sentences $S_t$ that are both relevant to the user and non-redundant.

• Relevance Scoring: A relevance score is computed as the dot product between the estimated preference vector and the sentence's aspect distribution: $Rel_{i,t} = \hat{w}_{u,t}^\top \phi_i$.
• Redundancy Penalty: Semantic similarity between sentences is measured using cosine similarity of their embeddings.
• Selection Strategy: The system optimizes a trade-off between relevance and redundancy using a Maximal Marginal Relevance (MMR) objective.
  • Deterministic MMR: Greedily selects the sentence with the highest marginal gain.
  • Stochastic Gumbel Extraction: To encourage exploration (especially early in learning), the system adds Gumbel noise to the scores, allowing for the selection of alternative sentence sets.

3. Abstract Summary Generation

The selected sentences are rewritten into a coherent, third-person summary through a lightweight hierarchical pipeline:

• Binning: Sentences are grouped into "High," "Mid," and "Low" support bins based on the number of distinct reviewers contributing to each aspect.
• Compression & Stitching: Each bin is compressed into an intermediate text block, which are then stitched together into a draft.
• Polishing: A final rewriting pass produces a polished, user-facing summary, ensuring third-person generalization and removing direct quote styles.

4. Online Preference Learning

After the user views the summary, they provide scalar feedback $f_t \in [0, 1]$. The system updates its estimate of the user's latent preference vector $\hat{w}_{u,t}$ using Online Mirror Descent (OMD) with an entropic distance-generating function.

• Surrogate Loss: The feedback is centered against a baseline (e.g., exponential moving average) to determine if the summary performed better or worse than expected.
• Update Rule: The preference vector is updated via an exponentiated-gradient rule. Aspects present in positively received summaries receive increased weight, while those in negatively received summaries are down-weighted. This update preserves the simplex constraint and ensures the estimate remains strictly positive.

Key Contributions

1. Framework for Adaptive Summarization: PREFER is the first framework to combine latent aspect discovery, structured evidence selection, and abstractive generation within a single online learning loop for review summarization.
2. Handling Latent and Drifting Preferences: The system infers user interests from interaction feedback rather than assuming fixed profiles, explicitly modeling the dynamics of preference drift.
3. Theoretical Guarantees: The authors provide a dynamic regret bound for the entropic OMD update. They prove that the system achieves no-regret performance (sublinear regret) even when user preferences change over time, recovering the standard stationary no-regret result as a special case.
4. Structured Action Space: Unlike standard bandit problems that select atomic items, PREFER operates over a structured space involving both combinatorial sentence selection and compositional summary generation.

Experimental Results

The framework was evaluated on the Amazon Reviews'23 dataset (specifically the ALL_BEAUTY category) through controlled simulations where ground-truth preferences were hidden and feedback was simulated.

• Cross-User Heterogeneity: Qualitative analysis demonstrated that conditioning on different preference vectors yields meaningfully different summaries for the same product. For instance, a user focused on "visual presentation" received summaries highlighting color variety, while a user focused on "exfoliation" received summaries emphasizing skin texture and efficacy.
• Convergence: In stationary preference settings, the empirical average regret decreased toward zero, aligning with the theoretical no-regret bounds. The online variants significantly outperformed static baselines in aligning selected evidence with target user interests.
• Adaptation to Drift: When user preferences were simulated to drift from one aspect to another, the online system successfully tracked the change. The stochastic Gumbel extraction method showed stronger recovery and adaptability to the new preference target compared to the deterministic MMR method, which exhibited slower recovery due to "lock-in" effects.

Significance and Claims

The paper claims that review summarization must move beyond producing a single generic summary for all users. By treating summarization as an interactive, feedback-adaptive process, PREFER demonstrates that systems can learn to align with specific user interests and adapt to preference evolution without requiring explicit user profiling or labeled data.

The authors maintain a modest stance, noting that their feedback is synthetically generated and the discovered aspects are latent rather than human-labeled. They acknowledge limitations regarding potential hallucinations in the LLM rewriting module and the lack of real-world user interaction data. The work suggests that future research should focus on validating these mechanisms with human feedback, incorporating stronger factuality constraints, and exploring richer feedback signals (e.g., clicks, dwell time). The primary contribution is the proof-of-concept that online preference learning can effectively personalize review summarization while maintaining theoretical guarantees on learning efficiency.