Modeling Stage-wise Evolution of User Interests for News Recommendation

This paper proposes a unified framework for news recommendation that addresses the time-sensitive nature of user interests by combining global collaborative signals for long-term preferences with a stage-wise temporal subgraph approach, enhanced by LSTM and self-attention mechanisms, to effectively model both stable habits and rapidly evolving short-term dynamics.

The Gist

Imagine you are a personal news curator for a friend. Your job is to hand them a stack of newspapers every morning that they will actually want to read.

The problem? Your friend's interests are a bit like a chameleon.

• Long-term: They always love sports and cooking (their "stable" habits).
• Short-term: But this week, they are obsessed with a sudden viral meme about cats, and next week, they might be frantic about a breaking political scandal.

Most old recommendation systems are like curators who only look at a static photo of your friend from five years ago. They know your friend likes sports, so they keep shoving football articles at them, even if your friend is currently bored with sports and only cares about the cat meme. They miss the "now."

Other systems are like curators who only look at what your friend clicked five minutes ago. If they clicked one cat video, the system assumes they only want cat videos forever, ignoring that they still love their daily sports fix.

This paper proposes a super-curator that does both at the same time. Here is how it works, broken down into simple concepts:

1. The Two-Part Brain

The authors built a system with two distinct "brains" working together:

Brain A: The "Long-Term Memory" (Global Preference)

Think of this as a library catalog built over years. It looks at your friend's entire history of clicks.

• What it does: It learns that your friend is a "Sports Fan" and a "Foodie." These are deep, stable habits that don't change overnight.
• The Analogy: It's like knowing your friend's favorite color is blue. No matter what they are doing today, you know they generally like blue things. This part uses a Graph Neural Network (GCN), which is like a map connecting all the dots between users and news to find these deep patterns.

Brain B: The "Short-Term Focus" (Local Preference)

This is the chameleon brain. It doesn't look at the whole year; it looks at stages or chunks of time.

• The Analogy: Imagine breaking your friend's life into "seasons" or "weeks."
  • Week 1: They were watching the World Cup.
  • Week 2: They were obsessed with a new diet trend.
  • Week 3: They are following a local election.
• How it works: The system slices the timeline into these "stages." Inside each stage, it uses two tools:
  1. The "LSTM" (The Step-by-Step Tracker): This is like a relay race runner. It passes the baton from one day to the next. If your friend clicked a cat video on Monday, the runner remembers that on Tuesday, they might click another cat video. It tracks the progression of interest.
  2. The "Self-Attention" (The Spotlight): This is like a spotlight operator in a theater. It looks at the whole "season" of clicks and asks, "Which of these clicks was the most important?" It realizes that even though your friend clicked 100 things last month, the one about the election was the most significant, so it shines a light on that.

2. Putting It All Together

The magic happens when these two brains talk to each other.

• The Setup: The "Long-Term Memory" (Brain A) gives the "Short-Term Focus" (Brain B) a head start. It says, "Hey, remember, your friend is a sports fan. Start looking for sports news, but keep an eye out for what's new."
• The Evolution: As time moves forward, the system checks: "Is the friend still interested in sports? Or did they switch to cats?"
• The Smoothness: The system also has a rule: "Don't flip-flop too wildly." If your friend liked sports yesterday, they probably still like them today, even if they also like cats. The system ensures the transition is smooth, not a sudden jump from "Sports" to "Zero Sports."

3. Why This Matters (The Results)

The authors tested this on real-world data (millions of news clicks).

• The Old Way: Either gave outdated news (ignoring the cat meme) or gave random, irrelevant news (ignoring the sports habit).
• The New Way: It gave the friend the latest cat meme (because it's trending right now) but also kept the daily sports score (because that's their stable habit).

In a nutshell:
This paper teaches computers to understand that human interests are fluid. We have a "core self" (long-term habits) and a "current mood" (short-term trends). By building a system that respects both the history and the moment, it can recommend news that feels perfectly timely and personal, rather than just a generic list of "what's popular."

It's the difference between a robot that reads a resume once and never updates it, and a human friend who knows your history but is also excited to hear about your new hobby.

Technical Summary

Here is a detailed technical summary of the paper "Modeling Stage-wise Evolution of User Interests for News Recommendation."

1. Problem Statement

Personalized news recommendation faces a unique challenge due to the time-sensitive nature of news content. User interests are driven by a complex interplay of:

• Long-term preferences: Stable reading habits and high-order collaborative patterns (e.g., a user consistently interested in finance).
• Short-term dynamics: Rapidly shifting interests triggered by emerging events, trending topics, or seasonal contexts (e.g., a sudden spike in health news during a pandemic).

Limitations of Existing Approaches:

• Static Graph Models: Methods relying on a single global interaction graph (e.g., standard GCNs) fail to capture the temporal evolution of interests, often leading to outdated recommendations.
• Sequential Models: Models focusing solely on recent sequences often miss stable, long-term collaborative signals, resulting in representations biased toward temporary trends.
• The Gap: There is a lack of unified frameworks that can simultaneously preserve stable global preferences while adapting to rapid, stage-wise local changes.

2. Methodology

The authors propose a unified framework that models user interests from global and local temporal perspectives. The architecture consists of three main components:

A. Global Preference Modeling (GPM)

• Goal: Capture stable, long-term collaborative signals.
• Mechanism: Constructs a global user-news interaction graph ($G$) containing all historical interactions.
• Algorithm: Utilizes LightGCN to propagate embeddings across the graph. It aggregates layer-wise representations to generate global embeddings ($e^g_u, e^g_i$) that encode high-order connectivity and stable thematic interests.
• Role: These global embeddings serve as high-quality initializations for the local temporal modules, ensuring that sparse temporal subgraphs are grounded in rich prior knowledge.

B. Local Preference Modeling (LPM)

• Goal: Capture short-term dynamics and stage-wise evolution.
• Mechanism: Partitions the user's chronological click history into discrete temporal subgraphs ($G_1, G_2, ..., G_T$) based on time intervals (e.g., weekly or monthly windows).
• Two Complementary Branches:
  1. Short-term Evolution Branch (LSTM):
     • Learns representations on each temporal subgraph conditioned on global embeddings.
     • Uses an LSTM to model sequential dependencies between adjacent time stages ($t$ and $t-1$).
     • Captures progressive, recency-sensitive preference shifts.
  2. Long-range Aggregation Branch (Self-Attention):
     • Constructs a cumulative representation of all historical items up to the current stage.
     • Uses Self-Attention with positional encodings to weigh historical signals.
     • Captures long-range dependencies and highlights influential past interactions that shape current interests.

C. Prediction and Optimization

• Fusion: The final user representation is a fusion of the short-term evolution vector, the long-range aggregated vector, and the global preference vector.
• Loss Function: The model is optimized using a composite objective:
  1. Prediction Loss: Binary cross-entropy for click prediction within each temporal segment.
  2. Consistency Regularization: A contrastive loss ensuring temporal embeddings remain aligned with the global preference embedding (preventing semantic drift).
  3. Smoothness Regularization: A constraint enforcing smooth transitions between adjacent temporal embeddings to model gradual interest evolution.

3. Key Contributions

1. Unified Framework: Proposes a novel architecture that jointly learns global collaborative patterns and local stage-specific dynamics, addressing the trade-off between stability and adaptability.
2. Stage-wise Temporal Modeling: Introduces a mechanism to partition user history into temporal subgraphs, utilizing both LSTM (for progressive evolution) and Self-Attention (for long-range context) to model interest shifts at different granularities.
3. Global-Local Initialization: Demonstrates the effectiveness of using global GCN embeddings to initialize sparse temporal subgraphs, significantly improving representation learning in data-sparse periods.
4. Robust Regularization: Incorporates consistency and smoothness constraints to ensure the model balances short-term sensitivity with long-term coherence.

4. Experimental Results

The model was evaluated on two large-scale real-world datasets: Adressa (3 months of data) and MIND (6 weeks of data).

• Overall Performance (RQ1): The proposed model significantly outperformed state-of-the-art baselines (including NRMS, NAML, LightGCN, TCCM, and CROWN) across all metrics (AUC, MRR, nDCG@5, nDCG@10).
  • Example: On Adressa, the model achieved an AUC of 0.7993, compared to the second-best (CROWN) at 0.6553.
• Ablation Study (RQ2): Removing any component (Global Modeling, Short-term Evolution, or Long-range Aggregation) led to performance degradation.
  • Removing the Local module (w/o LPM) caused the largest drop, confirming the necessity of temporal modeling.
  • Removing the Global module (w/o GPM) showed that global initialization is critical for handling sparse temporal data.
• Hyperparameter Sensitivity (RQ3): A 2-week temporal window size was found to be optimal, balancing interaction density with the ability to capture fine-grained shifts.
• Model Capability (RQ4): The model demonstrated superior ability to recommend new, time-sensitive articles (New%) while maintaining relevance for historical content. It effectively prioritized emerging news for high-interaction users and balanced novelty/relevance for low-interaction users.

5. Significance

This work addresses a critical gap in news recommendation systems by moving beyond static or purely sequential models.

• Practical Impact: It enables news platforms to deliver fresher, more relevant content that adapts to real-world events without losing sight of a user's core interests.
• Theoretical Contribution: It establishes a new paradigm for modeling "stage-wise evolution," proving that decomposing user history into temporal subgraphs and modeling them with both recurrent and attention mechanisms yields superior results compared to monolithic graph or sequence models.
• Reproducibility: The authors have released the code and parameters, facilitating further research in dynamic recommendation systems.