💬 NLP

Learning User Interests via Reasoning and Distillation for Cross-Domain News Recommendation

This paper proposes a reinforcement learning framework that leverages large language models to generate interest-driven search queries from cross-domain signals, employing GRPO-based policy optimization and on-policy distillation to achieve scalable, high-performance cross-domain news recommendation.

Original authors: Mengdan Zhu, Yufan Zhao, Tao Di, Yulan Yan, Liang Zhao

Published 2026-02-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mengdan Zhu, Yufan Zhao, Tao Di, Yulan Yan, Liang Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you walk into a massive, chaotic library with millions of books. You've never been there before, and you don't know the librarian. You want to find a book you'll love, but you can't ask for a specific title because you don't know what exists yet.

This is exactly the problem news recommendation systems face every day. They have to guess what you want to read based on your messy, scattered history: what you clicked on, what you searched for, and what you skimmed on other websites.

This paper describes a new, smarter way to solve this puzzle using Artificial Intelligence (AI). Here is the story of how they did it, broken down into simple concepts.

1. The Problem: The "Noisy" User

Think of a user's digital history like a jumbled box of clues.

Some clues are gold: "I clicked on a story about space exploration."
Some clues are junk: "I accidentally clicked a link while looking for my phone number," or "I searched for 'weather' just to check the time."

Old systems tried to guess your interests by just counting how many times you clicked similar things. But they often got confused by the "junk" clues and missed the deeper, hidden interests (like your secret love for 19th-century poetry) because they were too focused on surface-level patterns.

2. The Solution: The "Super-Sherlock" Teacher

The authors built a Large Language Model (LLM)—let's call him the "Super-Sherlock."

Instead of just counting clicks, this Super-Sherlock looks at your messy box of clues and uses reasoning to figure out who you really are.

The Task: The AI's job is to write a list of search queries that you would type if you were trying to find your perfect news article.
The Magic: If you clicked on a few tech articles and a history blog, the Super-Sherlock doesn't just say "Tech." It might reason, "Ah, this person is interested in the history of artificial intelligence," and generate a specific query like "History of AI development."

3. The Training: The "Game Show" (Reinforcement Learning)

How do you teach a Super-Sherlock to be good at this? You can't just give it a textbook answer because there isn't one. Instead, they used a method called Reinforcement Learning (specifically GRPO).

Imagine a game show where the AI generates 5 different lists of search queries.

The Judges: Five different "judges" (rewards) score each list:
1. The Librarian: Did this query actually find real, relevant articles?
2. The Broadcaster: Does this list cover all the different topics the user likes?
3. The Specifics: Are the queries too vague (like "News") or specific enough (like "SpaceX rocket launch details")?
4. The Variety: Did the AI just repeat the same thing five times, or is there variety?
5. The Rule-Follower: Did the AI follow the formatting rules?

The AI plays the game thousands of times, getting points for good lists and losing points for bad ones. Over time, it learns the "perfect strategy" for guessing user interests.

4. The Problem: The "Heavy" Teacher

Here's the catch: The Super-Sherlock is huge. It's like a genius professor who can solve the puzzle perfectly, but it takes him 10 seconds to think and costs a fortune to keep in the room. In a real news app, you need an answer in milliseconds, and you have millions of users. You can't afford to hire a professor for every single user.

5. The Fix: The "Apprentice" (Distillation)

This is where the paper's second big idea comes in: Distillation.

The authors took the "heavy" Super-Sherlock (the Teacher) and taught a tiny, fast Apprentice (the Student).

They didn't just give the Apprentice the final answers. They let the Apprentice watch the Teacher think through the problems.
The Apprentice learns to mimic the Teacher's reasoning process but does it much faster and with a much smaller brain.

The Result: The Apprentice is 60 times smaller than the Teacher but keeps 90% of the Teacher's genius. Now, the news app can use this fast, cheap Apprentice to guess what you want to read in a split second.

6. The Proof: Does it Work?

They tested this in the real world on a massive news platform.

Offline Tests: The new system found better articles than all the old methods.
The "Cold Start" Win: It was especially good at helping new users (people with no history). Because the system could reason about other things the user did (like searching for a specific topic on Google), it could guess their news interests even before they clicked a single news article.
Real Life: When they turned it on for real users, more people came back to the site (DAU) and clicked on more articles (CTR).

Summary Analogy

Old Way: A robot that counts how many times you looked at red cars and assumes you only like red cars.
New Way (Teacher): A detective who looks at your red car photos, your search for "fast engines," and your history of visiting race tracks, then deduces: "This person loves high-speed sports cars."
The Innovation: They trained a genius detective, then taught a tiny, super-fast intern to do the same job so the company can afford to hire one for every customer.

This paper is a major step forward because it proves that reasoning (thinking deeply about why you clicked) is better than just pattern matching (counting clicks), and it shows how to make that heavy thinking fast enough for the real world.

1. Problem Statement

The paper addresses the challenge of Cross-Domain News Recommendation, specifically the difficulty of inferring deep, reusable user interests from heterogeneous and noisy signals (e.g., search logs, click logs, browsing history) that extend beyond direct news consumption.

Limitations of Existing Methods: Traditional embedding-based and collaborative filtering approaches often fail to capture deep sequential signals or learn reasonable personalized embeddings for all users, especially when data is sparse or noisy. They rely heavily on historical interaction similarity rather than semantic reasoning.
Core Objective: To move beyond surface-level behaviors and generate a list of interest-driven news search queries that represent a user's underlying needs. These queries serve as a bridge to retrieve relevant news articles via standard retrieval systems (e.g., ANN).

2. Methodology

The proposed framework is a reasoning-driven Reinforcement Learning (RL) system that operates in three main stages:

A. Data Cleaning (Noise Mitigation)

Raw behavioral logs contain significant noise (e.g., random strings, navigational inputs).

Approach: A lightweight binary classifier (based on RoBERTa) is trained to filter raw signals.
Supervision: Labels are generated using GPT-5 with a designed prompt to distinguish informative behaviors from irrelevant noise.
Output: A refined set of behavioral signals $\tilde{B}_u$ for each user.

B. Interest Generation via Multi-Reward RL (Teacher Model)

The core innovation is formulating interest discovery as a policy optimization problem using Group Relative Policy Optimization (GRPO).

Model: An instruction-tuned Large Language Model (LLM), specifically Qwen2.5-32B, acts as the "Teacher."
Process: Given cleaned signals, the policy generates a group of $G$ candidate lists of search queries.
Reward Function: The model is optimized using a composite reward signal $R$ $R$ composed of five dimensions:
1. Retrieval Alignment ( $R_{align}$ ): Ensures generated queries successfully retrieve relevant articles from an internal ANN index.
2. Interest Coverage ( $R_{cov}$ ): Encourages the list to cover diverse latent interests.
3. Query Specificity ( $R_{spec}$ ): Promotes content-rich, non-generic queries.
4. Intra-list Diversity ( $R_{div}$ ): Maximizes semantic distance between queries within the same list.
5. Structural Validity ( $R_{struct}$ ): A binary check for output format compliance.
Optimization: The objective maximizes expected return using group-relative advantage estimation, avoiding the need for a separate value network.

C. On-Policy Distillation (Student Model)

To make the system scalable for online serving (low latency/high throughput), the heavy teacher model is distilled into a compact student.

Target Model: Qwen2.5-0.5B.
Technique: On-Policy Distillation. Unlike standard supervised distillation, this minimizes the Kullback-Leibler (KL) divergence between the student's distribution and the teacher's distribution conditioned on the student's own generated prefix.
Benefit: This allows the student to learn the teacher's reasoning trajectory and policy directly, preserving high-quality interest modeling while reducing inference costs.

3. Key Contributions

Scaling Properties of LLMs for Interest Modeling: The paper demonstrates that increasing model capacity (space scaling) and inference-time sampling (time scaling) leads to consistent, scaling-law-like improvements in interest quality and downstream retrieval performance, even with noisy cross-domain signals.
On-Policy Distillation Framework: A novel approach to transfer the compute-intensive reasoning policy of a large LLM to a lightweight student model, enabling deployment in production systems without sacrificing significant performance.
Multi-Reward RL Formulation: A novel decomposition of user interest generation into five complementary criteria (retrieval, coverage, specificity, diversity, validity) optimized via GRPO.
Real-World Validation: The first known deployment of a reasoning-driven RL framework for user interest modeling in a large-scale commercial news recommendation system.

4. Experimental Results

Offline Evaluation

Dataset: Real-world logs from a large-scale production system (500k users, 1.2M news items).
Baselines: Compared against GRU, SASRec, NRHUB, PinSage, and HSTU.
Performance:
- The RL-trained Teacher (Qwen2.5-32B) achieved the best results across all metrics (Recall@10: 0.271, NDCG@10: 0.241), outperforming the strong HSTU baseline.
- The Distilled Student (Qwen2.5-0.5B) retained most of the gains (Recall@10: 0.261), significantly outperforming all traditional baselines despite being 64x smaller.
Scaling Laws:
- Model Capacity: Increasing parameters from 0.5B to 32B showed consistent gains.
- Inference-Time Sampling: Increasing the number of sampled lists ( $N$ ) via a "best-of-N" strategy improved performance, showing diminishing returns but clear benefits.

Online A/B Testing

Setup: 7-day flight in a production environment, deployed as an additional recall path.
Results:
- DAU (Daily Active Users): +0.95% lift ( $p=0.012$ ).
- CTR (Click-Through Rate): +0.22% lift ( $p=0.031$ ).
- Cold Start Users: A significant +4.38% lift in DAU for cold users, demonstrating the model's ability to infer interests from heterogeneous signals when direct news history is missing.

5. Significance

This work represents a paradigm shift in recommendation systems by moving from embedding-based similarity to reasoning-based interest generation.

Scalability: It solves the deployment bottleneck of LLMs in recommendation systems through effective on-policy distillation.
Robustness: It proves that LLMs can effectively reason over noisy, multi-domain data to uncover deep user intents, particularly benefiting cold-start scenarios.
Industrial Impact: It provides a blueprint for integrating large-scale RL and LLM reasoning into production-grade recommendation pipelines, achieving measurable business metrics (DAU/CTR) improvements.