💻 computer science

Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging

This paper proposes REAM, a novel training-free model merging framework that performs fine-grained, attention-head-level merging of slow-thinking and fast-thinking LLMs to significantly reduce reasoning verbosity in recommender systems while preserving recommendation accuracy.

Original authors: Linh Dieu Le, Tong Chen, Shazia Sadiq, Hongzhi Yin, Ming Jin, Junliang Yu

Published 2026-08-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Linh Dieu Le, Tong Chen, Shazia Sadiq, Hongzhi Yin, Ming Jin, Junliang Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot friend who loves giving you movie or book recommendations. Sometimes, this robot is a "fast thinker." It looks at what you like and instantly shouts out a suggestion. It's quick, but sometimes it misses the nuance because it didn't pause to think deeply. Other times, the robot is a "slow thinker." Before giving a suggestion, it writes out a long, detailed essay explaining why it thinks you'll like a specific item. It analyzes your past reviews, the book's themes, and how they match up. This "slow thinking" usually leads to better, more accurate recommendations, but there's a catch: the robot's essay is often way too long. It rambles on with unnecessary details, taking up a lot of time and computer power just to say something simple.

Scientists are trying to figure out how to make these "slow thinkers" more efficient without losing their smarts. They want a robot that thinks deeply enough to be accurate but stops rambling so it can give answers faster. The big question is: Can we teach the slow thinker to be concise without forcing it to forget how to think deeply? This paper explores a clever trick called "model merging," which is like blending two different versions of a robot into one super-robot, hoping to get the best of both worlds: the speed of the fast thinker and the accuracy of the slow thinker.

The Problem: The Over-Explainer Robot

In the world of AI recommenders, there are two main styles. The "Fast Thinker" jumps straight to the answer, like a friend who says, "You'll love this!" without explaining why. The "Slow Thinker" is more like a detective who says, "I looked at your history, saw you like romance, checked the book's plot, and concluded you'll love it because..." This detective style is great for accuracy, especially when the clues are tricky, but the detective often writes a novel when a short note would do. This "verbosity" wastes time and money.

Previous attempts to fix this involved either retraining the robot (which is expensive and slow) or telling it to "stop talking" during the process (which often makes it dumber). The authors of this paper wanted a smarter way to trim the fat without hurting the brain.

The Solution: Merging the Twins

The researchers, led by Linh Dieu Le and colleagues, proposed a new method called REAM (Reasoning-Head-Aware Merging). Think of it like this: imagine you have two versions of the same person. One is a fast, decisive version who gives short answers. The other is a slow, thoughtful version who writes long explanations. Instead of trying to teach the slow person to be fast (which is hard), they decided to "merge" the two personalities into one new person.

But here's the tricky part: you can't just mix them 50/50. If you blend them too much, the new person might become confused and give bad recommendations. If you don't blend them enough, they'll still talk too much. The old way of merging was like pouring two buckets of paint together and stirring them all the same. The new method, REAM, is much more precise.

How REAM Works: The "Head" Check

The secret sauce of REAM is that it doesn't treat the robot's brain as one big blob. Instead, it looks at the brain's tiny parts, called attention heads. You can think of these heads as different specialists in a team. Some specialists are great at finding specific clues (like "this user loves romance"), while others are great at connecting those clues to the final decision.

The researchers discovered that not all specialists are equally important for the "thinking" part.

Retrieval Criticality: Some heads are like librarians who find the right books. If you mess with them, the robot forgets what the user likes.
Decision Faithfulness: Other heads are like judges who decide the final rating. If you mess with them, the robot might give a random score instead of a thoughtful one.

The paper suggests that some heads are "critical" and must be protected, while others are "safe" to change. REAM uses a special math formula to figure out which heads are which. It then takes the "Fast Thinker's" concise style and injects it only into the safe heads, while leaving the critical heads alone so they can keep doing their deep thinking. It's like telling the chatty parts of the robot to "zip it," while letting the smart parts keep working.

What They Found

The team tested this on three different datasets (Amazon Books, Amazon Music, and Yelp reviews). They compared their new method against the original slow thinker, the fast thinker, and other merging techniques.

The results were promising. By using REAM, they were able to cut the length of the robot's reasoning by up to 24.3% on the Amazon Book dataset. That's a huge reduction in the "rambling." Even better, the robot didn't get dumber. In fact, REAM maintained the accuracy of the slow thinker better than any other method they tested.

For example, on the Amazon Book dataset, the original slow thinker generated about 313 tokens (words/pieces of text) per answer. REAM brought that down to 237 tokens, while keeping the error rate (how wrong the rating was) lower than the original slow thinker. Other methods that tried to merge the models often made the robot less accurate or didn't shorten the text as much.

What This Means

The paper suggests that we don't need to retrain robots from scratch to make them efficient. Instead, we can carefully blend a "fast" version with a "slow" version, but we have to be very picky about where we apply the fast style. By protecting the specific parts of the brain that do the heavy lifting for reasoning, we can get a robot that is both smart and concise.

The authors note that this is the first time this kind of "head-level" merging has been tried specifically for recommendation systems. While the results are strong, they also point out that as robots get bigger and more complex, we might need to tweak this method further. But for now, REAM shows a clear path to making AI recommenders faster and less chatty without losing their ability to understand us.

Technical Summary: Towards Efficient Reasoning in LLM-Based Recommender Systems via Model Merging

1. Problem Statement

Large Language Model (LLM)-based recommender systems are increasingly adopting "slow-thinking" architectures that generate explicit Chain-of-Thought (CoT) reasoning traces before predicting user ratings. While these models often achieve higher accuracy than "fast-thinking" models that predict directly, their reasoning traces are frequently unnecessarily verbose. This verbosity increases inference latency and computational costs without commensurate gains in accuracy.

Existing approaches to compressing reasoning traces face significant limitations:

Training-based methods (e.g., distillation, length-penalized fine-tuning) require substantial data construction and model adaptation costs, limiting scalability.
Inference-time methods (e.g., token budgets, prompting for shorter steps) are brittle, often varying in effectiveness across different inputs, and lack robustness.

The paper posits that model merging—combining models in a shared parameter space without additional training—offers a promising alternative. Specifically, merging a slow-thinking model (high accuracy, verbose) with a fast-thinking counterpart (lower accuracy, concise) could theoretically transfer the conciseness of the latter while preserving the reasoning capabilities of the former. However, existing merging methods apply uniform coefficients across model components or rely on coarse-grained signals (e.g., layer-wise activation statistics), failing to account for the heterogeneous roles of different components in recommendation reasoning. Indiscriminate merging risks disrupting critical evidence retrieval or decision-making processes, leading to degraded recommendation quality.

2. Methodology: REAM (Reasoning-Head-Aware Merging)

The authors propose REAM, the first model merging framework designed specifically for reasoning compression in recommender systems. REAM operates at the granularity of individual attention heads, assigning distinct merge coefficients based on a head's specific contribution to the reasoning process and its sensitivity to parameter changes.

The method involves three core stages:

A. Assessing Head-Level Reasoning Importance

REAM identifies which attention heads are critical for the reasoning underlying the final prediction using two complementary signals:

Retrieval Criticality ( $\kappa$ ): Measures a head's ability to retrieve relevant user-item evidence from the input context and previous reasoning steps to support the developing trace. It is calculated by counting successful retrieval events where a head attends to tokens matching the evidence region (user history, item description, or compatibility analysis) during the generation of reasoning segments.
Decision Faithfulness ( $\text{faith}$ ): Measures the strength of a head's connection between the final rating prediction and the preceding reasoning, specifically the "match" segment where user-item compatibility is synthesized. It is quantified as the share of attention the rating token directs toward the "match" segment relative to other regions (prompt and analysis).

B. Estimating Fast-Thinking Update Sensitivity

To ensure that merging does not destabilize the slow-thinking model, REAM estimates the update sensitivity ( $d$ ) for each head. This is derived using the diagonal empirical Fisher information matrix of the slow-thinking model, weighted by the squared magnitude of the fast-thinking update (task vector). Heads with high sensitivity indicate that even small parameter changes along the fast-thinking update direction could significantly alter the model's behavior (loss landscape is sharp), necessitating conservative merging.

C. Constrained Optimization for Coefficient Allocation

REAM formulates the merging process as a constrained optimization problem. The goal is to maximize the total applied fast-thinking update (to achieve conciseness) while limiting the aggregate "reasoning-grounded risk."

Perturbation Weight: A risk score $s_b$ is computed for each head $b$ by combining retrieval criticality, decision faithfulness, and update sensitivity: $s_b = \exp(\gamma(\kappa_b + \text{faith}_b)) \cdot d_b$ .
Optimization: The method solves for merge coefficients $\alpha_b$ that maximize the injected update magnitude subject to a constraint on the total perturbation cost ( $\sum s_b \alpha_b^2 \leq \varepsilon$ ).
Solution: This yields a "water-filling" solution where heads with high reasoning importance or high sensitivity receive lower merge coefficients (more conservative updates), while less critical and less sensitive heads absorb more of the fast-thinking update.

FFN Handling: Since Feed-Forward Networks (FFNs) lack head structure, REAM assigns coefficients at the layer level. Empirical analysis suggests that merging the final FFN layers degrades the accuracy-efficiency trade-off; thus, REAM excludes the final layers (e.g., layers 30–35 in Qwen2.5-3B) from merging.

3. Key Contributions

Problem Formulation: The paper formulates reasoning compression in slow-thinking recommenders as a selective merging problem, arguing that the fast-thinking update must be allocated according to the specific contribution of each model component to recommendation performance.
REAM Framework: The authors introduce REAM, a training-free framework that operates at the individual attention head level. Unlike prior work that uses uniform or layer-wise coefficients, REAM selectively transfers concise behavior while protecting heads critical for evidence retrieval and decision grounding.
Mechanistic Signals: The work identifies and utilizes two distinct, causally validated signals—retrieval criticality and decision faithfulness—to guide the merging process, demonstrating that these signals identify complementary, non-redundant head roles.

4. Experimental Results

Experiments were conducted on three benchmark datasets: Amazon Book, Amazon Music, and Yelp, using Qwen2.5-3B-Instruct as the base model. REAM was compared against the source slow-thinking model (RecZero), the fast-thinking model (TALLRec), and eight training-free merging baselines (including Task Arithmetic, DARE, AIM, ACM, and RAIN-Merging).

Accuracy-Efficiency Trade-off: REAM achieved the most consistent balance across all datasets. It reduced reasoning length by up to 24.3% (on Amazon Book) while maintaining or improving recommendation accuracy (MAE and RMSE) compared to the slow-thinking baseline.
Comparison with Baselines: General-purpose merging methods (e.g., Task Arithmetic, DARE) shortened reasoning but consistently degraded MAE or altered the rating behavior (e.g., reducing RMSE but increasing MAE, indicating error redistribution). Data-driven baselines like AIM and ACM, which operate at the layer level, failed to match REAM's performance, producing longer traces and higher MAE.
Ablation Studies: Removing Fisher sensitivity had the largest negative impact on performance, confirming its role in preventing destabilization. Ablating retrieval-critical or faithfulness-critical heads caused significant increases in Negative Log-Likelihood (NLL) and Jensen-Shannon Divergence (JSD) compared to random head ablation, validating the causal importance of the identified heads.
Generalizability: REAM demonstrated transferability across different model scales (Qwen2.5-7B), model families (Llama-3.2-3B), and training paradigms (RecOne), consistently outperforming baselines in the accuracy-efficiency trade-off.

5. Significance and Claims

The paper claims that REAM represents a significant step forward in making LLM-based recommender systems more efficient without sacrificing the quality of reasoning. Its primary significance lies in:

Training-Free Efficiency: It offers a practical, training-free alternative to costly distillation or fine-tuning for reasoning compression.
Granularity and Selectivity: It demonstrates that effective compression requires head-level selectivity grounded in the specific mechanics of recommendation reasoning (evidence retrieval and decision synthesis), rather than generic model merging heuristics.
Preservation of Quality: By selectively injecting conciseness only into non-critical components, REAM successfully reduces inference costs (token count) while preserving the "deliberative" advantages of slow-thinking models, addressing the "overthinking" problem in recommendation systems.

The authors modestly note that while REAM narrows the gap, a small accuracy gap remains compared to the original slow-thinking model at larger scales, suggesting a direction for future work in extending selective merging to larger recommenders.