Technical Summary: RRCM – Ranking-Driven Retrieval over Collaborative and Meta Memories

1. Problem Statement

Large Language Models (LLMs) have emerged as a promising paradigm for recommender systems due to their semantic understanding and reasoning capabilities. However, current LLM-based recommenders face two critical challenges in constructing decision-relevant contexts from heterogeneous evidence:

1. Fixed Context Construction: Existing methods typically rely on static strategies to incorporate collaborative behavioral evidence (e.g., cross-user interactions) and item metadata. These approaches use predefined prompts, static retrieval pipelines, or handcrafted injection mechanisms. Consequently, they fail to explicitly learn which evidence is beneficial for a specific instance, when it should be retrieved, or how it should be integrated into the LLM's decision-making process.
2. Context-Efficiency Bottleneck: Real-world recommendation data involves rich metadata and extensive interaction histories. Including all available information leads to prohibitively long context windows, while aggressive compression or heuristic filtering risks discarding fine-grained evidence crucial for accurate ranking, particularly for niche or new items where the LLM may hallucinate properties without grounded evidence.

The core problem is not merely providing more context, but adaptively constructing the right context for each recommendation decision to balance accuracy with computational efficiency.

2. Methodology: RRCM Framework

The authors propose RRCM (Ranking-driven Retrieval over Collaborative and Meta Memories), a framework that treats recommendation as an adaptive evidence acquisition and reasoning process.

2.1 Dual-Memory Retrieval Corpus

RRCM utilizes a unified retrieval corpus $\mathcal{M}$ containing two complementary memories, both represented in natural language:

• Collaborative Memory: Stores anonymized interaction sequences from historical users (e.g., "User 123 History: [Item A, Item B, ...]"). This provides behavioral evidence regarding how users with similar preferences continue their interactions.
• Meta Memory: Stores rich item metadata (e.g., "Movie: Inception; Director: Nolan; Genre: Sci-Fi"). This provides grounded item-level attributes.

Both memories are accessed via a single flexible interface, Retrieve(q), where $q$ is a natural-language query generated by the model.

2.2 Interleaved Reasoning and Retrieval

RRCM models inference as a multi-turn loop where the policy $\pi_\theta$ interleaves internal reasoning with external memory access:

1. Initial State: The process begins with a lightweight context consisting primarily of item titles ($S_{init}$).
2. Reasoning & Assessment: The model analyzes the current state to determine if the available evidence is sufficient for a reliable ranking decision.
3. Action Decision:
   • Retrieval Action: If additional evidence is needed, the model generates a natural-language query to retrieve from either the Collaborative or Meta memory. The retrieved passage is appended to the context, and reasoning resumes.
   • Termination Action: If the context is deemed sufficient, the model stops retrieval and generates the final recommendation string.

This design allows the model to dynamically expand the context only when external evidence is expected to improve the decision, such as retrieving metadata to disambiguate a niche item or collaborative histories to identify behaviorally supported candidates.

2.3 Ranking-Driven Policy Optimization

Since the optimal timing, memory type, and query formulation are unknown, RRCM optimizes the policy using Reinforcement Learning (RL) driven directly by final recommendation quality.

• Generation Grounding: The LLM's natural-language output is mapped to the discrete item space using a semantic embedding approach (e.g., Sentence Transformers) to generate a ranked candidate list.
• Reward Formulation: The reward function $R(\tau)$ is a weighted sum of top-$n$ ranking metrics (Hit Ratio and NDCG) on the grounded candidate list compared to the ground truth, plus a format-validity term.
• Optimization: The authors instantiate the objective using Group Relative Policy Optimization (GRPO). This method estimates the baseline from group rewards without a separate critic, encouraging the policy to retrieve selectively only when external evidence improves the final top-$k$ ranking quality.

3. Key Contributions

1. Unified Heterogeneous Memory Retrieval: The paper formulates LLM-based recommendation as a ranking-driven context construction problem over heterogeneous memories. It proposes a dual-memory framework representing both collaborative histories and item metadata in natural language, accessible via a unified interface.
2. RL-Based Adaptive Retrieval and Reasoning: RRCM introduces a policy that learns when to retrieve, which memory to query, and how to reason over the evidence. This policy is optimized via an outcome-driven ranking objective, eliminating the need for handcrafted retrieval rules or fixed CF injection mechanisms.
3. Selective Evidence Acquisition: The framework learns to retrieve collaborative or meta memories only when necessary, reducing unnecessary context construction and improving efficiency while maintaining high recommendation performance.

4. Experimental Results

The authors evaluated RRCM on three public e-commerce datasets: Goodreads, MovieLens, and Amazon CDs & Vinyl.

• Performance: RRCM consistently outperformed strong baselines, including traditional sequential models (SASRec), supervised fine-tuned LLMs (BIGRec, D3), DPO-based methods (DMPO, SPRec), reasoning-oriented models (REPR, LatentR3), and retrieval-augmented or CF-injection baselines (K-RagRec, BinLLM, CoLLM).
  • On Goodreads, RRCM achieved a 16.04% improvement in HR@10 over the best baseline (LatentR3).
  • On Amazon CDs & Vinyl, it showed a 15.18% improvement in HR@10.
• Ablation Studies: Removing any key component (Collaborative Memory, Meta Memory, or Reasoning) resulted in performance degradation across all datasets, confirming the complementary value of each. The importance of specific components varied by domain (e.g., metadata was crucial for Goodreads, while collaborative history was vital for CDs & Vinyl).
• Policy Behavior: Analysis of the training process revealed that the policy learns to become more selective over time. The average number of retrieval calls and response token length decreased significantly as the model converged, indicating it learned to rely on internal knowledge when sufficient and retrieve only when necessary.
• Long-Tail Performance: RRCM showed a more pronounced advantage over baselines on long-tail items, validating its ability to use metadata retrieval to disambiguate obscure or new items where LLMs lack prior knowledge.

5. Significance and Claims

The paper claims that RRCM addresses the fundamental limitations of current LLM recommenders by shifting from static, fixed-context strategies to an adaptive, agentic approach.

• Efficiency vs. Accuracy: By learning to retrieve selectively, RRCM mitigates the context-length bottleneck without sacrificing the benefits of external evidence. It demonstrates that "more context" is not always better; "the right context" is.
• End-to-End Learning: Unlike hybrid systems that use fixed rules for retrieval or injection, RRCM learns the entire retrieval-reasoning-recommendation loop end-to-end, driven solely by the final ranking objective.
• Generalizability: The framework is shown to be robust across different domains (books, movies, music) and backbone models (Qwen3-1.7B, Llama-3.2-3B), suggesting that adaptive memory retrieval is a generalizable strategy for enhancing LLM-based recommendation.

The authors conclude that RRCM provides a flexible framework for more efficient and accurate LLM recommendation, though they note limitations regarding the need for online A/B testing and the potential for expanding the retrieval corpus to include richer user features or temporal dynamics in future work.