Technical Summary: Towards Root Memories: Benchmarking and Enhancing Implicit Logical Memory Retrieval for Personalized LLMs

1. Problem Statement

Personalized Large Language Models (LLMs) rely on memory systems to retrieve relevant context for generating accurate responses. However, current retrieval mechanisms predominantly depend on semantic similarity. This approach fails in scenarios where the critical information required for a logical response is semantically distant from the user's query but logically essential.

The paper identifies a specific gap: existing methods often retrieve memories that overlap semantically with the query but lack the necessary logical constraints to form a safe or accurate personalized response. Conversely, they miss "implicit logical memories" that are semantically unrelated to the query yet dictate the correct decision (e.g., a user's current stress level overriding a past preference for a high-pressure task). Current benchmarks are inadequate because they do not explicitly model the conflict between semantic similarity and logical relevance, nor do they sufficiently test the retrieval of these "semantically distant but logically critical" memories.

2. Methodology

2.1 The IMLogic Benchmark

To address the evaluation gap, the authors constructed IMLogic, the first high-quality benchmark for implicit logical memory retrieval in long-dialogue scenarios.

• Data Source: Built upon realistic historical conversations and memory entries from HaluMem.
• Scale: Comprises 20 long conversations (avg. 8.3k tokens/session), over 15,000 memory entries, and 2,216 QA pairs.
• Core Design: The benchmark explicitly creates a "semantic gap" between the query and the critical memory. It pairs a Semantic Memory ($m_s$) (highly similar to the query but insufficient for the answer) with a Logical Memory ($m_l$) (semantically distant but logically indispensable).
• Construction Pipeline: A collaborative "Generator-Judger-Refiner" multi-agent pipeline ensures high quality. The Generator creates queries that align semantically with $m_s$ while hiding $m_l$; the Judger verifies logical rationality and semantic stealthiness; the Refiner iterates based on feedback.
• Evaluation Dimensions: The benchmark tests systems on Memory-level (retrieving from pre-annotated entries) and Conversation-level (processing raw dialogue streams). It categorizes errors into four types: Correct, Preference Trap (blindly following $m_s$), Fabrication Trap (inventing details to justify $m_s$), and Generic Trap (ignoring personal context).

2.2 The RootMem Framework

To solve the retrieval challenge, the authors propose RootMem, a plug-and-play framework that augments semantic retrieval with Root Memory.

• Root Memory Concept: A structured representation that distills reusable, personalized decision logic from long-term user histories. Unlike raw memory or user profiles, a root memory unit captures how facts constrain future decisions.
• Structure: Each unit $U_i$ is defined as $\langle R_i, E_i \rangle$:
  • Execution Rules ($R_i$): Specify activation conditions and response-guiding logic (e.g., "If the user is in a recovery phase, avoid high-stress recommendations").
  • Personalized Logical Evidence ($E_i$): Provides the specific user facts and constraints grounding the rules (e.g., "User is currently in financial rebuilding phase").
• Workflow:
  1. Construction (Offline): An LLM-based extractor ($\Phi_C$) aggregates raw memory batches to update, create, or merge root memory units, preserving the underlying decision logic.
  2. Routing (Online): During inference, a Root Memory Router ($\Phi_R$) evaluates the execution rules ($R_i$) of all units against the current query. It activates relevant units ($U_{act}$) that impose logical constraints, regardless of semantic overlap.
  3. Augmentation: The activated root memory blocks (containing rules and evidence) are concatenated with semantically retrieved memories to form a unified context for response generation.

3. Key Contributions

1. IMLogic Benchmark: The first benchmark specifically targeting implicit logical memory retrieval, explicitly modeling the divergence between semantic similarity and logical relevance in long conversations.
2. Root Memory Representation: The introduction of a structured memory unit that preserves reusable personalized decision logic, distinct from simple fact storage.
3. RootMem Framework: A plug-and-play system that constructs and routes root memories to complement semantic retrieval, significantly enhancing logical reasoning capabilities in personalized agents.

4. Experimental Results

Experiments were conducted on IMLogic using various baselines, including lexical (BM25), semantic (dense embeddings), graph-based, and hybrid retrieval methods.

• Memory-Level Performance: RootMem achieved 55.23% accuracy on the multiple-choice task, surpassing the strongest baseline (Query Reconstruction + Rerank at 43.41%) by a relative improvement of 27.23%. It significantly reduced "Preference" and "Generic" errors, which are caused by failing to retrieve the logical memory $m_l$.
• Conversation-Level Performance: When integrated as a compensatory component into existing memory agents (e.g., EverMemOS, LightMem), RootMem consistently boosted accuracy. The average relative improvement across the top three baselines was 26.36%, bringing all absolute accuracies above 50%.
• Ablation Studies: Removing the Root Memory Router caused the largest performance drop (-8.51%), confirming its necessity for activating relevant logic. Removing execution rules or evidence also degraded performance, indicating that both the logic and the grounding facts are essential.
• Generalization: RootMem demonstrated consistent improvements across different LLM backbones (GPT-4o-mini, Qwen3, GLM-4.6), with relative gains ranging from 15.58% to 71.52% in conversation-level settings.
• Open-Ended QA: In open-ended generation tasks, RootMem improved the ability of models to generate grounded, personalized responses, reducing reliance on generic or hallucinated answers.

5. Significance and Claims

The paper claims that RootMem addresses a fundamental limitation in current personalized LLMs: the inability to retrieve and utilize logically critical information that lacks semantic overlap with the user's immediate query. By shifting the focus from purely semantic matching to decision-preserving logic retrieval, the framework enables models to generate responses that are not only contextually relevant but also logically sound and safe.

The authors emphasize that their approach is practical, offering a favorable overhead-performance balance by shifting complex logic modeling to offline consolidation and using lightweight routing during inference. They position IMLogic as a necessary step forward for evaluating the true reasoning capabilities of memory systems in personalized agents, moving beyond simple retrieval of "relevant" information to the retrieval of "necessary" logic.

Limitations Acknowledged: The authors note that IMLogic is currently text-based and does not cover multimodal memories (images, voice). Additionally, while offline construction is efficient, further optimization is needed for real-time deployment, and the benchmark currently focuses on specific query types (recommendation, advice, chatting).