🤖 AI

QUMem: Personalized Memory for Query-Conditioned User-State Inference in LLM Agents

QUMem is a structured memory framework that enhances LLM agent personalization by segmenting interaction histories into semantic episodes, decomposing them into independently retrievable memory types, and employing a multi-agent retrieval process to infer temporally and contextually valid user states, thereby achieving state-of-the-art performance on personalization benchmarks.

Original authors: Heng Wang, Yifei Li, Lingling Zhang, Pengyu Li, Xinyu Che, Xinyu Zhang, Zesheng Yang

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Heng Wang, Yifei Li, Lingling Zhang, Pengyu Li, Xinyu Che, Xinyu Zhang, Zesheng Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a new generation of computer programs has emerged that can do more than just answer questions; they can act as persistent assistants, remembering past conversations and adapting their behavior over time. These systems, known as agents, rely on a form of digital memory to function effectively. Just as a human assistant recalls a client's favorite coffee order or remembers that a project deadline was moved last week, these agents need to store and retrieve information from long histories of interaction. However, human memory is not a simple filing cabinet where every conversation is stored as a single, unchangeable block. Instead, we naturally separate facts from feelings, distinguish between a temporary mood and a lasting preference, and understand that the context of a conversation changes as time passes. For artificial intelligence to truly understand a person, it must be able to do the same: to sift through years of dialogue, separate the relevant details from the noise, and reconstruct a clear picture of who the user is right now.

For a long time, researchers have tried to solve this by building better ways to store and search these digital conversations. The prevailing approach has been to treat memory like a library where a user's query acts as a search term, pulling back the most similar chunks of text. While this works for simple questions, it struggles when the situation is complex. If a user changes their mind about a topic over several weeks, or if a single conversation contains both a fleeting complaint and a permanent rule they want to follow, standard systems often get confused. They might mix unrelated events together or fail to see how a past decision connects to a current request. The result is an assistant that remembers the words but misses the meaning, unable to tell the difference between a temporary preference and a core value.

A team of researchers at Xi'an Jiaotong University has proposed a new way to handle this challenge, introducing a system called QUMem. Rather than simply searching for similar text, this system is designed to understand the structure of a conversation and the specific role of the information within it. The researchers argue that to build a truly personalized assistant, the computer must first organize the raw history of interactions into meaningful stories, or "episodes," based on the natural flow of the conversation. It then breaks these stories down into three distinct types of information: concrete facts about what happened, specific preferences or rules the user has stated, and broader insights or principles that can be applied to new situations. By separating these elements, the system can keep the full context of an event while still allowing the assistant to pick out just the specific detail it needs for the current moment.

The core innovation lies in how the system processes a new request. Instead of asking the computer to guess what to look for, QUMem uses a three-step reasoning process. First, an agent analyzes the user's current question to determine exactly what information is needed to answer it properly. It asks itself whether the task requires a specific fact, a past preference, or a general rule. Next, a second agent plans a search, directing the system to look in the specific "shelves" where that type of information is stored, rather than searching the entire history at once. Finally, a third agent takes the retrieved pieces of evidence and weaves them together into a coherent picture of the user's current state. This final picture is not just a list of old memories; it is a synthesized understanding of who the user is in this specific moment, taking into account how their preferences have evolved and which rules still apply.

To test this approach, the researchers compared their system against several existing methods using two different benchmarks designed to measure how well an AI can remember and adapt to a user. In one test, the system had to answer questions about a user's changing preferences over long conversations. The new method significantly outperformed the others, achieving state-of-the-art performance on the PersonaMem benchmark. With the GPT-4o-mini model, QUMem improved overall accuracy to 61.02%, and with Gemini-3.5-flash, it reached 70.58%, compared to roughly 53% and 63% respectively for the next best systems. The improvement was even more pronounced when the task required the assistant to track how a preference had changed over time or to apply a past lesson to a completely new scenario. In a second test involving a mobile assistant that had to perform tasks based on user habits, the system again achieved the highest success rate, proving that this structured way of thinking about memory helps the computer take better action, not just give better answers.

The researchers also found that the way the system is built makes it more efficient. Older methods often trigger a heavy computational process every time a new message arrives, regardless of whether it is important. The new system waits until a conversation naturally concludes a topic before organizing the memory, and then it uses a fixed, small number of steps to sort the information. This means it uses far less computing power to build its memory, allowing it to handle long histories without slowing down. The study suggests that the key to long-term personalization is not just storing more data, but organizing it in a way that respects the difference between a fact, a preference, and a lesson. By treating the user's history as a collection of structured, evolving stories rather than a flat list of words, the system can infer a much clearer and more accurate understanding of the person it is serving.

Technical Summary: QUMem

Problem Statement

Large Language Model (LLM) agents increasingly rely on external memory systems to support long-term personalization. However, existing memory frameworks face three critical limitations that hinder effective user-state inference:

Rigid Boundaries: Fixed-turn, fixed-token, or session-based boundaries often mix unrelated dialogue or fragment a single event from its causes, decisions, and outcomes. This disrupts the event-level context required for subsequent retrieval.
Coupled Information: Storing multiple pieces of information from a single interaction as a single memory unit binds distinct semantic functions (e.g., a general preference vs. a context-specific constraint) together. This prevents the system from retrieving only the components relevant to a specific query.
Inadequate Query Formulation: Treating the current task as a single top- $k$ similarity retrieval query often yields fragments that are individually relevant but fail to jointly capture preference evolution, temporal validity, and contextual applicability.

The core challenge is not merely retrieving relevant memories, but inferring a task-relevant user state—a structured representation of user information supported by historical evidence, relevant to the current query, and applicable in the current context.

Methodology: QUMem

QUMem is a structured memory framework designed for query-conditioned user-state inference. It operates through three sequential modules:

1. Dynamic Episode Construction

To preserve event-level context, QUMem abandons fixed boundaries in favor of semantic continuity.

Mechanism: A lightweight, fine-tuned classifier ( $f_\theta$ ) analyzes adjacent user utterances to determine if they belong to the same event or task.
Process: The system maintains an open candidate episode. If the classifier determines continuity ( $c_t=1$ ), the utterance is appended. If continuity is broken ( $c_t=0$ ), the current episode is finalized, and a new one begins.
Benefit: This ensures that causes, decisions, feedback, and outcomes of the same event are kept together, preventing the disruption of event context before memory storage.

2. Typed Memory Decomposition

Once an episode is formed, it is decomposed into independently retrievable atomic memories based on three distinct functional roles:

Factual Memories ( $F$ ): Concrete user experiences, behaviors, activities, or states without inferred tendencies.
Preference Memories ( $P$ ): User choices, tendencies, requirements, and constraints associated with specific objects or contexts, including their rationales. These are not assumed to be indefinitely valid; their temporal position is tracked.
Transferable Insight Memories ( $I$ ): Abstracted decision principles derived from concrete choices and feedback, applicable to new objects or contexts.
Storage: Each memory retains its content ( $v$ ), type ( $d$ ), temporal position ( $p$ ), and source evidence ( $E$ ). This allows for fine-grained retrieval and provenance tracing.

3. Query-Conditioned User-State Inference

Instead of a single retrieval operation, QUMem employs three sequential agents to transform the task objective into a structured user state ( $Z_q$ ):

Information-Need Agent ( $A_1$ ): Identifies what historical information must be verified for the current query and explains its relevance, without prescribing specific retrieval queries.
Retrieval Planning Agent ( $A_2$ ): Rewrites information needs into self-contained queries and selects the appropriate memory stores (Factual, Preference, or Insight) for each query. It generates a retrieval plan $P_q$ .
User-State Inference Agent ( $A_3$ ): Retrieves candidate memories, selects task-relevant evidence, and organizes it into a structured user state $Z_q = (F_q, T_q, I_q)$ $Z_{q} = (F_{q}, T_{q}, I_{q})$ , where:
- $F_q$ : Chronologically organized facts.
- $T_q$ : Evolution of preferences and context-applicable constraints.
- $I_q$ : Abstracted decision principles applied to the current task.

This structured state is then fed to the downstream response model ( $\Psi$ ) to generate a personalized output.

Key Contributions

Structured Memory Framework: QUMem organizes interaction histories into semantic episodes and decomposes them into complementary, independently retrievable factual, preference, and insight memories. This preserves event context while enabling fine-grained reuse.
Three-Agent Inference Mechanism: The paper introduces a query-conditioned pipeline that separates information-need identification, retrieval planning over typed stores, and evidence-based state inference. This enables the joint interpretation of distributed, temporally separated evidence.
Empirical Validation: The framework is evaluated on PersonaMem and KnowU-Bench, demonstrating state-of-the-art performance in long-term personalization tasks.

Experimental Results

The paper evaluates QUMem against three baselines: A-MEM, Mem0, and Zep.

PersonaMem Benchmark: QUMem achieved the best performance across all context lengths (32K, 128K, 1M tokens) and query categories using both GPT-4o-mini and Gemini-3.5-flash base models.
- With GPT-4o-mini, QUMem improved overall accuracy from 52.99% (best baseline) to 61.02%.
- With Gemini-3.5-flash, it improved the strongest baseline from 63.29% to 70.58%.
- Gains were most significant in tasks requiring the tracking of full preference evolution, preference-aligned recommendations, and cross-context transfer.
KnowU-Bench: QUMem achieved the highest Success Rate (17.4% overall) and Average Score (0.53) among evaluated methods, improving the success rate by 4.6 percentage points over the strongest baseline. This indicates the method's effectiveness in translating historical preferences into concrete actions.
Efficiency: QUMem demonstrated superior efficiency in memory construction, requiring the fewest LLM calls per 100 turns (81.4) and the lowest token cost (1,221.2) compared to baselines that trigger construction at every turn or use complex verification loops.
Ablation Studies: Removing any of the three core components (Episode Construction, Memory Decomposition, or State Reconstruction) consistently degraded performance, confirming their complementary roles. The study also identified an optimal retrieval depth ( $k=5$ ).

Significance and Claims

The paper claims that QUMem addresses the fundamental disconnect in current memory systems between retrieving isolated facts and inferring a coherent user state. By explicitly separating the identification of information needs, the planning of retrieval over typed stores, and the joint interpretation of evidence, QUMem enables agents to handle evolving preferences and context-dependent constraints more effectively.

The authors emphasize that the value of the framework lies in:

Preserving event-level context during construction.
Distinguishing the functional roles of historical evidence (fact vs. preference vs. insight).
Performing task-driven user-state inference rather than simple similarity matching.

The results suggest that structured, query-conditioned inference is a critical step toward robust long-term personalization, particularly as interaction histories grow longer and more complex. However, the paper modestly notes that while QUMem improves preference inference, generating responses that are simultaneously novel and perfectly preference-aligned remains a challenge, particularly in "suggesting new ideas" tasks.