💬 NLP

Learning What to Share and What to Personalize: Hierarchical Strategy Co-Evolution for Agent Memory

This paper proposes HiPS, a hierarchical framework that co-evolves a globally shared memory strategy with user-specific adaptive rules to dynamically optimize what information agents retain or discard for personalized interactions.

Original authors: Yupeng Han, Shuochen Liu, Kai Zhang, Ze Liu, Zhihong Pan, Xianquan Wang

Published 2026-08-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yupeng Han, Shuochen Liu, Kai Zhang, Ze Liu, Zhihong Pan, Xianquan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, researchers are teaching computer programs to act as personal assistants, capable of holding long conversations and remembering details about the people they talk to. For these digital agents to be truly helpful, they must do more than just process the words spoken in the moment; they need to build a lasting profile of the user, recalling past preferences, emotional states, and life events. However, human conversations can stretch on for thousands of turns, creating a massive amount of information that no computer can hold in its immediate memory at once. The challenge lies in deciding what to keep and what to forget. If an agent remembers too little, it becomes forgetful and generic; if it remembers too much, it gets overwhelmed by irrelevant details and loses its ability to focus on what matters. The goal is to find a way to curate a compact, useful summary of a person's history that allows the agent to respond with consistency and care, no matter how long the conversation has been going on.

For a long time, the standard approach to this problem was to apply a single, rigid set of rules to everyone. Imagine a librarian who uses the exact same filing system for every single patron, regardless of whether they are a casual reader or a research scholar. This "one-size-fits-all" method works well for common situations but fails when users have unique needs or when their behavior changes over time. Recent attempts to make these systems smarter involved teaching them to learn from experience, but even these advanced methods often settled on a fixed strategy before the learning began, or they treated every user as if they were exactly the same. The result was often a compromise that satisfied the average user but left many others with responses that felt slightly off, missing the subtle nuances that make a conversation feel personal.

A team of researchers at the University of Science and Technology of China has proposed a new way to solve this, calling their framework HiPS. Instead of forcing every user into the same mold, their system splits the memory management task into two parts: a shared foundation that applies to everyone, and a flexible layer that adapts specifically to the individual. Think of it like a universal grammar that everyone speaks, combined with a personal dialect that only you use. The shared part learns general principles about how to organize information effectively, such as how to handle conflicting facts or how to summarize a long story. The personal part, however, watches for users who behave differently from the crowd. If a user has a specific way of interacting that the general rules don't capture well, the system creates a custom set of instructions just for them.

What makes this approach distinct is that these rules are not static; they evolve alongside the agent's performance. The system constantly tests its own strategies by watching how well the agent does its job. If a rule helps the agent give a better answer, that rule gets stronger and more likely to be used. If a rule causes confusion or errors, it gets weakened or discarded. This creates a cycle of improvement where the shared rules and the personal rules influence each other. Sometimes, a rule that started as a personal quirk for one user proves so useful that the system promotes it to the shared foundation for everyone to use. Conversely, if a general rule turns out to be harmful for a specific type of user, the system replaces it with a tailored alternative for that person. This dynamic adjustment ensures that the memory system remains both efficient and deeply personalized.

The researchers tested this method against several existing systems using a variety of challenging benchmarks that simulate long, complex conversations with different types of users. In these tests, the new framework consistently outperformed the others, especially in situations involving very long histories or noisy, distracting information. In one specific test involving 128,000 words of conversation history, the new system achieved an accuracy score of 62.01, significantly higher than the next best method, which scored 55.37. The improvement was even more pronounced when the system had to handle users with very different personalities. For example, in a test with a finance enthusiast, the personalized layer helped the agent remember specific investment details that a generic system would have summarized away, leading to an 18.6 percentage point improvement in performance for that user. Meanwhile, for users whose needs were already well-covered by the general rules, the system avoided adding unnecessary complexity, preventing the performance from dropping.

The study also revealed that the system is robust and adaptable. The rules it learned were not tied to a specific computer model but could be transferred to different underlying technologies, suggesting that the principles it discovered are fundamental to how memory should work. Furthermore, the system managed to keep its memory size under control even as the conversations grew longer, proving that it could filter out the noise effectively without losing the signal. The researchers found that the most significant gains came from the ability to distinguish between what is universally true for all users and what is specific to an individual. By separating these two concerns and allowing them to co-evolve, the system avoids the pitfalls of being too rigid or too chaotic.

While the results are promising, the researchers note that their work is primarily focused on English-language conversations and specific types of tasks. They acknowledge that as these systems interact with users over years rather than just days, the fundamental nature of a person's personality might shift in ways that require even more sophisticated updates. Nevertheless, this work provides a clear path forward for building agents that can truly remember who you are, adapting their memory to fit your unique story rather than forcing your story to fit a pre-made template. The findings suggest that the future of personalized AI lies not in memorizing everything, but in learning exactly what to share and what to keep for yourself.

Technical Summary: HiPS (Hierarchical Personalized Strategy)

1. Problem Statement

Memory-augmented agents aim to maintain compact user profiles to enable personalized and consistent responses over extended conversations without processing the entire dialogue history. The core challenge lies in the memory management strategy: determining at each step what information to retain, compress, or discard.

Existing approaches suffer from two primary limitations:

Static, One-Size-Fits-All Strategies: Most methods employ fixed, pre-defined rules (manually crafted or learned before training) that apply identically to all users. This fails to account for the fact that optimal memory decisions are inherently user-specific and evolve dynamically alongside policy optimization.
Lack of Adaptability: While recent reinforcement learning (RL) approaches treat memory operations as learnable actions, they often optimize for population-averaged rewards. This causes signals for niche user behaviors to be overwhelmed by majority patterns, resulting in a compromise that fails to capture individual nuances. Furthermore, methods utilizing explicit strategies often freeze these rules during training, causing a misalignment between the prescribed strategy and actual rollout trajectories as the policy evolves.

The paper posits that the boundary between shared universal rules and user-specific adaptations must be discovered empirically rather than predefined, and that strategies must co-evolve with the policy.

2. Methodology: HiPS Framework

The authors propose HiPS (Hierarchical Personalized Strategy), a framework that decouples memory management into a globally shared foundation ( $S_u$ ) and a user-specific adaptive tier ( $\Delta_p$ ). The framework operates via a continuous co-evolution loop where strategy discovery and policy optimization are interleaved.

Core Components

1. Universal Strategy Distillation (USD)

Goal: Evolve shared rules ( $S_u$ ) applicable to the entire population.
Mechanism: USD samples top-k and bottom-k trajectories across diverse personas. An LLM meta-optimizer compares these contrastive sets to generate structured diffs (Validate, Hypothesize, Revise) rather than free-form summaries.
Evidence Tracking: To prevent noise, rules are assigned evidence levels (Tentative, Supported, Established). Rules are promoted or demoted based on validation signals and a Predictive Gain (PG) metric, which measures the absolute difference in task success rates between trajectories complying with a rule and those that do not.

2. Persona Delta Distillation (PDD)

Goal: Generate adaptive rules ( $\Delta_p$ ) for users whose behaviors diverge from the population norm.
Divergence Gating: To avoid computational waste and overfitting, PDD is only triggered for users where the aggregate discrepancy in rule efficacy ( $D_p$ ) exceeds a threshold ( $\theta_{div}$ ). Users below this threshold rely solely on $S_u$ .
Behavior-Conditioned Distillation: For divergent users, an LLM formulates management behaviors that strictly differ from $S_u$ , constrained to be management-oriented (prescribing actions) rather than factual assertions, ensuring generalization to unseen users.

3. Cross-Level Rule Flow

Goal: Dynamically calibrate the boundary between $S_u$ and $\Delta_p$ .
Generalization ( $\Delta_p \to S_u$ ): If a personalized rule proves beneficial across a sufficient fraction of personas ( $\ge \theta_{flow}$ ), it is promoted to the universal level and pruned from individual deltas.
Specialization ( $S_u \to \Delta_p$ ): If a universal rule is revised or demoted by USD, users who previously relied on it are provided with persona-specific replacements to prevent performance degradation.

4. Strategy Injection and Co-Evolution

Submodular Selection: The active strategy $S_p = S_u \cup \Delta_p$ is constrained by a token budget $B$ . A greedy submodular selection algorithm maximizes coverage (based on PG) while penalizing redundancy.
Reward Separation: To prevent self-reinforcing confirmation bias, the framework separates rewards:
- $R_{ans}$ (Task Correctness): Used exclusively for updating the distillation buffer and ranking trajectories for strategy refinement.
- $R_{follow}$ (Strategy Adherence): Used only for GRPO (Group Relative Policy Optimization) policy updates to align the model with the active strategy.
- This separation ensures that strategy evolution is anchored strictly to objective task outcomes, not merely compliance with potentially flawed rules.

3. Key Contributions

Problem Formulation: The paper formulates strategy personalization as a problem where the boundary between shared and user-specific rules must be empirically discovered and co-evolved, rather than statically predefined.
HiPS Framework: Introduction of an RL framework that co-evolves a universal strategy and user-specific adaptive rules via evidence-based distillation (USD/PDD) and cross-level rule flow.
Empirical Findings: Experiments reveal a "component importance flip": universal rules dominate in-domain performance, while adaptive mechanisms (PDD and Flow) are critical for out-of-domain generalization.

4. Experimental Results

The framework was evaluated on four benchmarks: PersonaMem (preference evolution), PrefEval (explicit/implicit queries), PersonaBench (personalized retrieval/QA), and PERMA (event-driven preference).

Overall Performance: HiPS consistently outperformed baselines (including Long Context, RAG, retrieval-based memory banks like Mem0/A-Mem, and RL-based agents like MemAgent/MemSkill) across all 12 evaluation settings.
- On PersonaMem (128K), HiPS achieved 62.01% accuracy, significantly surpassing the next best baseline (MemSkill at 55.37%) and the Long Context baseline (20.74%).
- On PrefEval, HiPS achieved 89.20% (Explicit) and 69.40% (Implicit), leading in all domain categories.
Ablation Studies:
- Removing USD caused the largest drops in in-domain tasks, confirming the necessity of universal rules.
- Removing Cross-Level Flow caused the most severe degradation in out-of-domain settings, reducing C-S accuracy from 66.95 to 45.39, resulting in an average PERMA decrease of 19.4 points, highlighting the importance of dynamic rule migration.
- Removing PDD resulted in moderate but consistent drops, validating the need for selective personalization.
Scaling and Transfer:
- Scaling: As dialogue length increased to 128K tokens, HiPS maintained bounded memory growth (~1,900 tokens) and linear time scaling, demonstrating efficiency in ultra-long contexts.
- Cross-Model Transfer: Strategies distilled using one backbone (e.g., Qwen2.5-7B) transferred effectively to others (GPT-4o-mini, Gemini 2.5 flash, GPT-5), confirming the model-agnostic nature of the learned strategies.

5. Significance and Claims

The paper claims that HiPS establishes a continuous co-evolution loop between multi-tiered strategies and active policies. By decoupling memory management into a universal baseline and a user-specific delta, the framework addresses the rigidity of static pipelines and the "one-size-fits-all" compromise of population-averaged RL.

The authors emphasize that their approach enables truly personalized long-term memory systems where:

Foundational rules are universally beneficial and shared.
Niche behaviors receive tailored handling without redundant computation for all users.
The system adapts dynamically as the policy evolves, ensuring strategies remain aligned with actual rollout trajectories.

The work is presented as a step toward robust agents capable of handling extended, personalized interactions without the token bloat or context limitations of direct history retention. The authors modestly note limitations regarding multilingual applicability and extremely long-term (multi-year) personality drift, suggesting these as avenues for future research.