💬 NLP

MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

MemSkill introduces a self-evolving framework for LLM agents that replaces static, hand-designed memory operations with learnable and adaptable skills, utilizing a closed-loop system of controllers, executors, and designers to dynamically refine memory management strategies and improve task performance across diverse benchmarks.

Original authors: Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, Wenya Wang

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, Wenya Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Rigid Librarian"

Imagine you have a personal assistant (an AI agent) who talks to you for hours, days, or even weeks. To be helpful, this assistant needs to remember things you said earlier.

Currently, most AI assistants use a fixed set of rules to decide what to remember. Think of this like a librarian who is only allowed to use three specific stamps: "Save," "Delete," or "Ignore."

If you tell the librarian, "I hate broccoli," they stamp it "Save."
If you say, "Actually, I love broccoli now," they have to manually find the old note and stamp it "Delete."

The problem is that these rules are hard-coded by humans. They don't change based on the situation. If the conversation gets long, complex, or weird, the librarian gets confused because their stamps don't fit the new scenario. They are rigid, slow, and often miss the point.

The Solution: MemSkill (The "Shape-Shifting Toolkit")

The authors of this paper propose MemSkill, a new way for AI to manage its memory. Instead of using a fixed set of stamps, MemSkill gives the AI a toolbox of "skills" that it can learn, choose from, and even improve over time.

Think of it like a Swiss Army Knife that can grow new blades.

1. The Skills (The Blades)

Instead of just "Save" or "Delete," MemSkill has specific, reusable instructions called skills.

Skill A: "Capture the time and date of events."
Skill B: "Track where objects are located."
Skill C: "Notice when a person changes their mind."

These skills are like recipes. They tell the AI: "When you see a date, write it down like this. When you see a location, write it down like that."

2. The Controller (The Foreman)

When the AI is talking to you, a "Controller" acts like a construction foreman.

It looks at what you just said.
It glances at the Skill Toolbox.
It picks the top 3 or 5 skills that are most useful for this specific moment.
It hands those skills to the AI to process the information.

Analogy: If you are talking about a cooking recipe, the Foreman picks the "List Ingredients" and "Note Temperatures" skills. If you switch to talking about a movie, the Foreman swaps those out for "Remember Character Names" and "Track Plot Twists."

3. The Designer (The Coach)

This is the most exciting part. Most systems stop learning once they are built. MemSkill has a Designer that acts like a sports coach.

The AI tries to answer questions using its current skills.
Sometimes, it fails. Maybe it forgot a crucial detail because it didn't have a skill for that specific type of information.
The Designer looks at these failures (the "hard cases").
It asks an advanced AI: "Why did we fail? Do we need a new skill? Do we need to tweak an old one?"
The Designer then writes new skills or fixes old ones and adds them to the toolbox.

Analogy: Imagine a soccer team keeps losing because they don't know how to defend against a specific type of kick. The Coach (Designer) watches the game, realizes the gap, and invents a new defensive drill (a new skill) to teach the team. The team gets better every time they play.

How It Works in Practice

The paper describes a closed loop:

Learn to Use: The AI practices picking the right skills to remember things.
Evolve the Tools: When the AI fails, the Designer fixes the toolbox, adding new skills or refining old ones.
Repeat: The AI starts over with the improved toolbox, getting smarter and more adaptable.

What the Paper Found (The Results)

The researchers tested MemSkill on four different challenges:

Long Conversations: Remembering details from very long chats (like a 10-hour dialogue).
Complex Questions: Answering questions that require connecting dots from different parts of a story (HotpotQA).
Robot Tasks: Helping a virtual robot navigate a house and find objects (ALFWorld).

The Results:

MemSkill beat all the other methods (the "rigid librarians") in almost every test.
It was better at generalizing: When they trained it on one type of task (like chatting) and tested it on a totally different one (like finding objects in a house), it still worked well. This is because the skills it learned were flexible, not just memorized answers.
It was more efficient: By processing chunks of text at once using the right skills, it didn't need to call the AI as many times as the old methods, saving time and money.

Summary

MemSkill changes how AI remembers things. Instead of forcing the AI to follow a rigid, human-written rulebook, it gives the AI a flexible, evolving set of skills. The AI learns which skills to use for the job, and when it makes a mistake, a "Designer" helps it invent new skills to prevent that mistake from happening again. It turns memory management from a static list of rules into a self-improving, adaptive system.

Technical Summary: MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

1. Problem Statement

Current Large Language Model (LLM) agent memory systems predominantly rely on static, hand-designed operations (e.g., fixed add/update/delete/skip primitives) and heuristic modules to govern what information to store, revise, or prune. These approaches hard-code human priors regarding memory management, resulting in systems that are rigid under diverse interaction patterns and inefficient when processing long interaction histories. The authors argue that treating memory extraction as a fixed procedural workflow creates a bottleneck, making memory behaviors brittle under distribution shifts and unable to scale effectively as agent histories grow.

2. Methodology: MemSkill

MemSkill reframes memory operations as a learnable and evolvable set of memory skills. Instead of fixed procedures, the system treats memory construction as the outcome of applying a small set of generic, reusable, structured behaviors (skills) that specify when and how interaction traces should be transformed into memory.

The architecture consists of three core components operating within a closed-loop optimization procedure:

A. The Skill Bank

A shared repository of structured memory skills. Each skill $s$ contains:

A short description for representation and selection.
A detailed content specification instructing the executor on extraction or revision (including purpose, usage conditions, application steps, and constraints).
Initialization: The system starts with a minimal set of four canonical primitives: INSERT, UPDATE, DELETE, and SKIP.

B. Learning to Use Skills (Controller & Executor)

Controller: A neural network that learns to select a small, context-relevant subset of skills (Top-K) from the Skill Bank for a given text span. It conditions its selection on the current text span and retrieved existing memories. The controller uses a shared scorer to handle a variable-size skill bank, computing semantic distances between the state and skill representations.
Executor: An LLM-based module that takes the selected skills and the current context to generate structured memory updates in a single pass. This allows for span-level processing rather than turn-by-turn processing, reducing redundant LLM calls.
Optimization: The controller is trained via Reinforcement Learning (PPO), using downstream task performance (e.g., F1 score, success rate) as the reward signal to optimize skill selection policies.

C. Skill Evolution (Designer)

To move beyond a fixed skill set, MemSkill introduces a Designer that periodically evolves the Skill Bank:

Hard-Case Buffer: During training, the system logs challenging cases where the current skill selection yields incorrect or incomplete memories.
Case Selection: The Designer clusters these hard cases and selects representative failures based on a difficulty score (low reward + high failure count).
Two-Stage Evolution: Using an LLM, the Designer analyzes failure patterns to:
- Refine existing skills (updating instructions to address specific gaps).
- Propose entirely new skills to capture missing memory behaviors.
Stability Mechanisms: The system maintains snapshots of the best-performing Skill Bank. If an evolution step degrades performance, it rolls back. Additionally, a brief exploration phase biases the controller toward newly introduced skills to facilitate their adoption.

3. Key Contributions

MemSkill Framework: A novel agent memory method that represents memory operations as an evolving skill bank, transforming memory construction from a fixed pipeline into an adaptive, skill-conditioned generation process.
Closed-Loop Optimization: A training recipe combining reinforcement learning for skill selection with LLM-guided skill evolution from hard cases, enabling the continual refinement and growth of the memory skill set.
Empirical Validation: Comprehensive evaluation demonstrating that MemSkill improves task performance and generalizes across settings without requiring task-specific memory engineering.

4. Experimental Results

The authors evaluated MemSkill on four benchmarks: LoCoMo and LongMemEval (conversational memory), HotpotQA (distribution shift/long-context reasoning), and ALFWorld (embodied interactive tasks).

Performance: MemSkill consistently outperformed strong baselines (including MemoryBank, A-MEM, MemoryOS, and Mem0) across all datasets. On conversational benchmarks, it achieved the highest LLM-judge scores; on ALFWorld, it achieved the highest success rates and required fewer interaction steps.
Generalization:
- Cross-Model: Skills learned on LLaMA-3.3-70B transferred effectively to Qwen3-Next-80B without retraining, outperforming baselines on both models.
- Cross-Dataset: Skills learned on LoCoMo transferred to LongMemEval and HotpotQA (a document-based QA task with different interaction formats), demonstrating robustness to distribution shifts.
Efficiency: By processing at the span level rather than turn-by-turn, MemSkill significantly reduced the number of LLM calls and input/output tokens while maintaining or improving memory quality compared to turn-level baselines.
Ablation Studies: Removing the Controller (random selection) or the Designer (fixed primitives) resulted in significant performance drops, confirming that both targeted skill selection and skill evolution are critical.

5. Significance and Claims

The paper claims that MemSkill offers a pathway toward self-evolving memory management systems for LLM agents. By decoupling memory behaviors from human priors and allowing them to be learned and refined from interaction data, the system can adapt to diverse domains (dialogue, embodied tasks, document QA) and evolving task demands.

The authors emphasize that their approach addresses the rigidity of current memory systems, which often fail under distribution shifts or long histories. The "closed-loop" nature of MemSkill, where the system not only learns how to use memory but also what memory operations are necessary, represents a step toward agents that can autonomously improve their own memory construction and maintenance mechanisms over time. The work is presented as a research contribution to benchmarked settings, with practical deployment considerations (privacy, consent) noted as orthogonal future work.