💻 computer science

AutoMem: A Text-Gradient Recursive Self-Improvement Framework for Automated Memory Architectures Search

The paper proposes AutoMem, a text-gradient recursive self-improvement framework that automatically discovers task-adaptive memory architectures by combining experience-guided search with failure-guided module diagnosis, consistently outperforming human-designed baselines while improving accuracy-efficiency trade-offs.

Original authors: Lin Du, Jie Zhou, Yuxuan Cai, Kai Chen, Qin Chen, Xin Li, Bo Zhang, Wei Li, Liang He

Published 2026-08-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Lin Du, Jie Zhou, Yuxuan Cai, Kai Chen, Qin Chen, Xin Li, Bo Zhang, Wei Li, Liang He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Artificial intelligence has reached a point where large language models can act as agents, capable of planning, using tools, and navigating the web to solve complex problems. To do this effectively, these agents need a way to remember things, much like a human relies on long-term memory to recall past lessons, facts, and strategies. However, building this memory for a machine is not as simple as adding a single notebook. It involves a series of decisions: what information should be written down, how it should be stored, how the agent finds it when needed, and how old or useless entries are removed. For years, researchers have built these memory systems by hand, choosing one specific design and sticking with it for every task. But just as a carpenter would not use the same tool for every job, a one-size-fits-all memory system often fails when the task changes.

A team of researchers at East China Normal University and the Shanghai AI Laboratory has tackled this problem by treating memory not as a fixed invention, but as a puzzle to be solved for each specific job. They built a system called AUTOMEM, which automatically searches for the best possible memory design for a given task. Instead of a human engineer guessing which combination of storage and retrieval methods works best, this system tests thousands of different configurations, learns from its mistakes, and refines its choices until it finds a setup that outperforms even the most skilled human-designed systems. The result is a memory architecture that adapts to the specific needs of the problem, whether it is navigating a website, solving a multi-step reasoning challenge, or digging deep into search results.

The researchers began by breaking down the concept of agent memory into four distinct parts, similar to how a library is organized. First, there is the encoder, which decides what information is worth saving. Second is the store, which determines how that information is kept, such as in a simple list, a complex web of connections, or a structured database. Third is the retriever, the mechanism that finds the right piece of information when the agent needs it. Finally, there is the manager, which handles the lifecycle of the memory, deleting old or conflicting data to keep the system efficient. By mixing and matching different options for each of these four parts, the researchers created a vast space of possible memory designs. Their initial experiments revealed a crucial truth: no single design works best for everything. A memory system that excels at solving math problems might fail miserably at navigating a website, and the best combination for one computer model might be poor for another.

To navigate this vast space of possibilities without wasting time and computing power, the team developed a method that learns from failure. When the system tries a memory design and the agent fails to solve a task, the system does not just discard the attempt. Instead, it acts like a careful investigator, analyzing exactly where the breakdown occurred. Did the agent fail because it never wrote down the right clue? Was the information stored in a format that was impossible to find? Did it pull up the wrong memory? Or did it hold onto too much outdated information? The system pinpoints the specific part of the memory process that caused the error and translates that technical failure into plain language instructions. It tells the search process, "The problem was in how we stored the data; try a different format next time."

This process of diagnosis and adjustment happens in a loop. The system proposes a new memory design based on what it has learned from previous failures, tests it, and then analyzes the results again. Over just a few rounds of this self-improvement, the system converges on a highly effective architecture. In their tests, the researchers used three different challenging benchmarks: a set of complex reasoning questions, a series of deep web navigation tasks, and a collection of difficult search queries. They ran these tests using two different powerful computer models as the agents. In every case, the memory architecture discovered by the system was superior to the best human-designed alternatives. On the reasoning benchmark, the system improved accuracy by nearly four percentage points compared to the strongest fixed memory system. On the web navigation task, it improved accuracy by nearly four points as well, and on the deep search challenge, it gained one full percentage point.

Beyond just being more accurate, the system also proved to be more efficient. The memory designs it found often required fewer computing resources to run. For instance, on the reasoning benchmark, the system achieved its higher accuracy while using significantly fewer tokens, which is a measure of the computational cost of processing text. This suggests that the system did not just find a way to work harder, but a way to work smarter. It discovered that for some tasks, the best approach was to store information in a graph-like structure that links related facts together, while for others, a simple structured list was more effective. For web navigation, it found that a mix of text and structured data worked best, whereas for deep search, it favored a system that could easily rerank past solutions to find the most relevant ones.

The researchers also tested whether their method was truly learning or just getting lucky. They compared their guided search against a method that simply picked memory designs at random. The random method occasionally found a good design, but it was inconsistent and often wasted resources testing designs that were clearly inferior. The guided system, by contrast, improved steadily with every round of testing, finding a better solution in half the time and with half the computational cost of the random approach. This confirmed that the system's ability to diagnose failures and translate them into specific instructions was the key to its success. Without this feedback loop, the system struggled to improve, often getting stuck in designs that were costly but ineffective.

The study concludes that the future of artificial intelligence agents lies not in building a single, perfect memory system, but in creating systems that can adapt their memory to the task at hand. The researchers found that the best memory design depends heavily on the specific nature of the work the agent is doing and the computer model it is running on. A design that works for one agent on one type of problem can be a hindrance on another. By automating the search for these designs, the team has shown that it is possible to build agents that are not only smarter but also more efficient. The work suggests that as artificial intelligence becomes more integrated into complex real-world tasks, the ability to dynamically tune how an agent remembers and uses information will be just as important as the intelligence of the agent itself. The system does not replace human ingenuity but rather extends it, allowing computers to find the right tools for the job in a way that is too complex for a human to design manually.

Technical Summary: AUTOMEM

Problem Statement

Long-term memory is increasingly critical for Large Language Model (LLM) agents, enabling them to preserve and reuse information from past interactions. However, current memory systems are predominantly manually designed and fixed upon deployment. The authors identify a fundamental limitation: memory effectiveness is not determined by a single component (e.g., a retriever) but by the complex, coupled interactions between four distinct modules: Encode (what to write), Store (how to represent), Retrieve (how to select), and Manage (how to maintain/evolve).

A pilot study reveals that:

Task Specificity: No single memory architecture dominates across all tasks or backbone models; the optimal configuration varies significantly by dataset.
Module Coupling: Module effects are non-additive. A strong retriever may fail if the encoder produces low-quality memories, and a useful store format may be ineffective if the retrieval policy is mismatched.
Inefficiency of Blind Search: While strong architectures exist within the discrete search space, random search is prohibitively expensive and fails to convert failed trials into reusable optimization signals.

The core problem is defined as Task-Adaptive Memory Architecture Selection: finding a feasible architecture $a = (e, s, r, m)$ within a constrained search space that maximizes task accuracy and memory-induced gain while minimizing inference cost, given a limited evaluation budget.

Methodology: AUTOMEM

The authors propose AUTOMEM, a text-gradient recursive self-improvement framework designed to navigate the discrete, coupled memory architecture space efficiently. Instead of analytic gradients, AUTOMEM constructs a "textual gradient" from failed agent rollouts to guide the search. The framework operates in iterative rounds, comprising two primary components:

1. Experience-Guided Architecture Search (EGAS)

EGAS proposes candidate architectures by conditioning a proposer LLM on a structured prior derived from historical search trajectories. It utilizes three specific state representations:

Experience Ledger ( $L$ ): Stores reusable cross-round knowledge as tuples of (condition, recommended/discouraged edit, evidence, status). This prevents the search from revisiting refuted directions.
Observation Graph ( $G$ ): Maintains task-pattern-level statistics (e.g., $\mu(c, u)$ ) linking specific task conditions to module choices and their observed outcomes, serving as soft priors.
Pareto Front ( $P_t$ ): Tracks non-dominated candidates based on a multi-objective ranking of accuracy, memory-induced gain, and rollout cost.

2. Failure-Guided Module Diagnosis (FGMD)

FGMD converts failed agent rollouts into targeted textual feedback for the next search round. The process involves:

Scope Filtering: A rule-based gate filters out failures unrelated to memory (e.g., tool errors, timeouts, reasoning mistakes) to ensure only memory-attributable failures drive the search.
Module Attribution: Failures are classified into specific modes (e.g., extraction gap, retrieval miss, stale memory) and mapped to the responsible module (Encode, Store, Retrieve, or Manage). An LLM diagnostician refines these attributions for ambiguous cases.
Textual Gradient Synthesis: The system aggregates failure histograms and representative traces to generate a structured textual gradient $\delta_{t+1}$ . This gradient identifies the primary bottleneck, provides grounded evidence, and recommends specific architectural edits (e.g., switching from semantic to hybrid retrieval).
Differential Validation: Recommendations are validated against previous rounds; if an edit fails to improve performance, it is downgraded in the ledger, closing the loop between diagnosis and search.

Key Contributions

Systematization of Memory Design: The paper formalizes long-term memory design as a search over a factored space of four modules (Encode, Store, Retrieve, Manage), demonstrating that existing systems are merely fixed points within this shared space and that optimal configurations are task-dependent.
AUTOMEM Framework: The introduction of a recursive self-improvement framework that combines experience-guided search with failure-guided diagnosis. This approach enables efficient discovery of task-adaptive architectures under limited evaluation budgets by converting failed rollouts into directional textual signals.
Empirical Validation: Comprehensive experiments across three benchmarks (GAIA, WebWalkerQA, xBench-DeepSearch) and multiple LLM backbones (Qwen3.5, gpt-5 variants) showing that AUTOMEM discovers architectures superior to human-designed baselines.

Experimental Results

Evaluated on GAIA, WebWalkerQA, and xBench-DeepSearch, AUTOMEM demonstrated the following:

Performance Gains: AUTOMEM consistently outperformed the strongest human-designed memory baselines. Across six benchmark-backbone settings, it improved accuracy by an average of 2.8 points.
- On GAIA (Qwen3.5-122B-A10B), AUTOMEM achieved 71.5% accuracy vs. 67.8% for the best baseline (MemoryBank).
- On xBench-DeepSearch, it reached 46.0% vs. 45.0% SOTA.
- On WebWalkerQA, it achieved 72.5% vs. 68.9% for the best baseline.
Efficiency: AUTOMEM achieved a favorable accuracy-efficiency trade-off. Under Qwen3.5-122B-A10B, it reduced token costs by 14.3% compared to the strongest accuracy baselines while maintaining higher accuracy.
Search Efficiency: AUTOMEM reached 71.5% accuracy in GAIA within 5 evolution rounds, surpassing the best result of 10 random trials (69.7%). It achieved this with roughly 55% of the token budget required for random search.
Ablation: Removing either FGMD or EGAS significantly degraded performance. Without FGMD, accuracy dropped to 64.2% (below the no-memory baseline in some contexts) with doubled token costs. Without EGAS, accuracy fell to 57.9%, indicating the necessity of accumulated search experience.

Discovered Architectures

The paper highlights that AUTOMEM converges to distinct architectures for different task distributions:

GAIA (Multi-step reasoning): Favors Tip+Workflow+Trajectory encoding, Graph storage, and Contrastive retrieval to organize entities and distinguish successful from failed tool traces.
WebWalkerQA (Navigation): Favors Insight+Workflow+Shortcut encoding, JSON storage, and Hybrid retrieval to preserve exact page facts and validate tool use.
xBench-DeepSearch (Deep search): Favors Tip+Insight+Trajectory encoding, JSON storage, and Case-based reranking to reuse solved cases effectively.

Significance and Claims

The paper claims that memory for LLM agents should not be treated as a fixed, hand-crafted mechanism but as an adaptive architecture that must be optimized according to the target task distribution. By treating memory design as a search problem and utilizing text-gradient feedback, AUTOMEM demonstrates that automated, task-adaptive search can discover architectures that outperform human intuition while improving the trade-off between accuracy and computational cost. The authors position this work as a first step toward task-adaptive memory search, noting that future work could extend this to sample-level adaptation and open-ended memory module generation.