🤖 AI

SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent

The paper proposes State-Adaptive Memory (SAM), a standalone framework that enhances long-horizon agentic reasoning by consolidating interaction histories into compact, intent-driven memory cues that allow agents to dynamically reconstruct distant information without retraining the underlying model, achieving superior performance across diverse benchmarks.

Original authors: Yuyang Hu, Hongjin Qian, Shuting Wang, Jiongnan Liu, Ziliang Zhao, Jiejun Tan, Zheng Liu, Zhicheng Dou

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuyang Hu, Hongjin Qian, Shuting Wang, Jiongnan Liu, Ziliang Zhao, Jiejun Tan, Zheng Liu, Zhicheng Dou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Too Long to Read" Notebook

Imagine you are a detective trying to solve a very complex mystery that takes days to crack. Every time you find a clue, talk to a witness, or check a document, you write it down in a giant notebook.

As the days go by, your notebook becomes huge—hundreds of pages long. When you need to make a decision today, you have to flip through all those pages to find the one tiny clue you found three days ago.

The current problem: Most AI agents (computer programs that act like humans) try to solve this by either:

Throwing away old pages: They delete the beginning of the notebook to make room for new ones. But sometimes, the answer was in the part they threw away!
Writing a summary: They try to condense the whole history into a short paragraph. But summaries often miss the tiny, specific details needed to solve the puzzle later.

The Solution: SAM (State-Adaptive Memory)

The authors propose a new system called SAM. Think of SAM not as a notebook, but as a smart filing cabinet with a magical index card system.

Here is how it works, broken down into three simple steps:

1. The "Filing" (Consolidation)

Instead of keeping the raw, messy pages of your conversation in your immediate workspace, SAM takes a chunk of your recent history (say, the last 50 pages) and files it away in a storage cabinet.

The Raw Pages: The actual details are kept safe in the cabinet, untouched.
The Index Card: SAM writes a tiny, high-quality "index card" (called a memory cue) that summarizes what happened in those 50 pages. It highlights the important stuff: "We found a suspect," "We ruled out the butler," or "We are stuck on this clue."
The Result: Your immediate workspace stays clean and light, but you still have a list of index cards telling you what's in the cabinet.

2. The "Search" (Intent-Driven Recall)

This is the magic part. In old systems, if you needed information, you might just grab the most recent pages or read the whole summary.
With SAM, the agent asks itself: "What do I need right now to solve this specific problem?"

If the agent is stuck on a clue about a "red car," it looks at its index cards.
It picks the specific card that mentions the red car.
It asks the filing cabinet: "Bring me the exact pages related to that red car."
The system retrieves only those specific pages, re-reads them, and gives the agent a fresh, focused summary of just that information.

The Analogy: Imagine you are cooking a complex meal. Instead of reading the entire 500-page cookbook every time you need to know how to chop an onion, you have a set of index cards. When you need to chop an onion, you pull the "Onion" card, which tells you exactly which pages to flip to in the big book to get the recipe. You don't read the whole book; you only read what you need.

3. The "Training" (Learning to be a Better Librarian)

The paper also explains how they taught this system to be smart. They didn't just tell the AI to "summarize." They trained it in two ways:

The Expert Teacher: They used super-smart AI models to show the system how to write the best possible index cards and how to find the right pages.
The "Try and Learn" Game: They let the system play the detective game thousands of times. If the system picked the wrong card or missed a clue, it got a "bad score." If it picked the right card and solved the mystery, it got a "good score." Over time, it learned exactly how to manage its memory to win the game.

Why This Matters (The Results)

The authors tested SAM on four difficult "long-horizon" tasks (like searching the web for complex facts or solving scientific puzzles).

The Outcome: SAM consistently beat other methods.
The Reason: It didn't just throw away old info, and it didn't just summarize it. It kept the raw data safe and created a smart way to retrieve exactly what was needed based on the current situation.
The Analogy: It's like the difference between a student who memorizes a textbook (and forgets details) versus a student who has a perfect library system where they can instantly find the exact paragraph they need to answer a specific question.

Summary

SAM is a new way for AI to handle long conversations. Instead of trying to remember everything at once or forgetting the past, it organizes history into a filing system. It keeps the raw details safe and uses smart index cards to pull up only the specific information needed for the current decision. This allows the AI to solve much harder, longer problems without getting confused or losing track of the clues.

Technical Summary: State-Adaptive Memory (SAM) for Long-Horizon Reasoning Agents

Problem Statement

Long-horizon agentic reasoning requires Large Language Models (LLMs) to operate over extended interaction histories containing interleaved thoughts, tool calls, observations, and partial conclusions. The primary challenge is not merely the increasing length of these histories, but the fact that information critical for a current decision may be scattered across distant steps and only become relevant later in the trajectory.

Existing context management strategies—such as truncating history, compressing it into summaries, or retrieving selected parts—often fail in these scenarios. These methods typically rely on recency or fixed-size windows, which are insufficient when useful information is distributed across non-adjacent steps and its importance is only apparent as the task unfolds. The paper argues that the core difficulty lies in organizing past trajectories so that the right information remains accessible to the agent's current decision state, rather than simply keeping the entire past in view or discarding it based on a static budget.

Methodology: State-Adaptive Memory (SAM)

The authors propose State-Adaptive Memory (SAM), a standalone framework designed to treat long-horizon reasoning as a problem of state-adaptive memory. SAM decouples the agent's reasoning backbone from its memory management, allowing the memory module to be optimized independently without retraining the underlying agent.

Core Architecture

SAM maintains two coupled views of the interaction history:

Compact Memory Cues: Lightweight summaries that remain visible in the agent's active context. These act as persistent handles or entry points to past progress.
Raw Trajectory Pages: Detailed interaction records stored externally, preserving the full fidelity of the original trajectory.

The framework operates through a Page-based Episodic Consolidation and Intent-driven Episodic Recall process:

Consolidation (Write Path): As the agent's live context reaches a predefined token budget, the recent trajectory is partitioned into contiguous "pages." A memory model generates a compact cue ( $m_k$ ) for each page, capturing continuation-relevant contributions (e.g., what was established, what was ruled out, what remains unresolved). The raw page is then removed from the active context and stored externally, while the cue is retained in a memory bank.
Recall (Read Path): When the agent needs to revisit the past, it issues a recall request with a specific intent ( $q_t$ ). The agent selects a subset of relevant cues based on its current state. The memory model then reconstructs the decision-relevant information from the corresponding raw pages, conditioned on the recall intent. This allows the agent to access temporally distant information on demand without replaying the entire history verbatim.

Optimization Strategy

SAM is optimized as an independent capability through a two-stage process, keeping the agent backbone frozen:

Expert-Guided Supervised Fine-Tuning (SFT): The memory model (initialized as Qwen3.5-9B) is trained on traces from leading LLMs (e.g., Claude-4.5-Opus, GPT-5.4) that successfully solved tasks. This provides paired targets for both consolidation (generating cues) and intent-driven recall (reconstructing information).
OAT-GRPO (Oracle-Anchored Tree GRPO): To align the memory model with trajectory-level utility, the authors introduce a specialized reinforcement learning objective.
- Tree-Structured Outcome Reward: Unlike standard GRPO which assigns a single reward to a whole rollout, OAT-GRPO structures rollouts as a memory-call tree. It branches at every memory action, allowing credit assignment to be localized to specific memory outputs based on the final task outcome.
- Oracle-Anchored Recoverability Reward: To address the sparsity of outcome signals, an "oracle" reward is computed using a committee of frontier models. This reward evaluates how well a recalled summary covers the target space of useful information (relevance, coverage, consistency) relative to expert references, densifying the learning signal.

Key Contributions

The paper outlines three primary contributions:

Formulation: It reframes long-horizon context management as a state-adaptive memory problem, prioritizing demand-driven access to temporally distant information over simple recency-based compression.
Architecture: It introduces a cue-page memory architecture that decouples lightweight write-time consolidation from intent-conditioned read-time reconstruction over preserved raw trajectory pages.
Optimization Recipe: It develops a training pipeline for standalone memory models that combines expert-guided supervision with OAT-GRPO, a memory-action-level RL objective that assigns credit through memory-call trees and oracle-anchored recoverability rewards.

Experimental Results

SAM was evaluated on four long-horizon agent benchmarks: BrowseComp, BrowseComp-ZH, WideSearch, and HLE. The experiments utilized diverse agent backbones, including proprietary (GLM-4.7) and open-source (Qwen3.5-35B-A3B) models.

Performance: SAM consistently outperformed strong baselines, including heuristic context management strategies (discarding tools, keeping recent $k$ steps, rolling summaries) and existing open-source agent systems (WebThinker, WebSailor, ReSum, etc.).
Generalization: A single Qwen3.5-9B memory model, trained once, achieved the best scores across all backbone-benchmark combinations, demonstrating that the memory mechanism is robust and generalizable rather than dependent on specific backbone-benchmark couplings.
Ablation Studies:
- Both SFT and OAT-GRPO stages were found to be complementary; removing either resulted in performance drops.
- The gains were attributed primarily to intent-driven recall rather than consolidation alone. Variants using only summaries or recency-based retrieval performed significantly worse.
- SAM maintained its advantage even as interaction rounds increased (e.g., beyond 80 rounds), where summary-based methods began to lose useful state.

Significance and Claims

The paper claims that explicit memory modeling provides a simple and effective foundation for long-horizon agentic reasoning. By treating memory as an external, reusable capability that consolidates interaction into navigable entries while preserving raw data for reconstruction, SAM enables agents to recover information from arbitrary stages of their trajectory without retraining the underlying backbone.

The authors position SAM not as a replacement for the agent's reasoning policy, but as a modular component that transforms long-horizon history from a passive burden into a navigable memory space. The results suggest that the ability to access the right past information for the current decision state is a critical, yet often overlooked, factor in scaling agentic reasoning. The work is presented as a step toward modular memory systems for increasingly capable agents, with the code made available for further research.