Technical Summary: Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents

Problem: Behavioral State Decay

Long-horizon tasks, such as autonomous command-line execution and multi-step tool use, require agents to maintain decision-relevant state over extended trajectories. The authors identify a critical failure mode termed behavioral state decay. In this mode, information essential for future decisions—such as initial task requirements, environment facts, previous failed attempts, error diagnoses, and open subgoals—becomes inactive in the agent's decision-making process.

This decay occurs even when the information remains within the model's context window or transcript. The issue is not merely the inability to store or retrieve data, but the failure to determine when remembered information should actively influence the next action. Existing memory systems often focus on passive retrieval or summarization, which can lead to agents repeating mistakes, ignoring prior discoveries, or drifting from constraints. The paper argues that effective memory for long-horizon agents must be an active intervention mechanism that decides not only what to remember but whether and when to intervene.

Methodology: Proactive Memory Intervention

The paper proposes a two-phase memory intervention architecture that decouples memory maintenance from action selection. The system consists of an unmodified action agent and a separate memory agent that runs alongside it.

Architecture Components

1. Structured Memory Bank ($B_t$): A compact, structured representation of execution state containing three components:

   • Status ($s_t$): A private field tracking progress, open issues, and risks, visible only to the memory agent.
   • Knowledge ($K_t$): Stable facts (e.g., task requirements, environment properties, file paths).
   • Procedural ($P_t$): Dynamic records of attempts, outcomes, failed commands, successful fixes, and diagnostic signals.

2. Two-Phase Memory Agent Workflow:

   • Phase 1 (Memory Management): The memory agent observes a sliding window of the recent trajectory and the current bank. It issues tool calls to update the bank (e.g., memory_save_knowledge, memory_save_procedural, memory_update_status, memory_delete). This phase externalizes execution state into a structured format without rewriting the trajectory.
   • Phase 2 (Intervention Selection): Based on the updated bank and recent trajectory, the agent decides whether to inject a concise, memory-grounded reminder into the next action-agent call or to remain silent (<no_intervention/>).
     • Intervention: A targeted reminder (e.g., a forgotten requirement, a relevant environment fact, or a diagnosis) injected as transient context.
     • Silence: An explicit decision to withhold memory if the current state is sufficient, avoiding token overhead and distraction.

3. Triggering: The memory agent is invoked at fixed intervals (e.g., every $N$ steps) to update the bank and decide on intervention, though the authors note that more selective triggers (e.g., upon errors) are possible.

Learning the Policy

While the architecture can be implemented with a prompted frontier model, the authors explore training an open-weight model to learn the intervention policy:

• Supervised Fine-Tuning (SFT): Distills trajectories from a prompted memory agent to teach the interface and basic memory management discipline.
• Reinforcement Learning (GRPO): Optimizes the intervention policy to maximize downstream task success, specifically calibrating when to intervene versus when to remain silent.

Key Contributions

1. Identification of Behavioral State Decay: The paper formalizes the failure mode where decision-relevant state loses its influence over time, distinguishing it from simple context window limitations.
2. Proactive Intervention Architecture: A novel two-phase system that treats memory as a selective intervention policy. It decouples state maintenance from action selection, allowing a separate agent to decide if and how to reactivate execution state.
3. Empirical Validation: Demonstrates that selective intervention outperforms passive memory exposure, always-on injection, and general advisor guidance.
4. Learnability of Intervention Policies: Provides preliminary evidence that the intervention policy can be distilled into an open-weight model (Qwen3.5-27B) using SFT and GRPO, achieving partial transfer to held-out benchmarks.

Experimental Results

The system was evaluated on Terminal-Bench 2.0 (autonomous command-line execution) and $\tau^2$-Bench (interactive tool use in airline, retail, and telecom domains).

• Performance Gains:

  • Terminal-Bench 2.0: Using Claude Opus 4.6 as the memory agent, the system improved pass@1 for the weaker Claude Sonnet 4.5 from 37.6% to 45.9% (+8.3 pp) and for the stronger Opus 4.6 from 43.5% to 45.9% (+2.4 pp).
  • $\tau^2$-Bench: For Sonnet 4.5, the task-weighted average improved from 55.0% to 61.8% (+6.8 pp). For Opus 4.6, it improved from 66.2% to 68.7% (+2.5 pp).
  • Gains were consistent across domains, with the largest improvements seen in airline and retail tasks where policy adherence is critical.

• Ablation Studies:

  • Full System vs. Passive Exposure: Simply exposing the full memory bank to the action agent (without selective intervention) yielded lower gains, confirming that selection is crucial.
  • Always-Inject vs. Selective Silence: Forcing a reminder at every step was competitive but less balanced than the selective approach, particularly in domains requiring silence to avoid distraction.
  • Advisor-Only vs. Memory-Grounded: An advisor model without a persistent memory bank was less stable and failed to ground guidance in long-term execution state.
  • Mem0 Comparison: A general persistent memory layer (Mem0) improved average scores but failed to match the full system's performance, highlighting the distinction between retrieving records and deciding when to intervene.

• Open-Weight Training: Training a Qwen3.5-27B memory agent on the SETA benchmark improved validation rewards. When transferred to Terminal-Bench 2.0, the trained agent improved the frozen action agent's pass@1 from 37.6% to 41.1%, demonstrating that the intervention policy is learnable.

Significance and Claims

The paper positions memory not as a passive storage layer but as an active control mechanism. The authors claim that the central challenge in long-horizon agents is not just retaining information, but calibrating the timing and necessity of its reactivation.

• Modest Claims on Training: The authors describe the training of the open-weight memory agent as an "early step" and "preliminary evidence." They acknowledge that the current training relies on distilling from a prompted frontier model and that the intervention calibration (knowing when to be silent) is imperfect, often requiring reinforcement learning to refine.
• Generalizability: The architecture is presented as "plug-and-play" with existing agent harnesses and unmodified action agents.
• Future Directions: The paper suggests future work in jointly training memory and action agents, learning dynamic invocation schedules, and distinguishing between verbatim reminders and task-specific abstractions.

In summary, the paper argues that selective intervention is the key to mitigating behavioral state decay, offering a robust, modular solution that improves long-horizon task performance across diverse benchmarks and agent capabilities.