M$^2$: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight Retrieval

The paper proposes M$^2$, a training-free, dual-memory framework that enhances long-horizon web agents by combining dynamic trajectory summarization for internal state compression with offline insight retrieval for external guidance, achieving significant improvements in success rates and token efficiency across multiple benchmarks.

The Gist

Imagine you are trying to navigate a massive, chaotic library to find a very specific book. This is what an AI agent does when it tries to browse the web to complete a task (like "buy a specific laptop" or "find a flight").

The problem with current AI agents is that they have no memory and no experience. They try to remember everything they've seen since the moment they started.

The Problem: The "Hoarding" Agent

Imagine you are walking through the library. Every time you turn a corner, you take a photo of the entire room, write down every word on every sign, and paste it into a notebook.

• The Result: By step 10, your notebook is 500 pages long. You are so overwhelmed by all the photos of empty hallways, ads, and irrelevant signs that you forget what you were looking for. You get stuck, confused, and give up.
• In AI terms: This is called "Context Explosion." The AI tries to feed every screenshot and text click from the last 50 steps into its brain. It gets expensive (costs a lot of money/computer power) and the AI gets "lost in the middle" of all that noise.

The Solution: M2 (The "Smart Librarian" System)

The paper proposes a new system called M2 (Dual-Memory Augmentation). Instead of hoarding everything, M2 gives the AI two superpowers: a Personal Journal and a Mentor's Cheat Sheet.

1. Internal Memory: The "Personal Journal" (Trajectory Summarization)

Instead of pasting 50 photos of the library into the notebook, the AI is taught to write a one-sentence summary after every step.

• How it works:
  • Old Way: "I saw a red door, then a blue door, then a sign saying 'Exit', then a cat, then a poster..." (Too much detail).
  • M2 Way: "Step 1: I entered the main hall. Step 2: I turned left toward the technology section. Step 3: I am now looking at the laptop aisle."
• The Analogy: It's like playing a video game. Instead of recording the entire 10-hour gameplay video to remember where you are, you just write down in your journal: "I am at the Castle Gate, I have the key, and I need to find the dragon."
• The Benefit: The AI stays focused on the current state and the goal, ignoring the visual noise (ads, sidebars) that doesn't matter. This saves a massive amount of computer power.

2. External Memory: The "Mentor's Cheat Sheet" (Insight Retrieval)

Sometimes, even with a good journal, you get stuck because you don't know the tricks of the library. Maybe there's a hidden shortcut, or a specific way to ask the librarian that works better.

• How it works:
  • The system has a giant database of successful trips made by other AIs in the past.
  • When your AI gets stuck or starts a new task, it asks: "Has anyone else tried to find a laptop on this website before? What worked?"
  • It pulls out a "Cheat Sheet" with tips like: "Always click the 'Sort by Price' button first," or "If the search bar gives zero results, try removing the brand name."
• The Analogy: It's like having a wise old librarian whisper in your ear: "Hey, don't walk down the history aisle; the books you want are in the back. Also, if the door is locked, try the side handle, not the knob."
• The Benefit: The AI doesn't have to learn these tricks from scratch. It instantly knows how to avoid common traps and dead ends.

The Result: A Super-Efficient Agent

By combining these two tools, the M2 system creates an agent that is:

1. Lighter: It doesn't carry a heavy backpack of useless photos (saves 50-60% on computer costs).
2. Smarter: It knows the tricks of the trade and doesn't get confused by distractions.
3. Training-Free: The best part? You don't need to spend months teaching the AI new skills. You just give it the journal and the cheat sheet, and it works immediately.

In short: M2 turns a confused, overwhelmed tourist into a seasoned guide who knows exactly where they are, what they need to do next, and how to avoid the tourist traps.

Technical Summary

1. Problem Statement

Autonomous web agents powered by Multimodal Large Language Models (MLLMs) struggle with long-horizon tasks (complex navigation requiring many interaction steps). Current mainstream approaches rely on a "full-context" strategy, concatenating the entire history of HTML screenshots and interaction text into the prompt. This leads to two critical bottlenecks:

• Exorbitant Computational Cost: The accumulation of high-resolution screenshots and verbose text causes the context window to grow rapidly, imposing severe pressure on inference resources and token costs.
• Performance Degradation ("Lost-in-the-Middle"): Overly long and noisy contexts distract the model, burying critical task-relevant cues under redundant historical information, leading to reasoning errors and "context hallucinations."

Existing solutions often require heavy Supervised Fine-Tuning (SFT) or Reinforcement Learning (RL), which demand massive datasets, computational resources, and complex reward shaping. Others rely on multi-agent architectures that suffer from high communication overhead and deployment complexity.

2. Methodology: The M2 Framework

The authors propose M2, a training-free, memory-augmented framework designed to optimize context efficiency and decision-making robustness. M2 introduces a Dual-Tier Memory Mechanism that replaces the raw history with two specialized components:

A. Internal Memory (In-Mem): Dynamic Trajectory Summarization

Instead of retaining raw observations ($O_{t-k:t-1}$) and full interaction logs, the agent is instructed to perform self-summarization at every step.

• Mechanism: The agent generates a concise textual abstraction ($s_t$) representing the triplet of (Thought, Action, Observation) for the current step.
• Process:
  1. The agent receives the current observation ($O_t$) and the existing internal memory chain ($M^{int}_{t} = \{s_1, ..., s_{t-1}\}$).
  2. It outputs a new summary $s_t$ and the next action.
  3. The raw history ($O_{t-1}, T_{t-1}, A_{t-1}$) is discarded from the active context window.
• Benefit: This transforms the context from a linearly growing list of raw data into a sub-linear chain of semantic abstractions, drastically reducing token consumption while maintaining a coherent state history.

B. External Memory (Ex-Mem): Insight Retrieval Augmentation

To provide global strategic foresight and prevent recurring errors, M2 utilizes an offline Insight Bank.

• Construction: An "Abstractor" model distills High-Leverage Interaction Rules from a database of 55,000 successful trajectories generated by various frontier models. These rules focus on:
  • Search & Filter Strategy: Optimizing query construction.
  • Navigation Efficiency: Identifying "Anchor Elements" to avoid deep, irrelevant paths.
  • State Validation: Heuristics for verifying UI states and avoiding infinite loops.
• Retrieval: During inference, the agent encodes the current user query and retrieves the Top-$i$ most relevant insights from the bank using semantic similarity (Sentence Transformers).
• Injection: These insights are injected into the system prompt as "Defensive Hints," guiding the agent to avoid common pitfalls specific to the task type.

C. Inference Architecture

The final decision context ($C'_t$) at any step $t$ consists of:
$$C'_t = \{P'_{sys}, Q, M^{int}_t, M^{ext}, O_t\}$$
Where $P'_{sys}$ is the optimized prompt, $Q$ is the query, $M^{int}_t$ is the recursive summary chain, $M^{ext}$ is the retrieved insights, and $O_t$ is the current observation.

3. Key Contributions

1. Training-Free Dual-Memory Architecture: A lightweight framework that integrates recursive internal tracking with external guidance, eliminating the need for costly SFT/RL or complex multi-agent systems.
2. Intra-Trajectory Compression & Inter-Trajectory Retrieval:
   • Compresses execution history into concise summary chains (Internal Memory).
   • Retrieves cross-task expert insights from a distilled knowledge base (External Memory).
3. Scalable Efficacy: Demonstrates that local open-source models can match or exceed proprietary model performance through superior memory management, achieving high-fidelity navigation with sustainable computational costs.

4. Experimental Results

The framework was evaluated on WebVoyager and OnlineMind2Web using diverse backbones (Claude-3.7-Sonnet, Claude-Sonnet-4, and open-source Qwen3-VL-32B).

• Success Rate Improvements:
  • Qwen3-VL-32B: Achieved a 16.2% increase on WebVoyager and 19.6% on OnlineMind2Web. Notably, the augmented open-source model (74.0% accuracy) outperformed the vanilla proprietary Claude-3.7-Sonnet (72.0%).
  • Claude Models: Achieved accuracy gains of up to 12.5% (Claude-3.7) and 5.5% (Claude-Sonnet-4).
• Efficiency Gains:
  • Token Reduction: M2 reduced token consumption by 57% for Qwen3-VL-32B on WebVoyager (from 215.2k to 92.3k tokens per task) and up to 55% on OnlineMind2Web.
  • Latency: The retrieval of insights adds negligible overhead (~6ms per task).
• Ablation Studies: Confirmed that the combination of both memory types yields the best results. The internal memory is crucial for reducing context bloat, while external memory provides the strategic "defensive hints" that prevent specific failure modes.

5. Significance

The M2 framework addresses a fundamental limitation in current web agents: the trade-off between context length and reasoning quality. By decoupling task performance from context growth, M2 offers a scalable and sustainable path for deploying autonomous agents in real-world environments.

• Cost-Effectiveness: It enables the use of smaller, open-source models for complex tasks by optimizing their context usage, reducing reliance on expensive proprietary APIs.
• Generalizability: The "Insight Bank" approach proves that high-level UI interaction rules are transferable across different websites and models without retraining.
• Practicality: The training-free nature allows for immediate deployment and integration into existing agent pipelines without the need for massive data collection or model fine-tuning.

In conclusion, M2 demonstrates that better memory management (summarization + retrieval) is a more effective lever for improving long-horizon web navigation than simply scaling model parameters or training data.