Technical Summary: Mitigating Conversational Inertia in Multi-Turn Agents

Problem Statement

While Large Language Models (LLMs) excel as few-shot learners when provided with high-quality demonstrations, this capability becomes a liability in multi-turn agent scenarios. The paper identifies a phenomenon termed conversational inertia, where models erroneously mimic their own previous responses as few-shot examples. Through attention analysis, the authors observe that models exhibit strong diagonal attention to previous assistant outputs; specifically, the $i$-th token in a current response disproportionately attends to tokens around the $i$-th position in prior assistant responses.

This attention pattern creates an imitation bias that constrains exploration. As the conversation lengthens, the model's attention to system prompts diminishes, causing it to rely more heavily on interaction history rather than task-specific instructions. This leads to a tension: while longer contexts provide more environmental feedback for exploitation, they simultaneously amplify conversational inertia, which undermines exploration and causes performance degradation even when the context remains within the model's capacity.

Methodology

The proposed framework addresses this issue through two primary components: Context Preference Learning (CPL) and Clip Context Management.

1. Context Preference Learning (CPL)

The core insight driving CPL is that for identical states, actions generated with longer input contexts exhibit stronger conversational inertia than those generated with shorter contexts. This allows for the construction of preference pairs without requiring environment rewards or expert demonstrations.

• Data Construction: For a given state, the system generates two actions: $a_{long}$ (using the full context history) and $a_{short}$ (using only recent context). The $a_{long}$ action is executed to obtain the next observation, which synchronously updates both trajectories.
• Training Objective: The model is fine-tuned using Direct Preference Optimization (DPO). The training input context ($C_{short}$) is kept consistent for both the "chosen" ($a_{short}$) and "rejected" ($a_{long}$) actions. This teaches the model to prefer decisions made with shorter contexts, effectively penalizing the inertia associated with accumulated history.
• Efficiency: The method uses LoRA (Low-Rank Adaptation) to fine-tune only 0.4% of the model's parameters.

2. Clip Context Management

To manage inertia at inference time, the authors propose Clip Context, a strategy that periodically clears interaction history.

• Mechanism: Unlike a standard sliding window (Window Context) that maintains a fixed number of recent rounds, Clip Context allows the context to grow up to a threshold $H$ (e.g., 12 rounds) and then resets to retain only the most recent $L$ rounds (e.g., 1 round).
• Advantages:
  • Inertia Mitigation: Periodic clearing breaks the accumulation of inertial patterns and prevents error propagation.
  • Computational Efficiency: Unlike sliding windows which shift boundaries and invalidate KV caches, Clip Context allows for effective KV cache reuse within a clearing cycle, offering a theoretical $W \times$ speedup in prefill computation compared to window methods.
  • Exploration-Exploitation Balance: It alternates between short contexts (encouraging exploration) and longer contexts (leveraging history for exploitation).

Key Contributions

1. Identification of Conversational Inertia: The paper formally identifies and analyzes conversational inertia as a distinct failure mode in multi-turn agents, characterized by strong diagonal attention to previous assistant outputs that leads to imitation bias and constrained exploration.
2. Context Preference Learning (CPL): A novel training method that calibrates model preferences to favor low-inertia responses. It constructs preference pairs using long-short context differentials, requiring no ground-truth environment rewards.
3. Clip Context Strategy: A context management approach that achieves a superior balance between exploration and exploitation. It outperforms standard windowing and summarization methods while offering significant computational speedups via KV cache optimization.

Experimental Results

The framework was evaluated across eight diverse agentic environments (including Maze, ALFWorld, WebShop, TextCraft, 2048, FrozenLake, Hangman, RushHour) and one deep research scenario (BrowseComp).

• Performance Gains: Across all tested models (Qwen3-8B, Llama3.1-8B, GPT-4o-mini), the combination of CPL and Clip Context yielded significant improvements. For instance, Qwen3-8B with CPL and Clip Context achieved a 72.5% average score, a 7.6% improvement over the base model with Window context.
• Inertia Reduction: Attention analysis confirmed that Clip Context reduced diagonal attention by 7%, while CPL reduced it by over 10% across all context management strategies. The reduction was particularly pronounced in Long Context configurations (14.5% reduction), indicating CPL effectively mitigates high-inertia baselines.
• Comparison with Summarization: The authors found that the performance gains in summarization-based methods were largely driven by the reduced inertia from context clipping rather than the quality of the generated summaries. In deep research tasks, Clip Context (Clip-12to6) outperformed Window methods and matched summarization methods while avoiding the accuracy drops associated with summary-induced over-confidence.
• General Capabilities: Fine-tuning with CPL preserved the models' general reasoning and knowledge capabilities, as evidenced by stable performance on GPQA-Diamond and MMLU-Redux benchmarks.

Significance and Claims

The paper claims that conversational inertia is a previously unrecognized contributor to multi-turn agent performance degradation. By addressing the fundamental mechanism of self-imitation bias through preference calibration and inference-time context control, the proposed framework enables more reliable autonomous agents.

The authors emphasize that their approach is distinct from existing context management strategies (sliding windows, summarization, retrieval) which often treat performance degradation as a result of context length or information overload. Instead, this work targets the specific structural bias of diagonal attention. The framework is presented as a principled and efficient solution that improves agent performance while reducing computational overhead, suggesting that managing inertia is a critical dimension in multi-turn system design. The authors acknowledge limitations, noting that Clip Context involves an information-inertia trade-off and may struggle with tasks requiring context beyond the maximum retention window, but argue that inertia reduction and information preservation can be addressed as separate, combinable dimensions.