Technical Summary: TOOLSELF

Problem Statement

Current Large Language Model (LLM) powered agentic systems face a fundamental tension between domain-specific performance and cross-task generalization. Existing systems typically rely on static configurations (fixed before execution) that define the agent's strategy, toolbox, context management, and reasoning style.

• Specialization vs. Generalization: Strong domain priors and compact tool spaces aid performance on specific tasks but weaken transferability. Conversely, general-purpose agents with broad toolspaces and task-agnostic workflows expand coverage but often suffer from diluted guidance and degraded long-horizon performance.
• Limitations of Existing Solutions: Prior approaches attempt to resolve this via:
  • Pre-execution optimization: Derives specialized configurations offline but cannot adapt at runtime.
  • Planner-worker orchestration: Decouples planning from execution, leading to information loss and fragmented credit assignment.
  • Configuration patching: Modifies isolated dimensions (e.g., context or tools) via external modules, resulting in fragmented optimization that fails to coordinate system-level changes.
• Core Challenge: These methods decouple adaptation from execution, preventing the agent from proactively updating its configuration based on real-time task progress and feedback.

Methodology: TOOLSELF

The authors propose TOOLSELF (Tool-driven Self-reconfiguration), a paradigm that unifies task execution and runtime self-reconfiguration within a single, optimizable policy.

1. Core Mechanism: Unified Action Space

TOOLSELF treats configuration not as a pre-fixed constant but as a dynamic, tool-updatable state. The execution agent $\pi_i$ operates under a configuration $C_i = (q_i, \sigma_i, T_i, K_i, m_i)$, where:

• $q_i$: Current sub-goal.
• $\sigma_i$: Execution strategy.
• $T_i$: Stage-specific toolbox (subset of global pool).
• $K_i$: Local knowledge and constraints.
• $m_i$: Context-management mode.

The agent is equipped with a standardized Reconfiguration Tool ($T_{reconfig}$). When the agent determines the current configuration is misaligned with task demands (e.g., strategy failure, missing knowledge, or completed sub-goal), it invokes $T_{reconfig}$. This tool:

1. Archives the current stage summary into a global history pool.
2. Returns a configuration-generation specification ($z_i$) that guides the same policy to synthesize the next configuration $C_{i+1}$.

This design ensures that configuration updates are native actions within the agent's action space, allowing for holistic updates to sub-goals, strategies, tools, and context management simultaneously.

2. Training Framework: Configuration-Aware Two-stage Training (CAT)

To internalize self-reconfiguration as a meta-capability, the authors introduce CAT, an end-to-end training framework:

• Stage I: Cold-Start Initialization (RFT): Uses Rejection Sampling Fine-tuning (RFT) on successful trajectories generated by a teacher model (Qwen3-235B). The policy learns to generate valid stage trajectories and configuration updates conditioned on successful outcomes.
• Stage II: Reinforcement Learning Refinement (KTO): Employs Kahneman-Tversky Optimization (KTO) with trajectory-level binary feedback. Since self-reconfiguration quality is only observable through downstream task success, credit assignment is propagated to all decisions (execution and reconfiguration) within a trajectory based on the final task outcome (Success/Failure).

Key Contributions

1. TOOLSELF Paradigm: A novel architecture that abstracts configuration updates as a standardized tool interface, unifying execution and adaptation within a single policy's action space. This enables joint optimization and emergent adaptability without manually injected guidance.
2. CAT Framework: A two-stage training approach (RFT + KTO) that teaches agents to perform cross-stage self-reconfiguration, optimizing configuration decisions explicitly for downstream task success.
3. Unified Optimization: By colocating adaptation and execution, the method eliminates information loss from cross-module relay and enables end-to-end credit assignment, fostering capabilities like long-horizon planning and self-correction.

Experimental Results

The authors evaluated TOOLSELF on five datasets across deep research (FRAMES, xbench), general AI assistance (GAIA, GAIA(WS)), and software engineering (SWE-Lite), using Qwen3-8B and Qwen3-14B models.

• Zero-Shot Performance: Even without CAT training, zero-shot TOOLSELF rivals task-specialized agents (e.g., WebSailor, SWE-Agent) while maintaining strong cross-task generalization.
• Training Gains: After CAT training, TOOLSELF achieves an average improvement of 28.8 points over static-configuration baselines.
  • On Qwen3-8B, the average accuracy rose from 16.5% (Vanilla Agent) to 45.3% (TOOLSELF + RFT + KTO).
  • On FRAMES (Qwen3-14B), it reached 61.5%, outperforming specialized baselines like WebSailor (42.0%) and Co-Sight (46.0%).
• Ablation Studies:
  • Integrated vs. Decoupled: The integrated design (unified policy) significantly outperforms a decoupled variant where a separate LLM generates configurations, confirming that fine-grained execution context is critical for effective reconfiguration.
  • Component Analysis: Knowledge transfer and sub-goal planning were identified as the most critical configuration dimensions for long-horizon reasoning.
  • Efficiency: TOOLSELF demonstrated higher token efficiency compared to specialized agents like Co-Sight, despite deeper exploration.

Significance and Claims

The paper claims that TOOLSELF illuminates a path toward emergent adaptivity in agentic systems. By replacing manually injected guidance with learnable, runtime self-reconfiguration, the system can:

• Dynamically specialize for diverse tasks and execution stages while preserving generalist flexibility.
• Internalize advanced capabilities such as long-horizon planning, self-refinement, and self-correction.
• Achieve state-of-the-art performance on complex benchmarks without requiring task-specific static configurations or external orchestrators.

The authors emphasize that this approach resolves the specialization-generalization trade-off by making configuration a learnable, dynamic variable rather than a static constraint, enabling agents to "learn to adapt" through trajectory-level feedback.