🤖 AI

Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning

This paper proposes "Planner Matters," an efficient multi-agent framework that demonstrates through systematic analysis and reinforcement learning that concentrating model capacity and optimization on a high-level planner—while freezing execution and memory components—significantly improves long-horizon task automation across web navigation, OS control, and tool use.

Original authors: Wenyi Wu, Sibo Zhu, Kun Zhou, Biwei Huang

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenyi Wu, Sibo Zhu, Kun Zhou, Biwei Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Why One Brain Isn't Enough

Imagine you are trying to build a complex piece of furniture from a flat-pack box while blindfolded, relying only on someone else to describe what you see. If you try to do everything at once—figure out the blueprint, find the right screw, and tighten it—you might get overwhelmed. You might tighten the wrong screw because you were too focused on the blueprint, or you might forget the blueprint because you were too focused on the screw.

This is the problem current AI agents face. They are trying to be a "super-brain" that plans the whole journey and drives the car at the same time. The authors of this paper argue that this approach is inefficient. Instead, they propose splitting the job into three distinct roles, like a small construction crew:

The Planner (The Foreman): This agent never touches the tools. Its only job is to look at the big picture, break the project into steps, and decide the strategy.
The Actor (The Worker): This agent does the physical work. It reads the Foreman's instructions and clicks buttons, types text, or scrolls the screen. It doesn't worry about the "why," just the "how."
The Memory Manager (The Librarian): This agent keeps a notebook of what has happened. It reminds the Foreman of past mistakes or successful tricks so they don't have to reinvent the wheel.

The Big Discovery: The Foreman is the Most Important

The researchers ran a series of experiments to see where they should spend their "computing power" (the brainpower of the AI). They asked: If we make the Foreman smarter, or the Worker stronger, or the Librarian faster, which one helps the most?

The answer was surprising: It's all about the Foreman (the Planner).

The Analogy: Imagine you have a team of three people. If you hire a genius architect (Planner) but give them a regular bricklayer (Actor) and a standard note-taker (Memory), the building gets built perfectly. But if you hire a world-class bricklayer and a super-fast note-taker but give them a confused architect, the building will still fall apart.
The Finding: The paper shows that upgrading the Planner's intelligence yields massive improvements in success rates. In contrast, making the Actor or Memory Manager much bigger or smarter barely changes the outcome. The "bottleneck" is almost always the planning.

The Solution: Train the Foreman, Freeze the Rest

Based on this discovery, the authors created a new training method. Usually, when you train an AI, you try to improve the whole system at once. But here, they did something different:

They kept the Actor and Memory Manager exactly the same (frozen). They didn't change them at all.
They focused 100% of their training energy on the Planner.
They used a "VLM-as-judge" (a very smart AI judge) to look at the whole finished task. Did the agent succeed? If yes, the Planner gets a high score. If no, the Planner gets a low score.
The Planner learns from these scores to get better at making plans, while the Worker just keeps doing what it's told.

The Results: Smarter Planning, Cheaper Computing

The results were impressive. By focusing only on the Planner:

Performance Skyrocketed: An open-source AI model (Qwen2.5-7B) with this "Planner-Centric" setup performed better than much more expensive, closed-source giants like GPT-4o and Gemini-2.5-Pro on complex web tasks.
Efficiency: They didn't need to buy more powerful computers for the whole team. They just made the "Foreman" smarter, and the whole crew worked better.
Versatility: This approach worked not just for browsing websites, but also for controlling computer operating systems and using software tools.

Summary

The paper argues that for AI agents to handle long, complex tasks, we shouldn't just make the whole AI bigger. Instead, we should specialize. We need a dedicated, highly intelligent "Planner" to handle the strategy, while letting simpler, smaller models handle the execution and memory. By training the Planner specifically to be the best decision-maker, we can build agents that are smarter, more reliable, and more efficient.

Technical Summary: Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning

Problem Statement

Language model (LM)–based agents have demonstrated capabilities in automating complex tasks from natural language instructions but continue to struggle with long-horizon planning and reasoning. A core difficulty identified is that a single monolithic agent often fails to simultaneously excel at high-level goal management and low-level action execution. When an agent focuses excessively on low-level details, it may execute local steps correctly but miss the overall objective (e.g., clicking correctly but failing to compare options). Conversely, focusing too much on high-level objectives can lead to execution failures due to minor mistakes (e.g., missing state changes). Existing agents also face challenges in tasks requiring complex compositional intents, multi-source comparisons, and constrained filtering.

Methodology

1. Multi-Agent Framework Decomposition

The authors propose a multi-agent framework that decomposes automation into three specialized roles:

Planner: Handles high-level goal decomposition, generates initial plans, and incrementally updates them based on observations and execution history.
Actor: Translates the Planner's subgoals into concrete, executable actions (e.g., Click, Type, Scroll, ToolInvoke) within the digital environment.
Memory Manager: Retrieves and updates relevant context, providing the Planner with hybrid memory consisting of:
- Discrete Memory ( $M^d_t$ ): Structured, text-based summaries of critical steps from past successful trajectories.
- Continuous Memory ( $M^c_t$ ): Latent representations encoding long-term, multimodal trajectory information into compact embeddings.

The system operates in an iterative loop where the Planner communicates focused subgoals to the Actor, and the Memory Manager supports context retrieval.

2. Systematic Compute-Allocation Analysis

A central contribution of the work is a systematic analysis of how model capacity should be allocated across these components. Through controlled scaling experiments using models of varying sizes (e.g., Qwen2.5-VL-3B/7B/32B), the authors found that:

Planning is the dominant bottleneck: Scaling the Planner yields the largest marginal performance gains.
Unbalanced efficiency: Upgrading the Planner alone achieves performance gains comparable to scaling all components simultaneously.
Low sensitivity of other components: The Actor and Memory Manager require significantly less compute and model capacity to achieve competitive results. The Memory Manager showed the smallest sensitivity to scale.

3. Planner-Centric Reinforcement Learning (RL)

Building on the insight that planning is the primary bottleneck, the authors introduce a planner-centric reinforcement learning approach:

Frozen Components: The Actor and Memory Manager are frozen during training.
Optimization Target: Only the Planner is optimized.
Reward Signal: The system uses trajectory-level rewards derived from a VLM-as-judge (Qwen2.5-VL-32B). The judge evaluates the full episode (plans, screenshots, final answer) based on task correctness, reasoning coherence, and interaction efficiency, assigning a score of 1, 3, or 5.
Algorithm: The authors apply Group Relative Policy Optimization (GRPO) with on-policy rollouts. The trajectory-level reward is broadcast to all steps within a rollout to update the Planner's policy.

Key Contributions

Identification of the Planning Bottleneck: The paper provides empirical evidence that in long-horizon agent tasks, planning capacity is the primary driver of success, while execution and memory management are less computationally demanding.
Unbalanced Multi-Agent Architecture: The authors propose a framework that leverages this insight by allocating the majority of model capacity to the Planner, allowing for the use of smaller, more efficient models for the Actor and Memory Manager.
Planner-Centric RL Strategy: A novel training strategy that exclusively optimizes the Planner using trajectory-level rewards, avoiding the role confusion and competing objectives often seen in joint training of planning and execution modules.
Comprehensive Evaluation: Extensive experiments across web navigation (WebVoyager, Mind2Web, MMInA), OS control (OSWorld), and tool use (MCPBench).

Results

The proposed framework demonstrates robust improvements across various benchmarks and model scales:

Performance Gains: On the WebVoyager benchmark, a Qwen2.5-VL-7B backbone with the multi-agent setup and planner-centric RL achieved a 35.1% overall success rate, a 180.8% relative improvement over its single-model baseline (12.5%) and surpassing closed-source baselines like GPT-4o (19.7%) and Gemini-2.5-Pro (29.6%).
Efficiency: The framework achieves these gains with an unbalanced configuration (e.g., 7B Planner + 3B Actor + 3B Memory), demonstrating that concentrating capacity on planning is more efficient than uniformly scaling all components.
Generalization: The method shows strong generalization to out-of-domain tasks (e.g., OSWorld and MCPBench) and different interface layouts, indicating that the learned planning abilities transfer effectively.
Ablation Studies:
- Training the Planner alone outperformed training the Actor alone or jointly training both, confirming that focused optimization of high-level reasoning is superior.
- The inclusion of continuous memory provided consistent, albeit smaller, improvements over discrete memory alone.

Significance and Claims

The paper claims that disentangling and prioritizing planning capacity is a critical strategy for achieving state-of-the-art performance in long-horizon agents under constrained compute budgets. The authors argue that their findings align with principles in cognitive neuroscience regarding high-level executive control as a bottleneck in complex tasks.

The work provides practical guidance for compute allocation in multi-agent systems, suggesting that rather than scaling all components equally, developers should focus resources on the planning module. The authors position their approach as a compute-efficient alternative to monolithic scaling, enabling smaller open-source models to compete with or surpass larger closed-source models in complex automation tasks. The code is publicly released to support reproducibility.