💬 NLP

Multi-Agent Computer Use

This paper proposes Multi-Agent Computer Use (MACU), a system where a manager model orchestrates parallel subagents via a dynamic directed acyclic graph (DAG) to effectively handle complex, long-horizon tasks, demonstrating significant performance improvements over single-agent baselines across desktop and web navigation benchmarks.

Original authors: Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you need to organize a massive, complex event, like planning a wedding or a corporate retreat.

The Old Way (Single-Agent):
In the past, we tried to get this done by hiring one incredibly smart but very busy assistant. This assistant had to do everything alone: call the caterer, book the venue, check the weather, and buy the flowers. If the caterer said, "We're booked on that date," the assistant had to stop, call the venue to check if that date was open, call the florist to see if they were available, and then try to call the caterer again with a new date. If the assistant got stuck on one phone call, the whole event planning stalled. They were working in a straight line, one step after another, and if they hit a wall, they had to back up all the way to the beginning to try a different path.

The New Way (Multi-Agent Computer Use - MACU):
This paper proposes a smarter way: hiring a Project Manager and a team of Specialist Assistants.

Here is how the new system works, using the wedding planning analogy:

1. The Project Manager (The "Manager")

Instead of doing the work, the Manager's only job is to look at the big picture and draw a map (called a Directed Acyclic Graph, or DAG).

Breaking it down: The Manager looks at the task "Plan a wedding" and breaks it into smaller pieces: "Find a venue," "Book a caterer," "Choose flowers," and "Send invites."
The Map: The Manager draws a map showing which tasks can happen at the same time. For example, "Find a venue" and "Book a caterer" don't depend on each other, so they can happen simultaneously.
The Brain: The Manager is the only one who sees the whole map. If a new piece of information comes in (e.g., "The venue is too expensive"), the Manager can instantly redraw the map, cancel the expensive venue search, and send a new assistant to find a cheaper one, all without stopping the other assistants.

2. The Team of Assistants (The "Subagents")

The Manager sends out a team of identical assistants (subagents) to do the actual work.

Parallel Power: Instead of waiting for one person to finish, the Manager sends four assistants out at once. One goes to check hotels, another checks flights, a third checks restaurants, and a fourth checks weather. They work in parallel, like a pit crew changing tires on a race car.
Isolation: Each assistant works in their own private "sandbox" (a virtual computer). If one assistant crashes or gets confused, it doesn't break the others. They are isolated from each other's mistakes.

3. The "Live" Map (Re-planning)

This is the most important part. In the old system, if you made a plan, you stuck to it even if things changed.
In this new system, the map is alive.

The Feedback Loop: As soon as an assistant finishes a task (e.g., "I found a hotel, but it's full"), they report back to the Manager.
The Pivot: The Manager looks at the new info and immediately updates the map. Maybe the hotel is full, so the Manager adds a new task to "Find a backup hotel" and sends a fresh assistant to do it. If an assistant gets stuck in a loop (like trying to click a button that doesn't work), the Manager can say, "Stop that one, try a different approach," and send a new person with a different strategy.
Remembering the Past: Sometimes, an assistant finds a file or a piece of information that the next person needs. The Manager acts like a librarian, saving that file and handing it to the next assistant so they don't have to start from scratch.

Why is this better?

The paper tested this on four different types of "computer tasks," ranging from simple desktop chores (like changing a font) to complex, multi-day travel planning on the web.

Speed: For complex tasks, the team finished about 1.5 times faster than the single assistant because they were doing things at the same time.
Success Rate: The team was much better at solving hard problems. On the hardest travel-planning tests, the single assistant failed 90% of the time, but the team succeeded 34% of the time. That's a huge jump.
Getting Unstuck: When the single assistant got stuck on a dead end, the whole process stopped. The team, however, could send a different person to try a different door. If one path was blocked, another path was already open.

The Bottom Line

The paper argues that for complex, long-term computer tasks, we shouldn't rely on a single "super-bot" trying to do everything in a straight line. Instead, we should use a Manager who draws a flexible map and a Team of workers who can run in parallel, stop, start, and change direction instantly based on what they find.

It's the difference between a lone explorer trying to cross a jungle alone versus a guided expedition with a leader, a map, and a team that can split up, cover more ground, and adapt to obstacles instantly.

Technical Summary: Multi-Agent Computer Use (MACU)

Problem Statement

Current Computer Use Agents (CUAs) are predominantly deployed as single, serial agents. While effective for simple tasks, this paradigm is suboptimal for complex, long-horizon tasks. Single agents struggle with:

Lack of Decomposition: Inability to break down complex goals into manageable subtasks.
Sequential Bottlenecks: Inability to execute independent subtasks in parallel.
Rigid Re-planning: Difficulty in consistently revising plans based on new information, particularly in partially observable environments.
State Loss: In computer use, the environment (filesystem, open tabs, cookies, background processes) is only partially observable at any moment. Once a subtask ends, the specific state relevant to a downstream subtask often cannot be re-observed, leading to information loss that single agents cannot recover from.

Methodology: The MACU Framework

The authors propose Multi-Agent Computer Use (MACU), a framework that coordinates a pool of identical CUA subagents via a central Manager model. The system operates on a Directed Acyclic Graph (DAG) of subtasks.

Core Components

Manager Model:
- Role: Orchestrates the workflow, decomposes the initial user task into a DAG, and continuously replans based on feedback.
- Mechanism: The manager receives the task instruction, the current DAG state, and observations (screenshots, responses, file diffs) from completed subagents.
- Actions: It can add nodes (new subtasks), remove nodes, modify instructions/dependencies, or cancel running nodes. It operates under a finite replan budget ( $B$ ) to limit computational cost.
- File Management: The manager tracks filesystem diffs to identify and archive relevant files created by subagents, passing them as inputs to subsequent subtasks (e.g., handing a document from Subtask A to Subtask B).
CUA Subagents:
- Role: Execute specific nodes on the DAG.
- Environment: Each subagent runs in an isolated Virtual Machine (VM) to maintain state independence and allow for parallel execution.
- Behavior: They follow a standard ReAct loop (Observe $\rightarrow$ Reason $\rightarrow$ Act) using a base CUA model (e.g., Qwen3.6-27B). They are treated as "black boxes" by the manager, making the framework agnostic to the specific CUA backbone used.
Execution Flow:
- Initialization: The manager generates an initial DAG based on the user task.
- Parallel Dispatch: Subagents are launched in parallel to tackle nodes on the "ready frontier" (nodes whose dependencies are satisfied).
- Iterative Replanning: As subagents complete tasks, the manager reviews their outputs. If new information suggests a different path (e.g., a search failed, a file is missing), the manager modifies the DAG (adding retry branches, alternative strategies, or canceling stuck nodes).
- Aggregation: A final node aggregates results from parent nodes to produce the user deliverable.

Handling Partial Observability

A key design choice is treating partial observability as a first-class challenge. The DAG structure and the Manager explicitly retain information (screenshots, file states, reasoning traces) that downstream agents cannot re-observe. This ensures that context lost during the execution of a specific subtask is preserved and passed forward through the graph structure.

Key Contributions

General Multi-Agent Setup: A flexible framework where a manager decomposes tasks into a DAG of parallel subagents, capable of dynamic replanning and structural modification.
State Preservation Mechanism: Explicit handling of file management and state transfer between isolated VMs, solving the "information loss" problem inherent in serial computer use.
Model Agnosticism: The system uses identical subagents for execution, allowing it to leverage improvements in base CUA models without architectural changes.
Empirical Validation: Comprehensive evaluation across four diverse benchmarks (OSWorld, Online-Mind2Web, WebTailBench-v2, Odysseys).

Results

The paper reports consistent improvements of MACU over strong single-agent baselines across all tested benchmarks:

Success Rate Improvements:
- OSWorld: +4.7% (43.8% $\rightarrow$ 48.5%) with an Opus 4.6 manager.
- Online-Mind2Web: +3.4% (52.2% $\rightarrow$ 55.6%).
- WebTailBench-v2: +8.7% (20.8% $\rightarrow$ 29.5%).
- Odysseys (Long-Horizon): +25.5% (8.5% $\rightarrow$ 34.0%).
Efficiency and Scaling:
- Wall-Clock Time: MACU reduced task completion time by $\sim$ 1.5 $\times$ on Odysseys (162 min $\rightarrow$ 110 min) due to parallel execution. Conversely, on highly serial tasks (Online-Mind2Web), wall-clock time increased slightly due to manager overhead, though success rates still improved.
- Test-Time Scaling: MACU demonstrates superior scaling trends with increased inference budget (more actions executed) compared to single agents, which plateau earlier. It solves long-horizon tasks that single agents get stuck on.
Ablation Insights:
- Replanning is Critical: Allowing the manager to modify the graph during runtime ( $B > 1$ ) significantly boosts performance (e.g., from 25% to 58.3% success rate on a subset), whereas static decomposition offers minimal gains.
- Manager Strength: Stronger manager models (e.g., Opus 4.6) yield substantially higher success rates than weaker ones, even with the same subagent backbone.
- Parallelism: Increasing the number of parallel subagents ( $N$ ) from 1 to 4 on Odysseys reduced wall-clock time by $\sim$ 3.2 $\times$ while improving success rates.

Significance and Claims

The authors argue that multi-agent coordination is a promising axis for scaling computer use agents. The paper claims that:

Decomposition and Parallelism are essential for tackling complex, long-horizon tasks where serial agents fail.
Continuous Replanning is necessary to handle the dynamic and partially observable nature of real-world computer environments.
MACU is a General Solution: It is not limited to specific domains but applies to desktop control, web navigation, and complex multi-step workflows.
Future Potential: The framework is designed to seamlessly integrate with future, more capable CUA models. The authors suggest that further fine-tuning or reinforcement learning specifically for multi-agent coordination could further enhance system performance.

The paper concludes that while single-agent CUAs have reached high performance on many benchmarks, the shift to multi-agent systems like MACU is necessary to unlock productivity for longer, more complex, and more robust computer use tasks.