💻 computer science

Multi-Agent LLM Collaborative Reasoning and Task Planning for Complex Task Solving

This paper proposes a multi-agent LLM framework featuring distinct planner, executor, and verifier roles operating on a dependency-aware task graph with calibrated parameters and shared-memory mechanisms, which significantly outperforms single-agent baselines in complex task solving by improving completion rates, accuracy, and factual consistency while reducing tool errors.

Original authors: Wentao Zhang, Tian Liao, Yihui Feng

Published 2026-07-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wentao Zhang, Tian Liao, Yihui Feng

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, complicated puzzle, like building a skyscraper out of LEGOs while blindfolded, relying only on a single person's memory and instructions. That is essentially what happens when we ask a single Large Language Model (LLM) to solve a complex problem. It tries to do everything at once: plan the structure, build the walls, check the measurements, and fix mistakes. Often, it gets confused, makes a mistake early on, and then builds the rest of the tower on top of that error, leading to a collapse.

This paper proposes a better way: instead of one person doing everything, we create a specialized construction crew made of three distinct AI agents who work together in a strict, organized line.

Here is how their "construction crew" works, using simple analogies:

1. The Three Roles (The Crew)

The paper splits the work into three specific jobs, just like a real-world project team:

The Planner (The Architect): This agent looks at the big picture. Instead of just saying "build a tower," it breaks the job down into a map of dependencies. It says, "First, we need a foundation. Then, we need pillars. We can't put the roof on until the pillars are done." It assigns a "priority score" to each step based on how important it is and how risky it might be.
The Executor (The Builder): This agent does the actual heavy lifting. It follows the Architect's map, grabs the necessary tools (like a calculator, a search engine, or a code interpreter), and builds the specific parts. It doesn't worry about the whole building; it just focuses on finishing its specific task correctly.
The Verifier (The Inspector): This is the new, crucial addition. Before the Builder's work is accepted, the Inspector checks it. They look at the evidence: "Did you use the right tool? Is the math correct? Does this match the blueprint?" If the Inspector isn't confident (below a specific "trust score" of 0.72), they send the Builder back to fix that specific part. They don't tear down the whole building; they just fix the broken brick.

2. The Shared Notebook (Shared Memory)

In a normal conversation, people might forget what was said five minutes ago. In this system, all three agents share a digital notebook.

When the Architect draws a plan, it goes in the notebook.
When the Builder finds a piece of evidence, it gets written down with a "source tag."
When the Inspector rejects an idea, that rejection is also recorded so no one tries to build on that broken idea again.
This ensures that if the team has to restart a section, they don't have to re-learn everything from scratch. They can just look at the notes.

3. The "Traffic Light" System (The Protocol)

To prevent the agents from talking over each other or getting stuck in an endless loop of arguing, they follow a strict set of rules:

Green Light: The Architect gives a task.
Yellow Light: The Builder does the work and shows their evidence.
Red or Green Light: The Inspector checks the work.
- Green: "Good to go." The work is saved.
- Red: "Stop." The Inspector explains exactly why it failed (e.g., "Wrong tool used" or "Missing data"). The Architect then updates the plan to fix just that one issue.

4. The Results: Why It Works Better

The researchers tested this "Crew" against a single "Solo Worker" (a standard GPT-4 model) on difficult tasks like complex math, using multiple tools, and answering tricky factual questions.

Think of the Solo Worker as a brilliant but tired student who tries to do a 10-page essay in one sitting. They might make a small error in paragraph 2, and by paragraph 10, the whole essay is off-track.

The "Crew" approach acts like a team of editors and writers checking each other's work as they go. The paper found that by using this system:

They finished more tasks: The success rate went up by 27.3%.
They got the answers right more often: Accuracy improved by 19.6%.
They made fewer tool mistakes: Errors like using the wrong calculator or searching for the wrong thing dropped by 31.5%.
They were more honest: They were less likely to make up facts, improving factual consistency by 24.8%.

The Bottom Line

The paper argues that the secret to solving hard problems isn't just making the AI "smarter." It's about organization. By separating the roles of planning, doing, and checking, and forcing them to communicate through a structured, evidence-based system, the AI team avoids the "one-person bias" where a single mistake ruins the whole project. It turns a chaotic brainstorming session into a disciplined, auditable construction project.

Technical Summary: Multi-Agent LLM Collaborative Reasoning and Task Planning for Complex Task Solving

1. Problem Statement

Complex open-domain tasks require large language models (LLMs) to perform long-range reasoning, controlled decomposition, and reliable tool-mediated execution. While single-agent LLM workflows have advanced through techniques like Chain-of-Thought and ReAct, they suffer from three primary limitations:

One-perspective planning bias: A single model tends to follow a dominant perspective, leading to early mistaken assumptions that propagate silently.
Accumulated errors: Long reasoning chains accumulate errors that are difficult to correct without a full restart.
Weak tool control: Single agents often struggle with the precise sequencing, parameter selection, and validation of external tool invocations.

Existing multi-agent approaches (e.g., Generative Agents, Multiagent Debate, AutoGen) often rely on unconstrained conversation, which can lead to redundant dialogue and uncontrolled expansion without guaranteed reliability. This paper addresses the need for a controllable, auditable, and evidence-based multi-agent framework that separates planning, execution, and verification.

2. Methodology

The proposed framework introduces a Planner-Executor-Verifier collaborative architecture governed by a hierarchical task graph and an ordered interaction protocol.

A. Framework Architecture

The system separates context understanding from agent collaboration. It utilizes a CNN-enhanced context extractor to model structured and visual task signals (tables, diagrams, logs) alongside natural language, fusing these representations before task planning.

The core loop involves three specialized agents sharing an episodic memory:

Planner Agent: Converts input queries into a dependency-aware task graph ( $G_T = (V, E, \chi)$ ). It assigns priorities based on dependency centrality, user-goal relevance, uncertainty, and resource cost.
Executor Agent: Performs scheduled subtasks, invoking tools, generating artifacts, and producing local answers with compact evidence traces.
Verifier Agent: Evaluates claims against evidence, detects contradictions, and issues repair requests if confidence falls below a calibrated threshold.

B. Hierarchical Task Planning

Tasks are represented as directed graphs where nodes are subtasks with state tuples: $(priority, confidence, resource\_budget, status)$ .
The scheduling priority ( $p_i$ ) for a subtask is calculated as:
$p_i = \alpha d_i + \beta u_i + \gamma(1 - c_i) - \lambda r_i$
Where:

$d_i$ : Dependency centrality
$u_i$ : User-goal relevance
$c_i$ : Current confidence
$r_i$ : Estimated execution cost
$\alpha, \beta, \gamma, \lambda$ : Coefficients determined via grid search on a validation set.

Default Parameters: $\alpha = 0.35, \beta = 0.30, \gamma = 0.25, \lambda = 0.10$ .

C. Interaction Protocol and Memory

The system enforces a typed interaction protocol to ensure auditability:

Planner issues a subtask with prerequisites and termination criteria.
Executor returns a claim with an evidence trace.
Verifier outputs an Accept, Revise, or Split signal.

Shared Memory: Updates are managed via a function $\phi(\cdot)$ that stores typed records (positive/negative evidence, conflicts) with semantic and graph-based indexing. This allows the system to reuse verified information and avoid regenerating entire answers when only specific nodes fail.

Failure-Aware Repair: When the verifier rejects a claim, the system identifies the specific failure mode (missing evidence, wrong tool, etc.) and reopens only the affected node and its downstream dependencies, rather than restarting the entire task.

D. Tool Error Taxonomy

The paper defines a unified taxonomy for tool errors to ensure comparable metrics:

Wrong tool selection
Invalid parameters
Invalid invocation sequence
Execution failure
Inconsistent result integration

3. Key Contributions

End-to-End Collaborative Architecture: A multi-agent system integrating task decomposition, role-specific execution, cross-validation, and targeted repair for open-domain complex tasks.
Hierarchical Planning Algorithm: A formalized planning method with explicit parameter selection rules and sensitivity analysis for priority coefficients and verifier thresholds.
Concrete Shared-Memory Mechanism: A defined update function specifying information storage, indexing, retrieval, and conflict handling using typed records.
Unified Tool-Error Classification: A standardized taxonomy for counting tool-use errors, enabling fair comparison across methods.
Empirical Validation: Comparative experiments demonstrating consistent gains over a single-agent GPT-4 baseline across logical reasoning, tool orchestration, and open-domain problem solving.

4. Experimental Results

The framework was evaluated on three benchmark categories: Multi-step logical reasoning, Cross-tool orchestration, and Open-domain problem solving. The baseline was a single-agent GPT-4 instance using the same tools and instructions.

Performance Improvements (Relative to Baseline):

Task Completion Rate: +27.3% (Baseline: 63.0% $\to$ Proposed: 80.2%)
Answer Accuracy: +19.6% (Baseline: 68.4% $\to$ Proposed: 81.8%)
Reasoning Accuracy: +22.1%
Factual Consistency: +24.8%
Tool Error Rate: -31.5% (Baseline: 25.7% $\to$ Proposed: 17.6%)

Sensitivity and Ablation:

Parameter Sensitivity: Varying priority coefficients ( $\pm 0.10$ ) changed completion rates by no more than 2.6 percentage points. Varying the verifier threshold ( $\theta$ ) from 0.65 to 0.80 changed accuracy by no more than 3.2 percentage points, indicating robustness.
Ablation Study: Removing the Verifier caused the largest drop in factual consistency (-11.6%). Removing Hierarchical Planning reduced completion rates and increased redundant turns. Removing the Ordered Protocol led to unstable collaboration and longer, less effective dialogues.

5. Significance and Claims

The paper claims that the proposed framework addresses the limitations of single-path reasoning by treating reasoning as a controllable process rather than a single-pass generation event.

Controllable Collaboration: By assigning explicit roles and enforcing a typed protocol, the system avoids the "uncontrolled dialogue expansion" common in unconstrained multi-agent debates. Each agent message serves a specific function (planning, acting, or verifying).
Error Mitigation: The separation of concerns allows the system to detect and repair errors (e.g., invalid tool sequences) before they propagate to the final answer, significantly reducing tool error rates.
Interpretability: The architecture ensures that final answers can be traced back to accepted subtasks and evidence records, enhancing the auditability of the reasoning process.
Scalability: The framework is designed for long-horizon tasks where early errors would otherwise silently degrade the final output, offering a stable trade-off between reliability and interaction cost (average turns ~8.6).

The authors conclude that structured communication and role separation are more critical to performance than simply increasing the number of agents or the volume of dialogue. Future work is identified as investigating adaptive parameter tuning and deployment in real-world workflows with heterogeneous tools.