🤖 AI

MAR:Multi-Agent Reflexion Improves Reasoning Abilities in LLMs

The paper proposes MAR (Multi-Agent Reflexion), a method that employs multi-persona debaters to generate diverse reflections and overcome the thought degeneration seen in single-agent self-reflection, thereby significantly improving reasoning performance on tasks like HotPot QA and HumanEval.

Original authors: Onat Ozer, Yuchen Wang, Grace Wu, Daniel Dosti, Honghao Zhang, Vivi De La Rue

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Onat Ozer, Yuchen Wang, Grace Wu, Daniel Dosti, Honghao Zhang, Vivi De La Rue

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Echo Chamber" of AI

Imagine you are trying to solve a very difficult puzzle. You make a mistake, but instead of asking a friend for help, you are forced to look at your own mistake, write a note to yourself about what went wrong, and then try again.

This is how the original Reflexion system works. It's like a student taking a test alone in a room. If they get a question wrong, they write a note saying, "I got this wrong because I missed a detail," and then they try again.

The researchers found a major flaw in this approach: The student is too stuck in their own way of thinking.

If the student made a mistake because they misunderstood the rules, their self-note often just repeats that same misunderstanding.
They end up in an "echo chamber," where they keep making the exact same error over and over, justifying it with slightly different words.
In the paper's technical terms, this is called "degeneration of thought." The AI gets stuck in a loop of bad reasoning.

The Solution: The "Panel of Experts"

To fix this, the authors created Multi-Agent Reflexion (MAR).

Instead of one student talking to themselves, imagine the AI is now a team of experts sitting around a table to solve the problem. When the team makes a mistake, they don't just write a note; they hold a structured debate.

Here is how the new system works, step-by-step:

The Actor (The Doer): One AI tries to solve the problem first.
The Failure: If the answer is wrong, the system doesn't just ask the "Doer" to fix it.
The Debate (The Critics): The system calls in a team of different "personas" (specialized AI characters). Think of them like:
- The Skeptic: Someone who doubts everything and looks for hidden traps.
- The Logician: Someone who checks if the steps actually make sense mathematically.
- The Creative: Someone who suggests wild, alternative ideas.
- The Verifier: Someone who checks if the answer matches the exact rules.
The Judge: A "Judge" AI listens to all these different experts argue. It doesn't just pick a winner; it synthesizes their arguments into one clear, high-quality lesson.
The Retry: The "Doer" AI gets this new, rich lesson and tries again. Because the lesson came from a diverse group, it is much more likely to spot the real mistake rather than just repeating the old one.

The Results: Does It Work?

The researchers tested this on two very different types of challenges:

The "Detective" Test (HotPotQA): This involves answering questions that require connecting clues from multiple different documents.
- The Result: The single-AI system (Reflexion) got about 44% of the answers right. The new "Panel of Experts" system (MAR) got 47% right.
- Note: The authors admit this improvement was smaller than hoped, partly because the grading system for these questions is very strict about tiny formatting details (like a missing period), which sometimes penalized correct reasoning.
The "Coder" Test (HumanEval): This involves writing computer code that has to pass hidden tests.
- The Result: The single-AI system got 76.4% of the code working. The new "Panel of Experts" system got 82.6% working.
- Why it worked better here: Coding errors are often logical loops or off-by-one mistakes. The "Skeptic" and "Logician" personas were very good at spotting these specific types of errors that a single AI kept missing.

The Catch: It Costs More

The paper is honest about the downside.

The Price of Debate: Having a whole team argue takes a lot more energy and time than one person thinking alone.
The Cost: The new system uses about 3 times more computing power (and costs 3 times more money in API fees) than the original system because it has to run multiple AI models to hold the debate.

The Bottom Line

The paper argues that while AI is getting smarter, it still struggles to learn from its own mistakes because it gets stuck in its own habits. By forcing the AI to argue with different versions of itself, it breaks out of those bad habits, finds better solutions, and becomes more reliable—though it does so at a higher cost.

It's the difference between a solo artist trying to fix a painting alone versus a whole art critique team pointing out exactly what's wrong and how to fix it.

Technical Summary: Multi-Agent Reflexion (MAR)

Problem Statement

While Large Language Models (LLMs) demonstrate strong reasoning and program synthesis capabilities, enabling them to improve through experience without parameter updates remains a challenge. The Reflexion framework (Shinn et al., 2023) addressed this by allowing agents to generate natural language reflections on failures, storing them as episodic memory to guide future attempts. However, our replication of Reflexion on HotPotQA and HumanEval reveals systematic shortcomings in its single-agent design:

Degeneration of Thought: The same model acts as the generator, evaluator, and reflector. This leads to confirmation bias, where the agent repeats the same flawed reasoning or justifies errors with additional flawed logic rather than identifying root causes.
Mode Collapse: Corrections tend to follow narrow, repetitive templates. In code generation, this manifests as the reproduction of identical buggy structures (e.g., off-by-one errors) across iterations.
Limited Corrective Feedback: Self-reflections often fail to introduce new reasoning paths, causing the agent to stagnate on difficult examples.

Methodology

To address these limitations, we introduce Multi-Agent Reflexion (MAR), a structured extension that replaces single-agent self-critique with a coordinated team of diverse, persona-driven critics.

Architecture

MAR decomposes the reasoning process into distinct roles to reduce shared blind spots:

Actor ( $M_a$ ): Generates actions and final answers using a baseline model (e.g., GPT-3.5).
Evaluator ( $M_e$ ): Determines success or failure (e.g., via unit tests or exact match).
Multi-Agent Debate Module: Triggered upon failure, this module replaces the single "Self-Reflector" with a structured debate:
- Diverse Personas: Multiple critic agents (e.g., Verifier, Skeptic, Logician, Creative) analyze the failed trajectory from different perspectives. These personas are designed to vary along axes of Evidence Exploitation (demand for textual support), Exploration (pushing for alternative hypotheses), and Specification Strictness.
- Debate Process: Critics diagnose the failure, debate their critiques, and refine their arguments over up to two rounds.
- Judge ( $M_j$ ): A coordinator model synthesizes the debate into a unified "Consensus Reflection." This reflection aggregates error outputs, debate logs, and agreed-upon strategies.
Memory Update: The consensus reflection is appended to the Actor's episodic memory, guiding the next retry.

Experimental Setup

We evaluated MAR on two benchmarks using GPT-3.5 and GPT-4:

HotPotQA: A multi-hop question answering benchmark requiring evidence integration across documents. We measured Exact Match (EM) accuracy over 5 trials.
HumanEval: A program synthesis benchmark testing if generated functions pass hidden unit tests. We measured pass@1 accuracy over a limited budget (initial attempt + 1 reflection-guided retry).

Key Results

Our replication confirms the limitations of single-agent Reflexion and demonstrates MAR's ability to overcome them:

HotPotQA:
- Baseline (ReAct, GPT-3.5): 32.0% EM.
- Reflexion (Single-Agent): 44.0% EM.
- MAR: 47.0% EM.
- Note: The authors attribute the modest gain over Reflexion to the sensitivity of the Exact Match metric, which penalizes semantically correct answers for minor formatting differences.
HumanEval:
- Baseline (GPT-3.5): 67.1% pass@1.
- Reflexion (Single-Agent): 76.4% pass@1.
- MAR: 82.6% pass@1.
- This represents a 6.2-point gain over Reflexion, indicating that MAR more effectively escapes repeated failure modes in code generation.

Contributions and Significance

The paper claims the following contributions:

Identification of Failure Modes: Through detailed logging, we identified that single-agent Reflexion suffers from confirmation bias and mode collapse, leading to repeated reasoning errors.
Multi-Agent Reflexion (MAR): We propose a framework that separates acting, diagnosing, critiquing, and aggregating. By introducing diverse reasoning personas and a judge model, MAR reduces shared blind spots and prevents the reinforcement of earlier mistakes.
Empirical Validation: MAR consistently outperforms both the baseline and single-agent Reflexion on both reasoning and code synthesis tasks, demonstrating that multi-agent interaction offers a simple, training-free path to more reliable LLM agents.

Limitations and Societal Impact

The authors acknowledge that MAR incurs a significant computational cost. The multi-agent debate and judge synthesis steps result in roughly a 3x increase in API calls and latency compared to single-agent Reflexion. This raises concerns regarding scalability, energy consumption, and economic accessibility. Furthermore, the paper notes that self-refining agents introduce safety and governance challenges, as emergent multi-agent behaviors may become harder to predict or align.

In conclusion, while MAR offers a scalable, training-free method for improving LLM reliability through structured disagreement, its deployment requires careful evaluation of efficiency and environmental impact.