From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG

The paper proposes MA-RAG, a multi-round agentic RAG framework that iteratively refines medical reasoning by transforming semantic conflicts into targeted evidence retrieval and optimizing reasoning traces, thereby achieving substantial accuracy improvements over existing baselines across seven medical benchmarks.

The Gist

🩺 The Problem: The "Overconfident Doctor"

Imagine a brilliant medical student (the AI) who has read every textbook in the library. They are great at answering questions, but they have two big flaws:

1. Hallucinations: Sometimes, they confidently make up facts that sound real but are wrong.
2. Outdated Knowledge: Their textbooks are from 2020, but medicine changes fast. They don't know the latest guidelines.

Standard AI tools try to fix this by looking up answers in a library (Retrieval-Augmented Generation, or RAG). But most of these tools are like a student who asks one question, gets one book, and immediately writes an essay. If the first book they picked was slightly off-topic, they get the whole answer wrong. They don't know when to stop and ask for more help.

🚀 The Solution: The "Roundtable of Experts" (MA-RAG)

The authors propose a new system called MA-RAG. Instead of a single student writing an essay alone, imagine a medical roundtable where a team of agents works together in a loop until they agree on the perfect answer.

Think of it like a detective solving a mystery, but with three specific roles:

1. The Solver (The Brainstorming Team)

• What they do: The AI generates several different possible answers to the medical question at once.
• The Analogy: Imagine a group of detectives all writing down their own theory of the crime on a whiteboard.
• The Magic: If all detectives write the same theory, they are probably right. But if they write different theories (e.g., "It was the butler" vs. "It was the gardener"), that's a Conflict. In MA-RAG, Conflict is good! It tells the system, "Hey, we don't know the answer yet. We need more info."

2. The Retrieval Agent (The Investigator)

• What they do: This agent looks at the conflicting theories and asks, "Where are we disagreeing?" It turns that disagreement into a specific search query to find the truth in a medical database.
• The Analogy: The detectives realize they are arguing about the time of death. The Investigator doesn't just search "Who killed the victim?" (too vague). Instead, they search specifically for "Time of death in cases of poisoning." They go out and bring back new, specific evidence to settle the argument.
• The Upgrade: Unlike older systems that search based on "how unsure the AI feels" (which is often wrong), this agent searches based on actual disagreement between the answers.

3. The Ranking Agent (The Editor)

• What they do: As the team gathers more evidence and writes more theories, the conversation gets long and messy. The Ranking Agent sorts the whiteboard. It puts the best, most accurate theories at the top and pushes the bad ones to the bottom.
• The Analogy: Imagine a teacher grading a stack of essays. Instead of reading them in the order they were written, the teacher puts the "A+" essays at the top of the pile so the student can learn from the best examples first. This prevents the "Lost in the Middle" problem, where important clues get buried in a long chat history.

🔄 The Loop: From Conflict to Consensus

The system doesn't stop after one try. It runs in a loop:

1. Generate: The team writes 5 different answers.
2. Check: They see they disagree on a specific medical fact (The Conflict).
3. Search: The Investigator finds the correct fact in the medical database.
4. Refine: The team reads the new fact, throws out the wrong answers, and writes a new, better set of answers.
5. Repeat: They keep doing this until everyone agrees (Consensus).

This is like Boosting in machine learning. In simple terms, it's like a coach telling a player, "You missed that shot. Here is the video of the mistake. Try again, but focus on fixing that specific error." The system iteratively fixes its own mistakes until the error is gone.

🏆 Why It Matters

The paper tested this on 7 different medical exams (from basic licensing to expert-level clinical reasoning).

• The Result: MA-RAG beat all other methods, including standard AI and other "search-and-answer" tools.
• The Gain: It improved accuracy by nearly 7 points on average. On the hardest exams, it improved by 37%.
• The Takeaway: By letting the AI argue with itself, find its own mistakes, and fetch specific evidence to fix them, it becomes much safer and more reliable for real-world healthcare.

🌟 Summary Metaphor

If standard AI is a student taking a test alone, MA-RAG is a study group with a librarian.

• The student (Solver) tries to answer.
• The group notices they are arguing (Conflict).
• The librarian (Retrieval Agent) runs to the shelves to find the exact page that settles the argument.
• The group leader (Ranking Agent) organizes the notes so everyone learns from the best ideas.
• They keep studying until they all agree on the correct answer.

This ensures that when the AI gives medical advice, it's not just guessing confidently; it's proven to be right through evidence and consensus.

Technical Summary

Here is a detailed technical summary of the paper "From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG" (MA-RAG).

1. Problem Statement

Large Language Models (LLMs) show promise in medical question-answering but suffer from two critical issues:

1. Hallucinations and Outdated Knowledge: LLMs often generate fluent but factually incorrect information or rely on parametric knowledge that lacks the latest medical guidelines.
2. Limitations of Existing RAG: Traditional Retrieval-Augmented Generation (RAG) typically relies on a single round of retrieval based on the initial query. This fails to address complex, multi-step reasoning where the initial query may be insufficient. Furthermore, existing adaptive RAG methods often rely on noisy token-level signals (e.g., confidence scores, attention weights) to trigger retrieval. These signals are unreliable because LLMs can be highly confident yet factually wrong (hallucinations).

The core challenge is how to efficiently steer retrieval and reasoning in complex medical domains without relying on unreliable low-level uncertainty metrics.

2. Methodology: MA-RAG Framework

The authors propose MA-RAG (Multi-Round Agentic RAG), a framework that treats medical reasoning as an iterative refinement loop. Instead of a static prompt, the context evolves over multiple rounds ($T$) involving three specialized agents. The system leverages semantic conflict among candidate answers as a high-level signal to guide retrieval, mirroring a boosting mechanism that minimizes residual errors.

The Agentic Loop

At each round $t$, the system maintains a state $S_t = \{I, q, D_t, H_t\}$, where $I$ is the instruction, $q$ is the query, $D_t$ is the document context, and $H_t$ is the history of reasoning traces.

1. Solver Agent:

   • Generates a diverse set of $N$ candidate responses ($A_t$) based on the current state.
   • Uses temperature-controlled sampling to explore the solution space.
   • Termination Condition: If all candidates converge to the same answer (consensus), the loop terminates.

2. Retrieval Agent:

   • Core Innovation: Instead of using token-level uncertainty, this agent analyzes semantic conflicts (e.g., contradictory diagnoses or differing embryologic origins) among the candidates in $A_t$.
   • It formulates $K$ targeted retrieval queries ($R_t$) specifically designed to resolve these identified conflicts.
   • Retrieves new evidence from a local medical corpus, updating the document context $D_{t+1}$.

3. Ranking Agent:

   • Addresses the "lost-in-the-middle" problem in long contexts by optimizing the history context $H_t$.
   • Evaluates the quality of previous candidates ($A_{t-1}$) using a score function $Q(\cdot)$ and reorders them in descending order of quality.
   • Scoring Functions:
     • Intrinsic: Sequence-level entropy (uncertainty).
     • Extrinsic: A fine-tuned BERT-based verifier trained to detect semantic correctness.
   • The top-ranked traces are presented as high-quality in-context demonstrations to the Solver Agent in the next round.

Theoretical Grounding

• Adaptive Self-Consistency: Extends standard self-consistency by using inconsistency as a trigger for further reasoning and retrieval, rather than just a voting mechanism.
• Boosting Analogy: The framework treats semantic conflict as a "residual error." Each round attempts to "fit" this residual by retrieving specific evidence and optimizing context, iteratively reducing the error until a stable consensus is reached.

3. Key Contributions

1. Conflict-Guided Retrieval: Proposes a novel mechanism where semantic disagreement among multiple reasoning paths serves as a robust, high-level signal for adaptive retrieval, bypassing the noise of token-level uncertainty.
2. Context Optimization via Ranking: Introduces a Ranking Agent that actively restructures history reasoning traces to mitigate long-context degradation, ensuring the model focuses on high-quality demonstrations.
3. Agentic Refinement Loop: Combines external evidence evolution and internal reasoning history optimization into a unified multi-round framework that mimics human iterative verification.
4. Comprehensive Evaluation: Validated across seven diverse medical benchmarks, demonstrating superiority over both static RAG and advanced test-time scaling baselines.

4. Experimental Results

The authors evaluated MA-RAG on seven medical Q&A benchmarks (MedQA, MedMCQA, MedXpertQA, etc.) using the Qwen3-8B backbone.

• Performance Gains: MA-RAG achieved an average accuracy improvement of +6.8 points over the backbone model.
• Comparison with Baselines:
  • Outperformed Test-Time Scaling methods (e.g., Self-Consistency, Multi-Refine) which plateaued due to internal knowledge limits.
  • Significantly surpassed Adaptive RAG baselines (FLARE, TC-RAG) which rely on token-level signals.
  • Achieved a 37% improvement on the difficult MedXpertQA benchmark compared to baselines.
• Ablation Studies:
  • Retrieval Agent: Adding conflict-guided retrieval provided a +1.9 point gain, proving essential for bridging knowledge gaps.
  • Ranking Agent: Context optimization provided an additional +1.6 point gain, confirming the importance of prioritizing high-quality traces.
  • Scoring: The Extrinsic Verifier (BERT-based) outperformed the Intrinsic Entropy metric, highlighting the unreliability of token-level confidence in medical domains.
• Scaling: Performance scaled effectively with larger models (Qwen3-32B) and increased candidate pool sizes ($N$), showing the framework is model-agnostic.

5. Significance and Impact

• Reliability in Healthcare: MA-RAG offers a pathway to safer, evidence-grounded medical AI by actively resolving uncertainties through iterative retrieval rather than relying on static parametric knowledge.
• Efficiency: By triggering retrieval only when semantic conflicts are detected, the framework avoids the computational waste of indiscriminate multi-round retrieval.
• Generalizability: While focused on medicine, the principle of using "conflict as a signal for refinement" is applicable to any domain requiring high-fidelity, complex reasoning.
• Future Directions: The authors suggest extending the framework to incorporate web-scale search and structured databases, and improving the ranking agent's evaluation metrics to approach the "Oracle" upper bound.

In summary, MA-RAG represents a shift from passive retrieval to active, conflict-driven reasoning, effectively transforming the "noise" of model disagreement into a structured signal for knowledge acquisition and consensus building.