💬 NLP

SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning

This paper proposes SEMA-RAG, a self-evolving multi-agent retrieval-augmented generation framework that decouples medical reasoning into specialized interpretation, iterative exploration, and adjudication roles to significantly improve accuracy and mitigate hallucinations in medical question answering.

Original authors: Yongfeng Huang, Ruiying Chen, James Cheng

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yongfeng Huang, Ruiying Chen, James Cheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "One-Shot" Doctor

Imagine you are a doctor in a busy hospital. A patient walks in with a complex set of symptoms.

The Old Way (Standard RAG):
You are handed a single, thick medical textbook. You have exactly one chance to look up a keyword related to the patient’s symptoms. You flip to a page, read a paragraph, and immediately have to write down your diagnosis. You cannot go back to check another chapter, and you cannot ask yourself, "Wait, did I miss something about their age or history?"

This is how most current AI medical assistants work. They take a question, do one quick search, and guess the answer. This often leads to mistakes because:

Bad Translation: The AI might search for the wrong keywords because it didn’t fully understand the subtle clues in the question.
No Second Thoughts: If the first page it finds doesn’t have the full answer, it doesn’t know to keep looking. It just guesses with incomplete information.

The Solution: SEMA-RAG (The Specialist Team)

The authors of this paper argue that clinical reasoning isn’t a one-step process; it’s a multi-stage investigation. To fix this, they created SEMA-RAG, which replaces the "one-shot" approach with a team of three specialized AI agents working together.

Think of it like a high-end detective agency solving a medical mystery. Instead of one person trying to do everything, they have three specialists:

1. The Interpreter (The Translator)

Role: This agent doesn’t try to solve the case yet. Its only job is to understand the question deeply.
What it does: It takes the patient’s messy, natural-language question and breaks it down into a structured "clinical schema." It identifies the core intent (e.g., "We need to find the cause of infection"), the key entities (e.g., "stroke patient," "hospital day 7"), and the constraints (e.g., "late-onset").
Analogy: Imagine a detective who reads the witness statement and writes a clear, bulleted list of clues on a whiteboard before anyone starts searching for suspects. This ensures the search team knows exactly what to look for, rather than wandering aimlessly.

2. The Explorer (The Researcher)

Role: This agent is responsible for gathering evidence.
What it does: It doesn’t just search once. It uses the clues from the Interpreter to search the medical database. Then, it pauses and asks: "Do we have enough proof?"
- If No: It identifies what’s missing (the "gap") and generates new, targeted questions to fill that gap. It searches again. It repeats this loop until it feels the evidence is sufficient.
- If Yes: It stops searching and hands the files to the next agent.
Analogy: This is the field investigator. They don’t just grab the first file they see. They check the evidence, realize, "Hmm, we know it’s pneumonia, but we don’t know if it’s hospital-acquired or community-acquired," and then go back to the archives to find specific records about hospital-acquired infections. They keep digging until the picture is clear.

3. The Arbiter (The Judge)

Role: This agent makes the final decision.
What it does: It receives all the evidence gathered by the Explorer. It reviews the documents, resolves any contradictions (e.g., one source says X, another says Y), and writes a final report. It then selects the correct answer from the multiple-choice options.
Analogy: This is the senior partner or judge. They don’t do the legwork. They sit at the desk, review the complete file prepared by the researcher, weigh the pros and cons, and deliver the final verdict.

Why It Works Better

The paper shows that this "divide and conquer" strategy works significantly better than the old method.

Self-Evolving: The system "evolves" during the test. It doesn’t follow a rigid script. If the first search is weak, it adapts and searches again. If the first search is strong, it stops early to save time.
Better Accuracy: Across five different medical benchmarks and five different AI models, SEMA-RAG improved accuracy by an average of 6.46 points compared to the best existing methods.
Efficiency: Even though it does more steps, it’s smarter about it. It doesn’t waste time searching endlessly if it already has the answer.

A Real-World Example from the Paper

The paper gives a specific example to illustrate this:

The Question: A 62-year-old stroke patient develops a fever on hospital day 7. What is the likely cause?

The Old AI (MedRAG): It sees "pneumonia" and "stroke." It searches for general pneumonia causes. It finds that Streptococcus pneumoniae is the most common cause of pneumonia in general. It guesses that answer. Result: Wrong.
The SEMA-RAG Team:
1. Interpreter: Notes the critical constraint: "Hospital Day 7." This implies the infection was acquired in the hospital, not from the community.
2. Explorer: Searches for pneumonia. Realizes the evidence doesn’t distinguish between hospital-acquired and community-acquired. It generates a new query: "What are the common pathogens for hospital-acquired pneumonia after 5 days?" It finds that Staphylococcus aureus is common in this specific scenario.
3. Arbiter: Reviews the evidence. Sees that Staphylococcus aureus matches the "hospital day 7" constraint. Selects that as the answer. Result: Correct.

Summary

SEMA-RAG stops treating medical AI like a simple search engine. Instead, it treats it like a collaborative team that interprets the problem, iteratively gathers the right evidence, and carefully judges the final answer. This mimics how real doctors think: they don’t just guess; they investigate, refine their understanding, and then decide.

Technical Summary: SEMA-RAG

Problem Statement
Retrieval-Augmented Generation (RAG) is widely used to mitigate hallucinations and knowledge obsolescence in medical question answering. However, standard RAG frameworks typically employ a static, single-round retrieval paradigm that misaligns with the multi-stage nature of clinical reasoning. Clinicians typically interpret patient narratives, progressively gather and verify information to address evidence gaps, and weigh contradictory evidence before forming judgments. In contrast, single-round static RAG compresses this process, leading to two structural deficiencies: (1) the translation from question to query often lacks clinically grounded semantic interpretation, making implicit constraints difficult to articulate; and (2) the retrieval process lacks iterative sufficiency feedback, hindering the formation of reliable evidence chains. The authors argue these issues stem from overloading a single reasoning chain with heterogeneous tasks (interpretation, exploration, and adjudication), causing cognitive overload and interdependent steps that prevent timely adjustment when evidence is insufficient.

Methodology
To address these limitations, the authors propose SEMA-RAG (Self-Evolving Multi-Agent RAG), a framework that reconstructs the workflow via task decoupling and dynamic multi-round exploration. The system utilizes three specialist agents, all sharing the same underlying Large Language Model (LLM) but differentiated by role-specific prompts:

Interpreter Agent (I-Agent): This agent maps unstructured medical questions ( $Q$ ) into a structured Clinical Schema tuple ( $Q'$ ). This tuple consists of four components: clinical intent ( $o_{int}$ ), medical entities ( $o_{ent}$ ), clinical constraints ( $o_{cons}$ ), and an initial retrieval query ( $q_{init}$ ). The schema is linearized into a retrieval-ready query string ( $\hat{q}_{init}$ ) that explicitly incorporates intent, entities, and constraints, providing stable anchors for subsequent retrieval and reducing semantic drift.
Explorer Agent (E-Agent): This agent performs self-evolving, evidence-sufficiency-driven retrieval. Starting with $\hat{q}_{init}$ , the E-Agent iteratively retrieves documents from a dense vector space. After each round $t$ , it evaluates the accumulated evidence set ( $C_t$ ) against the clinical anchors ( $Q'$ ). It predicts a sufficiency flag ( $s_t$ ), a gap description ( $g_t$ ), and a next query set ( $Q_{t+1}$ ). If evidence is insufficient ( $s_t=0$ ), it generates targeted follow-up queries to fill specific gaps; if sufficient ( $s_t=1$ ), it terminates exploration. This loop continues until sufficiency is reached, a maximum iteration limit ( $T_{max}$ ) is hit, or stagnation occurs, resulting in a converged evidence set ( $C^*$ ).
Arbiter Agent (A-Agent): This agent performs evidence adjudication and answer selection. It organizes the converged evidence set ( $C^*$ ) into a traceable evidence report ( $R$ ), resolving conflicts and removing irrelevant content. Finally, it selects the discrete answer ( $\tilde{y}$ ) from the candidate set based on the grounded report $R$ .

Key Contributions

Framework Proposal: Introduction of SEMA-RAG, a multi-agent RAG framework that models clinical reasoning through role division (Interpret-Explore-Adjudicate) and collaboration.
Self-Evolving Mechanism: Development of an Explorer Agent that dynamically updates queries based on identified evidence gaps, steering retrieval toward medical reasoning objectives rather than relying on fixed iteration counts.
Empirical Validation: Comprehensive evaluation across five medical Q&A benchmarks and five LLM backbones, demonstrating consistent improvements over strong baselines.

Results
The authors evaluated SEMA-RAG on five benchmarks: MMLU-Med, MedQA-US, MedMCQA, PubMedQA*, and BioASQ-Y/N, using five LLM backbones (DeepSeek-v3.1, Kimi-k2, Qwen3-Coder-Plus, Gemini-2.0-Flash, and GLM-4.0-Flash).

Performance: SEMA-RAG consistently outperformed representative baselines (including CoT, MedCPT, MedRAG, and i-MedRAG). Across all benchmarks and backbones, it improved the strongest baseline by an average of +6.46 accuracy points.
Component Analysis: Ablation studies confirmed the necessity of each agent. Removing the I-Agent degraded performance due to underspecified queries; removing the E-Agent caused the largest drop, highlighting the importance of sufficiency-driven closed-loop retrieval; removing the A-Agent reduced accuracy, showing the value of explicit evidence adjudication.
Efficiency: While SEMA-RAG incurs higher inference costs than single-round methods due to multi-agent interaction, it achieved a more favorable accuracy-efficiency trade-off compared to the iterative baseline i-MedRAG. The sufficiency-driven early stopping mechanism allowed SEMA-RAG to allocate computation more effectively than fixed-step iteration, often converging within two rounds ( $T_{max}=2$ ).
Case Study: Qualitative analysis on MedQA-US demonstrated that SEMA-RAG correctly identified critical temporal constraints (e.g., "hospital day 7") to distinguish between community-acquired and hospital-acquired pneumonia, whereas single-round baselines failed to anchor retrieval to these constraints, leading to incorrect diagnoses.

Significance and Limitations
The paper claims that SEMA-RAG validates the hypothesis that restructuring RAG to match phased clinical workflows—via task decoupling and self-evolving retrieval—leads to more reliable evidence-chain construction. The framework demonstrates that medical RAG should move beyond static single-round retrieval toward adaptive evidence construction.

The authors acknowledge several limitations:

The study is limited to benchmark-based settings and does not cover realistic clinical workflows such as longitudinal Electronic Health Record (EHR) reasoning.
Performance depends on the quality and coverage of the retrieval corpus; missing or outdated evidence may lead to incomplete grounding.
The current sufficiency criterion is not explicitly designed for option-level separability or generative completeness.
The framework introduces additional inference costs compared to single-round methods, though it remains more efficient than fixed-step iterative baselines.