💬 NLP

SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning

The paper introduces SSE-Bio, a structured self-evolving agent that utilizes a trainable proxy policy and fine-grained template editing to optimize retrieval decisions and improve multi-hop biomedical reasoning, outperforming existing baselines on key benchmarks.

Original authors: Zhaohan Meng, Zaiqiao Meng, Siwei Liu, Hao Xu, Ke Yuan, Iadh Ounis

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhaohan Meng, Zaiqiao Meng, Siwei Liu, Hao Xu, Ke Yuan, Iadh Ounis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of modern medicine, discovering new treatments or understanding complex diseases often feels like solving a puzzle where the pieces are scattered across different fields of knowledge. A doctor might know that a specific drug treats a certain symptom, and a researcher might know that a particular protein causes that symptom, but connecting the drug to the protein requires bridging a gap that is not always obvious. This process, known as multi-hop reasoning, is essential for medical discovery, yet it is notoriously difficult for computers to master. Traditional artificial intelligence systems often struggle with this because they tend to follow rigid, pre-programmed paths or make broad guesses that can lead them astray when the evidence is complex. If a computer misses a single crucial link in the chain of logic, the entire conclusion can collapse, potentially leading to incomplete or incorrect medical advice.

To address this challenge, a team of researchers has developed a new type of artificial intelligence agent called SSE-Bio, designed specifically to navigate these complex medical puzzles. Unlike older systems that rely on static instructions or make sweeping changes to their own memory when they make a mistake, this new agent operates with a more disciplined approach. It maintains a clear, structured record of its current thinking process and uses a specialized decision-maker to choose exactly when and what information to look up. Instead of rewriting its entire set of rules from scratch after every attempt, it makes small, precise edits to its memory based on what worked. This allows the system to learn from its experiences without losing its way or inventing facts that aren't there.

The core innovation of SSE-Bio lies in how it manages its memory and its search for information. The system is built around a central planner that keeps a structured state, which acts like a short-term notebook tracking exactly where the reasoning stands at any given moment. When the system needs to solve a question, this planner consults a long-term library of past successful solutions, known as templates. However, the system does not blindly copy these old solutions. Instead, a dedicated component, acting as a retrieval policy, decides whether to fetch specific facts about diseases and drugs or to pull up a general strategy from the past. This decision is made carefully for each step of the reasoning process, ensuring that the system only gathers the evidence it truly needs. If the system makes an error, it does not discard its entire memory; rather, it performs a fine-grained edit, updating only the specific part of the template that was incorrect, leaving the rest of the knowledge intact.

To teach this agent how to make the best decisions about what to retrieve, the researchers used a training method that involves comparing different paths the system could take. They created scenarios where the agent would try different combinations of looking up facts versus looking up strategies. By analyzing which path led to the correct answer and which led to a dead end, the system learned to recognize the signs that indicated it needed more information. This training process was rigorous, focusing on both getting the final answer right and ensuring that every step of the reasoning was supported by actual evidence. The result is an agent that is not only accurate but also transparent in how it reaches its conclusions, as it can point to the specific pieces of evidence it used to build its argument.

When tested on three different benchmarks designed to evaluate medical reasoning, SSE-Bio consistently outperformed existing systems. On a dataset called BioHopR, which contains thousands of complex medical questions, the new agent improved its accuracy by a significant margin compared to the previous best-performing self-evolving system. It showed particular strength in handling questions that required finding multiple correct answers, a task where many other systems tend to give up too early or provide incomplete lists. The system also demonstrated an ability to generalize its skills, performing well on medical questions it had never seen before, suggesting that its method of learning is robust and adaptable.

The researchers found that the system's success was not due to any single feature but rather the combination of its structured memory and its intelligent retrieval policy. When they removed the ability to make fine-grained edits to its memory, or when they removed the component that decided what to search for, the system's performance dropped noticeably. This confirmed that the ability to update its knowledge precisely and to choose its information sources wisely were both essential for its success. The study also revealed that the system still has room to grow; even with its current capabilities, providing it with perfect information would allow it to perform even better, indicating that the bottleneck is not in its reasoning logic but in the quality of the information it can access.

In the broader context of artificial intelligence, this work represents a shift away from systems that try to learn everything by rewriting their own instructions in broad strokes. Instead, SSE-Bio demonstrates that a more controlled, structured approach to memory and information retrieval can lead to more reliable and trustworthy results. By keeping a clear record of its reasoning steps and making small, targeted adjustments to its knowledge base, the system avoids the confusion and errors that often plague more chaotic learning methods. This approach offers a promising path forward for building artificial intelligence that can assist in high-stakes fields like medicine, where accuracy and the ability to explain one's reasoning are just as important as the answer itself. The researchers acknowledge that while their system is a significant step forward, there is still work to be done in refining how it accesses information and managing its growing library of knowledge to ensure it remains efficient and up-to-date.

Technical Summary: SSE-Bio

Problem Statement

Biomedical multi-hop question answering (QA) requires models to reason across complex chains of entities (e.g., diseases, drugs, proteins) to connect evidence. While Retrieval-Augmented Generation (RAG) and multi-agent systems have improved performance, existing approaches face two critical limitations:

Static Retrieval Workflows: Most agents rely on fixed retrieval schedules or coarse-grained prompt rewriting rather than explicitly learned retrieval control, leading to sub-optimal evidence selection.
Instruction Drift in Memory Updates: Self-evolving agents often update their reasoning memory through global prompt rewriting. This coarse-grained approach can alter the entire reasoning scaffold due to local failures, causing instruction drift and increasing the risk of hallucinations.

These limitations result in unresolved intermediate "bridge" entities, incomplete answers in multi-answer scenarios, and a lack of adaptability in reasoning strategies across different questions.

Methodology: SSE-Bio

The authors propose SSE-Bio, a structured self-evolving agent designed for multi-hop biomedical reasoning. It operates within a unified agentic framework comprising four components: a Manager, a Proxy, a Dev agent, and a Critic.

1. Structured Memory Architecture

Unlike free-form text generation, SSE-Bio maintains two distinct forms of memory:

Short-Term Memory (Structured State): The Manager maintains a structured state ( $u_t$ ) for the current question, capturing the task type, information gaps, feedback from the Critic, and retrieval history. This explicit structure grounds retrieval decisions and planning.
Long-Term Memory (Template Evolution): The system stores reusable reasoning templates distilled from successful trajectories. Crucially, SSE-Bio employs fine-grained template editing rather than global rewriting. When a trajectory aligns with a retrieved template, only specific local fields (e.g., verification criteria, failure warnings) are updated. This constrains memory evolution to prevent instruction drift.

2. Agentic Retrieval Policy

A trainable Proxy model explicitly controls the retrieval process at each reasoning step. Based on the structured state, the Proxy decides whether to retrieve:

Knowledge Triplets: Factual biomedical relations (e.g., Disease-Drug-Phenotype) to support step-specific reasoning.
Prior Templates: Reusable reasoning flows from long-term memory to guide the planning process.
The Proxy selects from four actions: retrieving neither, only triplets, only templates, or both.

3. Training Strategy: GRPO with Decision-Contrastive Trajectories

The Proxy is the only trainable component; the Manager, Dev, Critic, and retrievers remain fixed. The training follows a two-stage process:

Supervised Fine-Tuning (SFT): Initializes the Proxy using pseudo-labels derived from comparing alternative retrieval branches to identify decisions leading to better outcomes.
Group Relative Policy Optimization (GRPO): The Proxy is refined using GRPO. The authors introduce decision-contrastive trajectory generation, where alternative retrieval actions are expanded from the same reasoning state into contrastive rollout branches.
Reward Shaping: The optimization uses an answer-grounded composite reward ( $R$ $R$ ) combining:
- Outcome Reward ( $R_{out}$ ): Based on final answer correctness.
- Behavioral Reward ( $R_{beh}$ ): Measures whether key reasoning steps (e.g., bridge resolution) are supported by retrieved evidence or templates, encouraging grounded reasoning over mere retrieval volume.

Key Contributions

Unified Agentic Framework: SSE-Bio integrates short-term structured state tracking, long-term template memory, and explicit retrieval management into a single system.
Fine-Grained Template Evolution: The paper introduces a mechanism for updating retrieved templates locally from successful trajectories, reducing the risks of instruction drift associated with whole-template rewriting.
GRPO-Based Retrieval Optimization: A novel training strategy for the Proxy that couples decision-contrastive trajectory generation with answer-grounded rewards to improve retrieval decisions under matched reasoning contexts.

Experimental Results

The authors evaluated SSE-Bio on three biomedical benchmarks: BioHopR, MedHop, and Humanity's Last Exam: Biomedicine (HLE).

Performance on BioHopR: SSE-Bio consistently outperformed strong baselines, including general LLMs, medical LLMs, RAG models, and existing self-evolving agents (e.g., STELLA).
- On the single-answer setting, SSE-Bio improved the Bothcor metric (proportion of pairs where both 1-hop and 2-hop questions are correct) by 6.56 absolute points over the strongest self-evolving baseline (STELLA), rising from 9.96% to 16.52%.
- In the multi-answer setting, SSE-Bio achieved the highest scores across all metrics, demonstrating robustness in retrieving multiple valid answers.
Generalization: The model showed strong zero-shot transfer capabilities to MedHop and HLE, outperforming agentic baselines without task-specific fine-tuning.
Ablation Studies:
- Removing the Proxy (fixed retrieval) significantly degraded performance, confirming the necessity of learned retrieval control.
- Replacing structured state and fine-grained editing with free-form state and coarse rewriting reduced performance, validating the design choices for memory and state management.
- Oracle experiments indicated that while SSE-Bio learns effective retrieval, there remains measurable headroom for improvement with stronger evidence and template matching.

Significance and Claims

The paper claims that combining structured retrieval control with fine-grained self-evolution offers a more controllable approach to biomedical reasoning agents. By explicitly managing retrieval decisions through a trainable Proxy and evolving memory through local editing rather than global rewriting, SSE-Bio mitigates the risks of instruction drift and hallucination. The results suggest that this architecture enables adaptive reasoning that is both evidence-grounded and capable of long-term strategy adaptation, addressing key limitations in current static or coarse-grained self-evolving systems.

The authors acknowledge limitations, including reliance on current retrieval quality, potential memory redundancy over time, and the open question of how further scaling the Proxy backbone might impact performance. They also note that the system currently relies on fixed commercial agents for the Manager, Dev, and Critic roles.