💻 computer science

An AI agent for treatment reasoning over a biomedical tool universe

The paper introduces ATHENA-R1, an AI agent trained via a two-level self-learning framework with reinforcement learning over 212 biomedical tools, which achieves state-of-the-art accuracy in treatment reasoning by iteratively gathering evidence and outperforms existing models in both benchmark tasks and expert evaluations.

Original authors: Shanghua Gao, Ayush Noori, Richard Zhu, Curtis Ginder, Zhenglun Kong, Xiaorui Su, Justin Kauffman, Benjamin S. Glicksberg, Joshua Lampert, Ankit Sakhuja, Ashwin Sawant, ATHENA-R1 Evaluation Consortium

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shanghua Gao, Ayush Noori, Richard Zhu, Curtis Ginder, Zhenglun Kong, Xiaorui Su, Justin Kauffman, Benjamin S. Glicksberg, Joshua Lampert, Ankit Sakhuja, Ashwin Sawant, ATHENA-R1 Evaluation Consortium, David A. Clifton, Noa Dagan, Ran Balicer, Marinka Zitnik

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a very complex medical mystery. You have a patient with a specific set of symptoms, other health issues, and a list of medications they are already taking. Your job is to figure out the best treatment.

In the past, AI models tried to solve this by acting like a super-remembering encyclopedia. They would try to recall everything they "learned" during their training to give an answer. But just like a human who hasn't read the latest news, these encyclopedias often missed new rules, forgot about specific drug interactions, or gave advice that was safe for one person but dangerous for another.

Enter ATHENA-R1: The AI Detective with a Live Toolkit

The paper introduces a new AI agent called ATHENA-R1. Instead of just relying on its memory, ATHENA-R1 is like a detective who carries a giant, live-updated toolbox containing 212 different medical reference books, databases, and safety manuals.

Here is how it works, using a simple analogy:

1. The "Self-Learning" Internship

Before ATHENA-R1 could solve real cases, it needed to learn how to think, not just what to think.

The Problem: Humans can't write down millions of examples of "how to solve a medical case" because it takes too long and is too complicated.
The Solution: The researchers built a team of AI "interns." These interns created their own practice cases, made up their own toolkits, and wrote out their own step-by-step reasoning guides.
The Result: ATHENA-R1 studied these millions of AI-generated guides. It learned that to solve a problem, it shouldn't just guess; it should ask: "What information am I missing? Which tool do I need to open? What does that tool say? Does that change my plan?"

2. The "Iterative" Investigation

When ATHENA-R1 faces a real patient case, it doesn't spit out an answer immediately. It plays a game of "20 Questions" with its own toolbox.

Step 1: It looks at the patient and says, "I need to check if this drug interacts with their other meds." Click! It opens a tool to check interactions.
Step 2: The tool says, "Warning: This causes kidney stress." ATHENA-R1 pauses and says, "Okay, I need to check the patient's kidney function." Click! It opens a second tool to check lab results.
Step 3: Based on the new info, it might say, "This drug is too risky. Let's try a different one."
The Magic: It keeps doing this, gathering evidence piece by piece, until it has a complete picture. It writes down every step of its investigation, so a human can see exactly why it made the decision.

3. The "Test Drive" Results

The researchers put ATHENA-R1 through five different "driving tests" to see if it was better than other AI models (like GPT-5 or DeepSeek-R1).

The Drug Quiz: They asked it 3,000+ questions about drug labels (dosage, safety, side effects). ATHENA-R1 got 94.7% correct. The best other model got 76.9%.
- Why? The other models tried to remember the answers. ATHENA-R1 looked them up in the "live" database, ensuring it had the most current info.
The Patient Puzzle: They gave it 456 complex patient stories where the "right" drug depended on specific details (like pregnancy or kidney disease). ATHENA-R1 got 82.9% correct. The others struggled because they couldn't connect the dots between the patient's unique history and the drug rules.
The Expert Review: Doctors and rare disease experts (from 28 different organizations) looked at the AI's answers without knowing which AI wrote them. They preferred ATHENA-R1's answers almost every time.
- Why? They loved that ATHENA-R1 showed its work. It didn't just say "Take this pill"; it said, "Take this pill because I checked the label, saw it's safe for pregnant women, and confirmed it doesn't interact with her other meds."

4. The "Hypothesis Generator"

Finally, the researchers tested if ATHENA-R1 could spot hidden dangers. They asked it to imagine a patient with a specific mix of diseases and drugs, and guess what bad side effect might happen.

ATHENA-R1 guessed things like: "If a patient with gout and high blood pressure takes beta-blockers, they might be at higher risk for kidney failure."
The researchers then checked 5.4 million real patient records to see if this was true.
The Result: The records confirmed that these specific groups of patients did have higher risks for these issues. The AI had successfully connected dots that human doctors hadn't explicitly linked before.

The Bottom Line

The paper claims that treatment reasoning (figuring out the right medicine for a specific person) is too hard for AI to do just by "remembering" facts.

Instead, ATHENA-R1 proves that if you teach an AI to act like a researcher—to know what questions to ask, to use the right tools to find the answers, and to update its plan as it learns new facts—it can make much safer and more accurate medical decisions than models that just rely on memory. It turns medical decision-making from a "guessing game" into a "step-by-step evidence hunt."

Technical Summary: ATHENA-R1 – An AI Agent for Treatment Reasoning

Problem Statement

Treatment reasoning is a foundational yet demanding task in medicine, requiring the integration of disease context, patient characteristics, comorbidities, concurrent medications, contraindications, and evolving biomedical knowledge to determine appropriate therapies. Unlike simple fact retrieval or pattern recognition, treatment reasoning is inherently iterative: candidate strategies must be gathered, evaluated against multiple constraints, and revised as new evidence emerges.

Current Large Language Models (LLMs) rely primarily on parametric knowledge stored in model weights. While they generate fluent responses and capture broad clinical patterns, they often lack access to updated, vetted medical information and may produce recommendations that fail to account for specific contraindications or patient constraints. While Retrieval Augmented Generation (RAG) and tool-augmented LLMs provide access to external data, mere access to biomedical tools does not guarantee treatment reasoning. A model must autonomously determine what evidence is needed, select appropriate sources, interpret results in context, and revise analysis when evidence is incomplete or conflicting. This capacity cannot be assumed from tool access alone; it must be learned.

Methodology

The authors introduce ATHENA-R1, an AI agent designed to perform treatment reasoning across all FDA-approved drugs since 1939. The system operates by combining multi-step analysis with direct access to a universe of 212 biomedical tools.

Core Architecture

ATHENA-R1 does not produce answers in a single step. Instead, it iteratively:

Identifies missing information required for a specific treatment decision.
Selects and executes relevant tools from a library of 212 biomedical resources (e.g., querying drug interactions, contraindications, pharmacology, and disease phenotypes).
Retrieves and interprets evidence from curated sources, including FDA prescribing information and biomedical knowledge bases.
Updates its analysis based on the retrieved evidence before proceeding to the next step.

Training Framework: Two-Level Self-Learning

Generating the massive, diverse, and multi-step reasoning traces required for training is infeasible for human annotators. Consequently, ATHENA-R1 is trained through a two-level self-learning framework:

Level 1: Supervised Fine-Tuning (SFT) via Multi-Agent Construction
- A multi-agent system automatically constructs biomedical tools, treatment tasks, and reasoning trajectories.
- This process generates ATHENA-R1-Instruct, a dataset of 378,027 instruction-tuning samples derived from 85,340 reasoning traces.
- The dataset comprises 177,626 reasoning steps and 281,695 tool calls, grounded in FDA drug labels since 1939.
- This level teaches the model the structure of treatment reasoning, including problem decomposition, evidence retrieval, tool use, and evidence interpretation.
Level 2: Reinforcement Learning (RL) with Scientific Feedback
- The model is refined in a live environment with the 212-tool library.
- It generates multi-turn reasoning trajectories for each prompt.
- A reward function provides scientific feedback across six dimensions: answer correctness, output format validity, evidence gathering, multi-step reasoning, tool-argument grounding, and reasoning non-redundancy.
- Group Relative Policy Optimization (GRPO) is used to increase the probability of higher-scoring trajectories, refining the agent's evidence-seeking strategy and tool selection capabilities.

Key Contributions

ATHENA-R1 Agent: A novel AI agent capable of iterative treatment reasoning that integrates real-time evidence gathering with multi-step analysis, moving beyond reliance on parametric knowledge.
Self-Learning Framework: A scalable methodology for training treatment reasoning agents without human-annotated traces, utilizing a multi-agent system to generate training data and RL with scientific feedback to refine reasoning strategies.
Comprehensive Tool Universe: Integration and utilization of a 212-tool biomedical library covering indications, dosing, safety, pharmacology, and patient-specific constraints.
Evaluation Suite: Development of rigorous benchmarks (DrugPC and TreatmentPC) and real-world validation protocols involving rare disease experts, practicing physicians, and population-scale electronic health record (EHR) analysis.

Results

Benchmark Performance

ATHENA-R1 was evaluated on five datasets spanning 3,168 drug reasoning tasks and 456 patient treatment cases.

DrugPC (Drug Prescribing Cards): ATHENA-R1 achieved 94.7% accuracy on open-ended drug reasoning, outperforming GPT-5 (76.9%), DeepSeek-R1 (68.8%), and Qwen3 (48.7%).
TreatmentPC (Patient Treatment Selection): ATHENA-R1 achieved 82.9% accuracy, exceeding GPT-5 (72.2%) by 10.7 percentage points and DeepSeek-R1 (67.5%) by 15.4 percentage points.
Tool-Use Necessity: Models with tool access but without the learned capacity to reason over tool outputs performed poorly. For instance, when GPT-5 was given optional tool access, it invoked tools in only 1% of cases, and its accuracy dropped below its no-tool baseline. ATHENA-R1 invoked tools on every case, demonstrating that the ability to reason over tool outputs is the limiting factor, not tool availability.

Expert Evaluations

Rare Disease Context: In blinded evaluations involving 23 experts from 28 rare disease organizations, ATHENA-R1 was preferred over reference models across all eight evaluation criteria. The largest gains were observed in cognitive traceability (95.5% preference) and helpfulness of rationale (94.5% preference).
Clinical Cases: Practicing physicians rated ATHENA-R1 favorably on complex hospitalized patient cases involving cardiovascular management and infectious disease, particularly in scenarios requiring the weighing of competing physiological constraints (e.g., post-CABG with CKD, anticoagulated patients with infections).

Population-Scale Validation

The authors tested adverse event hypotheses generated by ATHENA-R1 using longitudinal EHR data from 5.4 million patients.

Hypothesis Generation: The system prioritized predictions where prior pharmacovigilance evidence was limited.
Statistical Validation: Predicted associations reached adjusted odds ratios (OR) of 1.48–1.84 in high-risk subpopulations.
- Example: Increased risk of acute kidney failure (OR 1.84) and hyperkalemia (OR 1.78) in patients with hypertension and gout treated with $\beta$ -blockers.
- Example: Increased risk of hepatocellular carcinoma (OR 1.48) in patients with diabetes and ischemic heart disease treated with DPP-4 inhibitors.
Controls: Negative controls showed no elevation in risk, and positive controls successfully recovered established clinical effects, validating the pipeline's ability to distinguish plausible signals from noise.

Significance and Claims

The paper posits that treatment reasoning, long considered difficult for AI because it requires knowing what evidence to seek before a conclusion can be formed, can be reframed as a learnable process of iterative evidence gathering.

Key claims include:

Separation of Reasoning and Knowledge: By separating reasoning from knowledge storage, ATHENA-R1 can incorporate new evidence while maintaining a transparent record of how conclusions are formed, grounding decisions in retrievable sources rather than static model weights.
Effectiveness of RL over Scaling: The consistent performance of ATHENA-R1 (based on an 8B parameter model) above significantly larger models like DeepSeek-R1 (671B parameters) suggests that targeted tool-use training and reinforcement learning may be more effective than scaling alone for tasks requiring iterative evidence gathering.
Reframing AI in Medicine: The results establish that reinforcement learning can train AI to perform the evidence-seeking process that underlies treatment decisions, moving beyond simple fact retrieval to complex, context-aware clinical reasoning.

The authors note limitations, including dependence on tool library coverage, the lack of explicit uncertainty quantification, and the reliance on generated rather than human-written traces. They frame ATHENA-R1 not as a point-of-care reference tool, but as a research system demonstrating that AI can learn the iterative processes required for therapeutic decision-making.