💻 computer science

Timely Clinical Diagnosis through Active Test Selection

The paper introduces ACTMED, a diagnostic framework that combines Bayesian Experimental Design with large language models to enable adaptive, resource-aware test selection that emulates real-world clinical reasoning, improves diagnostic accuracy, and maintains clinician oversight without requiring extensive task-specific training data.

Original authors: Silas Ruhrberg Estévez, Nicolás Astorga, Mihaela van der Schaar

Published 2026-09-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Silas Ruhrberg Estévez, Nicolás Astorga, Mihaela van der Schaar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the daily rhythm of a doctor's visit, diagnosis is rarely a single moment of revelation but a careful, step-by-step journey. A physician gathers a patient's history, listens to their symptoms, and then faces a vast menu of potential tests. The challenge is not just knowing which tests exist, but deciding which ones to order next. Ordering too many wastes time and money, while ordering the wrong ones can delay the truth or miss a critical clue. This balancing act requires a form of reasoning that weighs the value of new information against the cost of obtaining it. For decades, medical guidelines have offered static checklists to help, but these rules are often built for the average patient, not the individual standing in the exam room. As artificial intelligence enters the field, a new question arises: can a computer learn to think like a doctor, navigating uncertainty and choosing the most helpful next step without needing a massive library of pre-written rules?

A team of researchers at the University of Cambridge has developed a new approach called ACTMED to answer this question. Rather than building a system that simply looks at a full set of data and spits out a final answer, they created a framework that mimics the active, iterative process of human diagnosis. The system treats the diagnostic process as a series of choices. At each step, it asks: "If we run this specific test, how much will it change our confidence in what is wrong with the patient?" To answer this, the system uses a large language model—a type of artificial intelligence trained on vast amounts of text—as a flexible simulator. Instead of needing a rigid mathematical formula for how diseases behave, the system asks the language model to imagine what a test result might look like for a specific patient, based on everything it already knows. It then runs this simulation many times to see how different possible outcomes would shift the probability of a diagnosis.

The core innovation lies in how the system decides when to stop. It does not simply pick the test that seems most obvious or the one that is cheapest. Instead, it calculates which test is expected to provide the greatest reduction in uncertainty. If a test is likely to confirm what the doctor already suspects with high certainty, the system may skip it. If a test is likely to clear up a confusing picture, the system prioritizes it. This process continues until the system reaches a point of sufficient confidence, at which stage it stops asking for more data. This mimics the way a skilled clinician knows when they have gathered enough evidence to make a decision, avoiding the trap of unnecessary procedures. The researchers tested this method on real-world medical data involving three distinct conditions: chronic kidney disease, hepatitis C, and diabetes. In these simulations, the system was limited to a small number of tests, just as a real doctor might be constrained by budget or time.

The results showed that this adaptive approach outperformed several other methods. In the case of hepatitis and diabetes, the system actually achieved higher diagnostic accuracy than a model that was allowed to see all available test results at once. This counter-intuitive finding suggests that by carefully selecting only the most informative tests, the system avoided the noise and confusion that can come from looking at too much data at once. The researchers also found that the system was highly efficient, often reaching a confident diagnosis with fewer than two tests on average, whereas other methods required the full set of three. When human doctors reviewed the system's choices, they found the reasoning to be clinically sound in nearly 95 percent of cases. The doctors noted that the system's step-by-step logic felt familiar, reflecting their own thought process when weighing uncertainty.

The study also highlighted the limitations of simply asking a large language model to guess a diagnosis directly. When the researchers let the models choose tests without the structured, probabilistic framework, the models tended to pick the same few tests for every patient, ignoring the unique details of the individual case. By contrast, the new framework forced the system to consider the specific context of each patient, leading to more personalized and effective decisions. The researchers emphasize that this tool is designed to work alongside a human doctor, not to replace them. It serves as a decision-support partner, offering suggestions on which tests to run next and explaining why those tests matter. While the current version works best with structured data like blood test numbers, the team sees a future where such systems could help manage the growing complexity of modern medicine, ensuring that every test ordered brings a doctor closer to the truth.

Technical Summary: ACTMED – Timely Clinical Diagnosis through Active Test Selection

Problem Statement
Clinical diagnosis is inherently sequential and resource-constrained, yet most existing machine learning (ML) approaches rely on static, fully observed datasets. These models fail to reflect the iterative, context-aware reasoning clinicians use in practice, often assuming complete data availability or treating diagnosis as a static classification task. This leads to inefficiencies, including unnecessary testing (estimated at 40–60% of cases) and delays in high-pressure or resource-limited settings. Furthermore, while Large Language Models (LLMs) show promise in medical decision-making, their direct deployment faces challenges regarding transparency, interpretability, and adherence to probabilistic reasoning. There is a critical gap for a framework that can dynamically reason under uncertainty, optimize test selection based on expected utility, and align with clinician workflows without requiring extensive task-specific training data.

Methodology: ACTMED
The authors propose ACTMED (Adaptive Clinical Test selection via Model-based Experimental Design), a diagnostic framework that integrates Bayesian Experimental Design (BED) with Large Language Models (LLMs). The core objective is to select the next diagnostic test that maximizes the reduction of epistemic uncertainty (knowledge-based uncertainty) relative to its cost.

Agent-Based Diagnosis Model:
The system operates over a discrete time horizon $T$ . At each step $t$ , an agent observes a subset of ground truth information $K_t$ and must select a test $U'_t$ from a set of available tests $U_t$ . The agent's goal is to minimize a Lagrangian objective that balances diagnostic error against the cost of information acquisition:
$L(y, u_t, \lambda) = \sum_t I[y \neq y_{d_t}] + \lambda \sum_t c(u_t)$
where $c(u_t)$ is the cost of the test and $\lambda$ controls the trade-off.
LLM as a Surrogate Sampler:
Since explicit mechanistic models of physiological dynamics are often unavailable in closed form, ACTMED utilizes LLMs as flexible, generative surrogate models. Given a patient's current knowledge base $K_t$ , the LLM acts as a conditional sampler to generate plausible realizations of unobserved test outcomes $u^{(i,j)}_t \sim P(U^{(i)}_t | K_t)$ . This allows the system to approximate the conditional distribution of test results without task-specific fine-tuning or structured training data.
KL-Divergence Based Test Selection:
To determine the utility of a candidate test, ACTMED computes the expected Kullback–Leibler (KL) divergence between the prior belief and the posterior belief conditioned on hypothetical test outcomes.
- For each candidate test, the LLM samples $M$ hypothetical outcomes.
- For each outcome, a posterior probability distribution is estimated.
- The expected information gain is calculated as the average KL divergence:
  $E[KL(B(p_{post}) \parallel B(p_{prior}))] = \frac{1}{M} \sum_{j=1}^M \left( p^{(j)}_{post} \log \frac{p^{(j)}_{post}}{p^{(j)}_{prior}} + (1 - p^{(j)}_{post}) \log \frac{1 - p^{(j)}_{post}}{1 - p^{(j)}_{prior}} \right)$
  The test with the highest expected KL divergence (normalized by cost) is selected.
Adaptive Stopping Criterion:
The framework employs a principled stopping rule based on the "confidence gap" $\delta = |p_{prior} - \theta|$ , where $\theta$ is a decision threshold (e.g., 0.5). Testing continues only if the expected KL divergence of a new test is sufficient to shift the belief toward a target posterior $q_{target} = \theta \pm \gamma\delta$ . If no remaining test is expected to meaningfully update the belief, acquisition stops.
Clinician-in-the-Loop:
The system is designed to support, not replace, clinicians. It provides step-by-step test suggestions, simulated outcomes, and updated risk probabilities, allowing clinicians to review, interpret, and override recommendations based on clinical judgment.

Key Contributions

Formalization: The authors formalize a transparent, stepwise diagnostic framework that aligns with clinical reasoning, moving away from static classification to sequential, resource-aware decision-making.
ACTMED Framework: They introduce a probabilistic approach using LLMs as surrogate simulators within a BED framework to adaptively select tests based on expected diagnostic utility.
Reasoning Shift: The paper demonstrates that shifting reasoning from the LLM's internal latent space to the natural language output space (via structured sampling and KL calculation) improves clinical decision-making.
Validation: The framework is validated on real-world datasets, showing improvements in diagnostic accuracy, interpretability, and resource efficiency.

Experimental Results
The authors evaluated ACTMED on three single-condition datasets (Chronic Kidney Disease, Hepatitis C, Diabetes) and a custom multi-condition OSCE-style dataset derived from AgentClinic.

Surrogate Fidelity: LLMs (specifically GPT-4o) demonstrated high fidelity in approximating the distributions of diagnostic test outcomes, achieving low Wasserstein and Energy distances compared to empirical data.
Diagnostic Accuracy: ACTMED consistently outperformed baselines, including random selection, global best fixed subsets, and implicit LLM selection. Notably, in Hepatitis and Diabetes tasks, ACTMED surpassed the "All Features" baseline, suggesting that targeted test selection reduces noise and overfitting.
Efficiency: The KL-based stopping criterion significantly reduced the number of tests required. At conservative thresholds, the average number of tests was reduced to under two (from a limit of three) while maintaining or improving accuracy.
Personalization: Unlike implicit selection methods that tended to choose globally optimal features regardless of patient specifics, ACTMED adapted test selection to individual patient uncertainty.
Clinician Evaluation: In a study involving experienced clinicians, 94.5% of ACTMED's test selections and risk adjustments were judged as clinically reasonable. Clinicians praised the transparency of the step-by-step process but noted that real-world protocols sometimes override purely Bayesian rationales.

Significance and Claims
The paper claims that ACTMED represents a step toward transparent, adaptive, and clinician-aligned diagnostic systems. Its primary significance lies in:

Generalizability: The framework operates in a zero-shot manner across different diseases without requiring domain-specific training data for each condition.
Resource Awareness: It explicitly optimizes for cost and time, addressing the global challenge of resource-limited settings.
Interpretability: By exposing the intermediate reasoning (simulated outcomes and belief updates), it mitigates the "black box" nature of LLMs, fostering trust and allowing for human oversight.
Complementarity: The authors emphasize that current LLMs are not ready for autonomous diagnosis; instead, ACTMED positions them as decision-support tools that structure and refine the diagnostic process, reducing cognitive burden and unnecessary testing.

The authors acknowledge limitations, including the current reliance on binary classification, the need for high-capacity LLMs to generate coherent samples, and the constraint of using structured numerical/categorical data rather than free-text reports. They conclude that while the framework improves data-efficient decision-making, future work must address multimodal data and multi-label comorbidities to fully reflect real-world clinical complexity.

Technical Summary: ACTMED – Timely Clinical Diagnosis through Active Test Selection

More like this