💬 NLP

Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs

This paper proposes a semantic verification framework and new metrics to evaluate the stability of 16 clinical and general-purpose LLMs against meaning-preserving prompt variations, revealing that domain specialization does not consistently guarantee greater robustness in healthcare settings.

Original authors: Mahdi Alkaeed, Adnan Qayyum, Nabeel Abo Kashreef, Muhammad Bilal, Junaid Qadir

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mahdi Alkaeed, Adnan Qayyum, Nabeel Abo Kashreef, Muhammad Bilal, Junaid Qadir

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Same Patient, Different Words, Different Diagnosis?

Imagine you are a doctor. You have a patient with a specific set of symptoms. You write down a note describing the case. Then, you ask a computer program (an AI) to guess the diagnosis.

Now, imagine you rewrite that note. You use different words, change the sentence structure, or swap a medical term for a common one (like saying "heart attack" instead of "myocardial infarction"). The meaning is exactly the same. The patient hasn't changed.

The Problem: The paper asks: If you ask the AI the exact same question but phrase it differently, will it give you the same answer?

Unfortunately, the researchers found that these AI models are often like fickle weather forecasters. If you ask, "Will it rain?" you might get a "Yes." But if you ask, "Is there a chance of precipitation?" the same AI might suddenly say "No," even though the weather hasn't changed. In a hospital, this kind of inconsistency is dangerous.

The Solution: A "Truth Filter"

To test this properly, the researchers had to be very careful. They couldn't just ask the AI to rephrase things, because sometimes the AI accidentally changes the meaning (e.g., turning "no pain" into "pain").

They built a multi-layered truth filter to ensure the questions were truly identical in meaning:

The Logic Check (NLI): They used a special "logic robot" that checks if two sentences logically imply each other. If Sentence A means Sentence B, and Sentence B means Sentence A, they are twins.
The AI Judge: They used a second, very smart AI to double-check the logic robot's work.
The Human Doctor: Finally, a real, licensed doctor reviewed the questions to make sure the medical facts (dosages, symptoms, timing) hadn't been accidentally altered.

Only the questions that passed all three tests were used for the experiment.

The Experiment: General vs. Specialized Doctors

The researchers tested 16 different AI models. They split them into two groups:

General-Purpose (GP): These are like smart generalists. They read everything on the internet and are good at many things, but they aren't medical experts by training.
Domain-Specific (DS): These are like specialized residents. They were trained specifically on medical books and patient records.

The Hypothesis: Everyone assumed the "Specialized Residents" (DS models) would be more stable and reliable than the "Generalists" (GP models) when the wording changed.

The Surprise: The specialized models did not consistently win.

Sometimes the specialized models were more stable.
Sometimes the general models were more stable.
Often, they were just as shaky as each other.

The Analogy: It's like testing two chefs. One is a master chef who only cooks Italian food (Specialized), and the other is a great cook who knows every cuisine (General). You ask them to make a pasta dish. If you describe the dish slightly differently ("boil the noodles" vs. "cook the pasta"), you might expect the Italian chef to be more consistent. But the study found that sometimes the Italian chef gets confused by the new words, while the general cook stays calm. Specialization alone doesn't guarantee stability.

The Confidence Trap: "I'm Sure, But I'm Wrong"

The researchers also looked at how confident the AI was in its answers.

The Finding: The AI often acts like a cocky student who is wrong but thinks they are right.
Even when the AI changed its answer because the wording changed (showing it was unstable), it often kept the same high level of confidence.
The Analogy: Imagine a student taking a test. If you rephrase a question, they might change their answer from "A" to "B." But if you ask, "How sure are you?" they still say, "100% sure!" The paper found that high confidence is not a good sign that the AI is actually correct or stable.

The Key Takeaways

Small changes matter: Changing a few words in a medical prompt can cause the AI to give a completely different diagnosis, even if the meaning is the same.
Training isn't a magic shield: Just because an AI is trained on medical data doesn't mean it will be more reliable when the language varies.
Confidence is misleading: An AI saying it is "99% sure" doesn't mean it won't change its mind if you ask the question slightly differently.
We need better testing: Before we trust these AIs in hospitals, we need to test them not just on whether they get the right answer once, but on whether they stay consistent when the words change.

In short: The paper warns us that current medical AIs are like unreliable translators. If you speak to them in "Medical English," they might give one answer. If you speak to them in "Casual English" (even if it means the same thing), they might give a different one. We need to fix this inconsistency before letting them help doctors treat patients.

Technical Summary: Evaluating Semantic Stability in Clinical LLMs

Problem Statement

Large Language Models (LLMs) are increasingly deployed in clinical settings for tasks such as diagnosis support and summary generation. However, these models exhibit significant sensitivity to minor linguistic variations in input prompts (e.g., rephrasing, synonym substitution, or syntactic reordering). In safety-critical healthcare environments, semantically equivalent inputs should yield consistent predictions; yet, empirical evidence suggests that performance can deviate by 8–50% under such variations.

A primary methodological challenge in evaluating this instability is the lack of reliable semantic verification. Existing approaches relying on embedding-based similarity metrics (e.g., cosine similarity) often fail to capture clinically critical distinctions involving negation, temporality, or severity. For instance, "patient has chest pain" and "patient denies chest pain" may receive high similarity scores despite opposite clinical meanings. Without rigorous verification that prompt variations truly preserve clinical meaning, it is impossible to distinguish genuine model instability from legitimate responses to altered content.

Methodology

The authors propose a systematic framework to evaluate semantic stability using a rigorously verified benchmark and three novel metrics.

1. Semantic Verification Framework

To ensure that prompt variations preserve clinical meaning, the authors developed a multi-stage verification pipeline:

Variation Generation: Starting from 200 base prompts (100 from MedQA-USMLE and 100 from DiagnosisQA), 10 meaning-preserving variations were generated per prompt using GPT-4o-mini. Variations included abbreviation expansion/compression, clinical-to-lay language conversion, syntactic reordering, and voice transformation. Strict constraints prohibited altering clinical facts, negation, temporality, or severity.
NLI-Based Filtering: Generated variations were filtered using three pre-trained Natural Language Inference (NLI) models (PubMedBERT-MNLI-MedNLI, roberta-large-mnli, and deberta-large-mnli). A variation was retained only if it satisfied bidirectional entailment (mutual implication) with the original prompt across at least two of the three models.
Hybrid Human-LLM Audit: A two-round verification process was applied:
- Round 1: NLI-verified samples were reviewed by an LLM-as-a-Judge (using GPT-5.2 and MedLLaMA3) and a licensed clinical expert.
- Round 2: Samples rejected by the LLM judges were re-evaluated by the clinical expert to recover valid reformulations conservatively filtered by automation.
- The final benchmark consists of 1,761 verified meaning-preserving variations.

2. Evaluation Metrics

The study introduces three metrics to quantify model sensitivity:

Meaning-Preserving Variation Sensitivity (MVS): The proportion of verified variations that result in a change in the model's predicted output relative to the base prompt.
Confidence Variation ( $\Delta$ C): The average absolute change in the model's maximum predicted probability (confidence) across variations, even when the predicted output remains unchanged.
Worst-Case Instability (WCI): A per-sample metric indicating whether any verified variation causes the model to flip from a correct to an incorrect prediction (or vice versa).

3. Experimental Setup

The framework evaluated 16 open-source LLMs (8 General-Purpose [GP] and 8 Domain-Specific [DS]) across matched families and parameter scales (1.5B to 8B). Models were tested under 4-bit NF4 quantization with half-precision inference. Predictions were derived deterministically from token probabilities to isolate semantic sensitivity from sampling noise.

Key Results

1. Domain Specialization Does Not Guarantee Robustness

Contrary to the assumption that medical fine-tuning ensures stability, the results show that robustness is highly model-dependent rather than strictly domain-dependent.

Mixed Performance: Several DS models ranked among the most robust, but strong GP baselines (e.g., LLaMA3.1-8B, Qwen2.5-7B) were equally or more competitive.
No Universal Advantage: Statistical analysis (paired permutation tests) revealed no consistent evidence that domain specialization improves robustness. In some cases (e.g., MedQwen2.5-7B on MedQA), DS models were significantly less robust than their GP counterparts.
Catastrophic Forgetting: Some DS models (e.g., MediChat-LLaMA3-8B) underperformed their GP counterparts, suggesting that adaptation strategies can degrade general reasoning capabilities.

2. Task Difficulty Amplifies Instability

Models exhibited higher sensitivity (higher MVS and WCI) on the more complex MedQA dataset compared to DiagnosisQA. This indicates that semantic stability degrades as reasoning demands increase. Specific linguistic transformations, such as Question Reframing and Clinical-to-Lay Language conversion, proved most disruptive, while shallow edits (e.g., abbreviation expansion) had minimal impact.

3. Weak Alignment Between Confidence and Stability

Confidence vs. Prediction: There is only a weak association between prediction instability (MVS) and confidence variation ( $\Delta$ C). Models often maintain high confidence even when their predictions flip under semantically equivalent reformulations.
Overconfidence: Many models remained overconfident despite reduced robustness or low accuracy. For example, Qwen2.5-3B achieved only 40% accuracy on MedQA while reporting 0.90 confidence.
Calibration Issues: The correlation between confidence and accuracy was modest on DiagnosisQA ( $r=0.41$ ) and weak on MedQA ( $r=0.18$ ), indicating that confidence scores are not reliable proxies for correctness or robustness in medical QA.

4. Impact of Quantization

The study found that 4-bit NF4 quantization had a negligible impact on stability metrics for most models, with only minor, dataset-dependent variations observed in WCI.

Significance and Claims

The paper argues that medical fine-tuning alone does not reliably guarantee stable behavior under natural prompt variations. The authors claim that:

Robustness is not inherent to domain specialization: Deploying a medically fine-tuned model does not automatically ensure semantic invariance; robustness depends more on architecture, scale, and training quality than on the presence of medical data.
Current evaluation is insufficient: Standard benchmark accuracy is an inadequate measure of clinical readiness. Models can appear competent on static prompts while failing catastrophically under minor, meaning-preserving rephrasings.
Confidence is misleading: High model confidence should not be interpreted as a signal of stability or correctness, as models often express strong certainty even when their outputs are fragile.
Need for Explicit Robustness Evaluation: Before clinical deployment, LLMs must undergo explicit robustness evaluation using meaning-preserving variations verified by principled semantic methods (like NLI) rather than simple similarity metrics.

The study concludes that semantic stability should be treated as a core requirement for healthcare AI, necessitating a shift from single-prompt accuracy assessments to systematic sensitivity analysis.