Contextual Cue Susceptibility in Clinical AI: A Randomized Controlled Clinician-Comparator Study

Technical Summary: Contextual Cue Susceptibility in Clinical AI

Problem Statement
Large Language Models (LLMs) are increasingly deployed in clinical workflows for tasks ranging from summarizing records to generating differential diagnoses and acting as autonomous agents. While these models demonstrate fluency and confidence, their outputs are highly sensitive to context. Unlike human clinicians, who may exhibit biases such as availability bias, LLMs operate at a scale and opacity where a single, clinically minor contextual detail (e.g., a brief travel history or occupational exposure) can disproportionately shift diagnostic reasoning. The core problem addressed is whether LLMs are significantly more susceptible to "contextual cueing"—where a minor detail redirects the model toward a less probable, "lure" diagnosis—compared to human clinicians, and how this susceptibility behaves under different prompting conditions.

Methodology
The authors conducted a randomized controlled clinician-comparator study involving two primary components:

1. Case Construction: 100 short, synthetic clinical cases were created and validated by physicians. Each case family included a "no-cue" version and a "cue" version. The cue was a brief, clinically plausible but statistically minor detail (e.g., travel, exposure) designed to suggest a specific "lure" diagnosis that was less probable than the correct diagnosis based on the core presentation.
2. Participant Groups:
   • Clinicians: 47 participants (attending physicians, residents, fellows, and medical students) from the US, Middle East, and Europe completed a subset of 21 matched case families. They selected the single most probable diagnosis from four options and rated confidence.
   • LLMs: 22 distinct LLMs (including frontier models from major providers) were tested across all 100 cases, generating approximately 92,000 usable responses.
3. Prompting Arms: LLMs were tested under three conditions:
   • Baseline: Standard instruction to choose the most probable diagnosis.
   • Base-rate Guardrail: Explicit prompting to consider base rates and common conditions.
   • Rare/Serious Stress: Explicit prompting to prioritize rare or serious diagnoses.
4. Analysis: The primary endpoint was the selection rate of the "cue-linked lure." Secondary endpoints included diagnostic accuracy. Statistical analysis utilized mixed-effects logistic regression and rater-clustered bootstrap resampling to compare the "cue effect" (the difference in lure selection between cue and no-cue conditions) between LLMs and clinicians. A secondary subanalysis examined model explanation text for rationalization patterns.

Key Results

• Disproportionate Susceptibility: In the matched 21-case comparison, baseline LLMs selected the cue-linked lure in 84.9% of cue cases, compared to 27.8% for clinician participants. This represents an "extra cue effect" of 57.6 percentage points (95% CI 50.2–65.2) for LLMs relative to clinicians.
• Impact on Accuracy: Diagnostic accuracy for LLMs dropped precipitously in cue cases, falling from 90.9% (no-cue) to 12.4% (cue). Clinician accuracy also decreased but remained significantly higher, dropping from 86.3% to 54.6%.
• Prompt Sensitivity: The susceptibility of LLMs was highly tunable via prompting alone.
  • Base-rate guardrail prompting reduced lure selection to 28.4% (near the clinician rate) and improved accuracy to 66.0%.
  • Rare/serious prompting increased lure selection to 93.2% and reduced accuracy to 4.1%.
• Rationalization: Analysis of model explanation text revealed that when LLMs selected the lure, they frequently rationalized the choice with high confidence, often citing the specific cue as the primary reason, whereas clinicians were more likely to attribute the choice to a specific detail rather than a wholesale diagnostic shift.

Key Contributions

• Quantification of Contextual Vulnerability: The study provides the first controlled, large-scale quantification of how much more susceptible LLMs are to minor contextual cues compared to human clinicians in diagnostic reasoning.
• Prompt as a Control Variable: It demonstrates that model susceptibility is not a fixed property but can be drastically amplified or mitigated by the framing of the prompt, suggesting that "context" includes both the case input and the system instructions.
• Safety Signal Definition: The authors propose "contextual cue susceptibility" as a distinct safety signal for clinical AI, separate from standard accuracy metrics.

Significance and Claims
The paper argues that the primary risk is not merely that models occasionally choose rare diagnoses, but that they lack a stable "clinical prior" to weigh minor details against base rates. When a model is given agency to act on context (e.g., drafting notes, triaging, routing referrals) with little human review, a minor cue can trigger a cascade of errors that are difficult to trace.

The authors claim that current evaluation metrics focusing solely on accuracy are insufficient. Instead, clinical AI systems must be evaluated on their stability under plausible context changes. The study concludes that susceptibility to context should be a named, tested property of any system granted clinical autonomy, as the same mechanisms that allow a prompt to "fix" a model on one task can cause it to fail catastrophically on another. The findings suggest that in agentic workflows, where models rewrite and add context, the risk of uncontrolled context-driven shifts is a near-term safety concern.