💬 NLP

Same Facts, Different Diagnosis: Measuring and Mitigating Narrative Anchoring in Clinical Language Models

This paper identifies "Narrative Anchoring," a bias where clinical language models produce divergent diagnoses for identical facts based solely on sociolinguistic register, and proposes "NarrativeShield," a three-agent pipeline that structurally extracts and verifies facts to nearly eliminate this gap while maintaining diagnostic stability.

Original authors: Prabhjot Singh, Pritam Deka, Vijay Chennareddy

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Prabhjot Singh, Pritam Deka, Vijay Chennareddy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of looking at the clues, you are judging the suspect based on how they tell their story. If they speak with a fancy, polished voice, you might trust them more. If they speak with a rough, working-class accent or use colorful metaphors, you might doubt them, even if the actual facts of the crime are exactly the same. This is the world of "Large Language Models" (LLMs), which are super-smart computer programs trained to read and write like humans. Scientists are now trying to use these programs as "digital doctors" to help diagnose illnesses. But there's a scary problem: these digital doctors might be biased not just by who the patient is (like their race or income), but by how the patient describes their pain. This paper dives into that hidden bias, asking a simple but profound question: If two patients have the exact same symptoms, will the computer give them the same diagnosis if one tells the story in a "rich" way and the other in a "struggling" way?

The researchers call this sneaky problem Narrative Anchoring. It's like a compass that gets stuck pointing toward the style of the story rather than the truth of the facts. To test this, they didn't just ask the computer to guess; they built a massive, super-strict game. They took 1,000 real medical exam questions (the kind doctors have to pass to get their licenses) and rewrote each one three times. One version was neutral, one was rewritten to sound like someone facing financial stress, and a third sounded like someone using cultural metaphors. Crucially, they made sure the medical facts—like the fever, the pain location, and the blood test results—stayed exactly the same in all three versions. They even hired a second computer and real human doctors to double-check that no facts were lost or changed during the rewriting.

When they fed these stories to seven different AI models, the results were startling. Even though the medical facts were identical, the AI models gave different diagnoses depending on the "voice" of the story. This happened in every single model they tested. The gap in accuracy was significant: for some models, the "neutral" voice got the right answer about 15% more often than the "struggling" voice. The researchers tried to fix this by asking the AI to "think step-by-step" (a method called Chain-of-Thought) or by simply telling it, "Ignore the style, focus on the facts." These tricks helped a little, but not enough. In fact, asking the AI to think harder sometimes made it worse at getting the right answer overall, like a student who over-analyzes a simple math problem and gets it wrong.

So, the team built a new tool called NarrativeShield. Think of it as a strict editor who sits between the patient and the doctor. Before the AI doctor ever sees the story, this editor strips away all the fancy words, the emotional tone, and the cultural metaphors, leaving only a bare-bones list of cold, hard medical facts. Then, the AI doctor only looks at that list. When they used NarrativeShield, the bias almost vanished. The gap between the different voices dropped to nearly zero (between -0.004 and 0.037), meaning the AI started treating all stories fairly, regardless of how they were told. However, there was a catch: for most models, this strict editing made the AI slightly less accurate overall, like a translator who removes all the flavor from a poem to ensure the meaning is perfect.

The paper also tested a "bare-bones" medical AI that hadn't been taught how to follow instructions well. This model failed completely when asked to follow the new rules, proving that you can't just tell a computer to "be fair" if it doesn't have the basic ability to listen to instructions in the first place. The authors conclude that the problem isn't just about what labels we give patients; it's about how the computer processes the way patients speak. To fix it, we can't just change the prompt; we have to change the architecture of the system itself, ensuring the "facts" are separated from the "story" before the diagnosis is made. While this is a promising step, the researchers are careful to say this is still a research tool, not a ready-made product for real hospitals yet, and more testing is needed to see if it works in the messy, real world of patient-doctor conversations.

Technical Summary: Same Facts, Different Diagnosis

Problem Definition: Narrative Anchoring
Current research on bias in clinical Large Language Models (LLMs) has largely focused on demographic bias, where explicit identity tokens (e.g., race, gender, income) alter diagnostic outputs. However, this work identifies a distinct and previously unexamined failure mode termed Narrative Anchoring. This phenomenon occurs when identical clinical facts, expressed through different sociolinguistic registers (e.g., varying socioeconomic or cultural framing), cause diagnostic outputs to diverge. Crucially, this bias operates without any explicit demographic markers; the model's reasoning shifts based solely on how the patient describes their symptoms, rather than who the patient is. Existing benchmarks often fail to detect this because they present each case only once, preventing the model from revealing register-dependent inconsistencies.

Methodology and Dataset Construction
To isolate and measure this bias, the authors constructed NARRATIVESHIELD-SDOH, a dataset of 1,000 USMLE-style clinical vignettes.

Source & Filtering: Vignettes were drawn from the MedQA-USMLE corpus and filtered for narrative depth, clinical signal terms, and specific target types (diagnosis, treatment, investigation).
Persona Generation: Each vignette was rewritten into three distinct sociolinguistic personas:
1. Control ( $P_\alpha$ ): Preserves the original register.
2. Socioeconomic ( $P_\beta$ ): Reframed through financial or occupational strain.
3. Cultural ( $P_\gamma$ ): Reframed through culturally situated metaphors.
  Constraint: No explicit demographic markers (race, income figures, cultural identifiers) were included in any persona.
Fact Preservation Audit: To ensure clinical facts remained constant across registers, a two-stage verification process was employed. First, an independently invoked "auditor" model (separate from the generator) verified that every clinical fact survived the rewrite. Second, a stratified sample of 300 encounters was validated by three human annotators with clinical backgrounds, who confirmed 100% fact preservation and register matching.

Experimental Setup
The study evaluated seven open-weight models (ranging from 3B to 12B parameters, including Llama, Mistral, Qwen, Gemma, and BioMistral) across four conditions:

Direct Prompting (B1): Baseline zero-shot prompting.
Chain-of-Thought (B2): Step-by-step reasoning before answering.
Explicit Debiasing (B3): A single instruction to disregard sociolinguistic framing.
NarrativeShield (NS): A proposed three-agent architectural intervention.

Key Contributions and Results

Evidence of Pervasive Narrative Anchoring:
Under direct prompting (B1), all seven models exhibited a statistically significant Narrative Anchoring Gap (NAG)—the accuracy advantage of the control register over marked registers. NAG ranged from 0.064 to 0.151 across models. This confirms that sociolinguistic register alone is sufficient to shift diagnostic outputs, even in the absence of demographic labels.
Limitations of Prompt-Level Mitigations:
- Chain-of-Thought (B2): While B2 reduced NAG for some models, this improvement was frequently confounded by a collapse in overall accuracy (e.g., Gemma-3-12b-it lost 21 points of accuracy). The authors argue that a lower NAG derived from a smaller set of correct answers does not constitute improved fairness.
- Explicit Debiasing (B3): Instructions to ignore framing reduced NAG modestly (e.g., from 0.151 to 0.135 for Llama-3.1-8B) without accuracy loss for instruction-tuned models. However, BioMistral (a domain-adapted base model without instruction tuning) failed completely under B3, achieving 0.00% parse rate, indicating that prompt-level interventions presuppose a baseline instruction-following capacity.
NarrativeShield Architecture:
The authors introduced NarrativeShield, a three-agent pipeline designed to structurally decouple clinical facts from narrative register:
- Agent 1 (Adversarial Intake Extractor): Strips all sociolinguistic register, economic framing, and metaphors, outputting a structured JSON of clinical facts.
- Deterministic Tool Router: Injects clinical decision support (e.g., risk calculators, drug knowledge bases) based on the extracted facts.
- Agent 2 (Clinical Reasoning Engine): Performs diagnostic reasoning only on the structured JSON and tool outputs, never seeing the original narrative.
- Agent 3 (Emergency Fallback): Extracts a final answer letter if Agent 2 fails to format its output correctly.
Performance: NarrativeShield reduced the NAG to near-zero (−0.004 to 0.037) across all models. It also achieved the lowest rate of severely unstable decisions (Diagnostic Stability Score < 0.8) compared to all other conditions. This came with a modest, mechanistically explainable accuracy cost for most models (attributed to information loss during extraction), though Gemma-4-E4B-it actually improved in accuracy, suggesting that for some models, removing narrative noise aids reasoning.

Significance and Claims
The paper argues that clinical bias in LLMs is not solely a function of demographic labeling but is deeply embedded in how patient narratives are processed. The primary significance of this work is threefold:

Identification: It establishes Narrative Anchoring as a pervasive, register-driven bias that survives standard prompt-level mitigations.
Architectural Solution: It demonstrates that structural interventions (separating fact extraction from reasoning) are more effective at eliminating this bias than instructional prompts.
Prerequisite Capacity: Through the stress testing of BioMistral, the authors show that any debiasing intervention—whether a prompt or a pipeline—requires a baseline zero-shot instruction-following capacity that not all models possess.

The authors conclude that closing the gap in clinical bias requires pipeline design that structurally isolates clinical facts from sociolinguistic framing, rather than relying solely on prompt engineering. They explicitly state that their work is a research-stage diagnostic and mitigation, not a deployment-ready clinical tool, and that the accuracy trade-offs of such architectures must be evaluated against specific clinical stakes.

Technical Summary: Same Facts, Different Diagnosis

More like this