💻 computer science

First, Do No Harm (With LLMs): Mitigating Racial Bias via Agentic Workflows

This study evaluates racial bias in five large language models for clinical tasks under the EU AI Act framework and demonstrates that embedding the best-performing model (DeepSeek V3) into a retrieval-based agentic workflow can effectively mitigate explicit bias in differential diagnosis.

Original authors: Sihao Xing, Zaur Gouliev

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sihao Xing, Zaur Gouliev

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a team of very smart, very fast digital doctors to help diagnose patients. You want them to be fair, accurate, and unbiased. But there's a problem: these digital doctors were trained on massive amounts of data from the real world, and unfortunately, the real world has a history of unfairness and racial inequality in healthcare.

This paper is like a quality control report testing five of the most popular "digital doctors" (Large Language Models or LLMs) to see if they treat patients of different races fairly. The researchers also tested a new trick: putting these digital doctors inside a "super-team" workflow to see if that helps them be more fair.

Here is the breakdown of their journey, explained simply:

1. The Problem: The "Broken Compass"

Think of these AI models as compasses trying to point toward the truth. However, because they learned from old maps (training data) that had errors, some compasses point slightly toward the wrong direction when it comes to race.

Implicit Bias: This is like a digital doctor unconsciously thinking, "Oh, this disease is more common in Group A," even when the patient is from Group B. It's a subtle, hidden assumption.
Explicit Bias: This is when the digital doctor makes a clear mistake, like suggesting a different list of diseases for a Black patient than for a White patient, even if their symptoms are identical.

2. The Test: Two Different Drills

The researchers put the five AI models through two specific drills to see how they performed.

Drill A: The "Story Writer" (Implicit Bias)
- The Task: The AI had to write fake patient stories for 10 different diseases (like diabetes or HIV).
- The Goal: Did the AI write stories where the race of the patients matched real-world statistics? (e.g., If Lupus is more common in Black women in real life, did the AI write more stories about Black women having Lupus?)
- The Result: All the AIs got this wrong to some degree. They didn't match the real world perfectly. However, GPT-4.1 was the "best student" here, with the smallest gap between its stories and reality.
Drill B: The "Diagnosis Detective" (Explicit Bias)
- The Task: The AI was given real medical cases but with the patient's race swapped (e.g., same symptoms, but one time the patient is White, the next time Black). The AI had to list the top 10 possible diagnoses.
- The Goal: Did the AI change its list of suspects based on the patient's race?
- The Result: This is where DeepSeek V3 shined. It was the most consistent detective. It gave the same high-quality list of diagnoses regardless of the patient's race, showing the least amount of unfairness.

3. The Solution: The "Super-Team" Workflow

The researchers asked: Can we fix the bias by giving the AI a helper?

They built an Agentic Workflow. Imagine the AI isn't just a lone genius sitting in a room guessing. Instead, it's a detective with a team:

The Search Agent: Runs to the library (the internet) to look up the latest facts.
The RAG Agent: Checks a trusted, pre-approved medical textbook (a database of expert knowledge) to verify the facts.
The AI: Combines the search results and the textbook info to make a final decision.

The Outcome:
When they put the best detective (DeepSeek V3) into this "Super-Team" workflow, it got even better at being fair.

It became less likely to give different diagnoses based on race.
It relied more on the trusted "textbook" facts rather than its own potentially biased guesses.

4. The Rules of the Game (The EU AI Act)

The researchers didn't just test the AI; they used the EU AI Act as a rulebook. Think of this as a strict safety inspector for high-risk tools. The Act says: "If you are building medical AI, you must check for bias, keep records, and have a human in the loop." The study used these rules to structure their testing, ensuring they weren't just guessing, but following a rigorous safety protocol.

The Big Takeaway

No AI is perfect yet: Even the best models (like GPT-4.1 and DeepSeek V3) still show some bias, especially when writing stories about patients.
Different models excel at different tasks: GPT-4.1 was better at writing fair stories; DeepSeek V3 was better at making fair diagnoses.
Teamwork helps: Putting the AI in a "Super-Team" workflow (using search and trusted databases) helped reduce the bias in its decision-making.

In short: We can't just trust a single AI to be fair. We need to test them rigorously, use the best ones for specific jobs, and give them access to trusted facts (like a "Super-Team") to help them make the fairest possible decisions for every patient, regardless of their race.

1. Problem Statement

The rapid integration of Large Language Models (LLMs) into healthcare (e.g., for text generation and clinical decision-making) raises critical concerns regarding racial bias. Existing literature has identified that LLMs can perpetuate or amplify structural inequalities, leading to disparate health outcomes for minority groups.

Implicit Bias: Unconscious associations between race and disease characteristics in generated text.
Explicit Bias: Systematic deviations in clinical reasoning, such as prioritizing different diagnoses or treatment predictions based on a patient's race.
Gap: Most existing studies focus on single models or lack robust mitigation strategies. There is a need for multi-model comparisons and empirical evidence on whether AI agentic workflows (systems combining LLMs with retrieval and search capabilities) can effectively mitigate these biases.

2. Methodology

The study employs a two-stage experimental design, using the EU AI Act as a governance lens to structure evaluation, documentation, and risk management principles.

A. Experimental Setup

Models Evaluated: Five mainstream LLMs: GPT-4.1, Llama 3.3, DeepSeek V3, Grok 3, and Gemini 2.5 Pro.
Datasets:
1. Synthetic Patient-Case Generation: Based on 10 US diseases with race-stratified epidemiological distributions (Black, White, Hispanic, Asian, Other).
2. Differential Diagnosis (DDx) Ranking: Based on 10 real emergency department cases from the NEJM Healer benchmark, where the patient's race is systematically varied.
Evaluation Metrics:
- Implicit Bias: Chi-square goodness-of-fit test comparing generated case distributions against real-world epidemiological data.
- Explicit Bias: Mann-Whitney U test comparing model-generated diagnosis rankings against expert baselines across racial groups. Metrics included mean/median p-values, bias detection rate, and mean rank difference.

B. Agentic Workflow Implementation (Experiment II)

After identifying the best-performing model, the study implemented an Agentic Workflow using Flowise.

Architecture:
- Search Agent: Uses Brave Search API to retrieve real-time medical information.
- RAG Agent: Uses a vector database containing expert differential diagnosis lists and patient case descriptions.
- Synthesis: The final output is a weighted synthesis (90% Knowledge Base, 10% Search Agent) to prioritize established medical knowledge while supplementing with external data.
Goal: To determine if retrieving external, expert-verified knowledge reduces the model's reliance on internal biased training data.

3. Key Contributions

Multi-Model Comparative Analysis: A systematic evaluation of five diverse LLMs across both implicit (generation) and explicit (diagnosis) bias tasks, moving beyond single-model studies.
Agentic Workflow Mitigation: Empirical demonstration that embedding LLMs in a Retrieval-Augmented Generation (RAG) workflow can improve bias metrics in clinical decision-making tasks.
Governance Framework Application: Practical application of the EU AI Act principles (risk management, transparency, traceability) to structure prompt engineering and bias evaluation, rather than treating it as a purely legal audit.
Open Resources: Release of prompt templates, experimental datasets, and code pipelines to facilitate reproducibility.

4. Results

Experiment I: Baseline Model Performance

Implicit Bias (Case Generation): All models deviated significantly from real-world racial distributions ( $p < 0.05$ $p < 0.05$ ).
- Best Performer: GPT-4.1 showed the smallest deviation (Mean Chi-square: 152.23).
- Runner-up: DeepSeek V3 (Mean Chi-square: 164.06).
Explicit Bias (Differential Diagnosis):
- Best Performer: DeepSeek V3 outperformed all other models across all metrics.
  - Highest Mean p-value: 0.7859 (indicating less statistical difference between racial groups).
  - Lowest Mean Difference: 0.2588.
  - Zero cases of statistically significant bias detection ( $p < 0.05$ ) in the top 3 diagnoses.
- Other Models: Llama 3.3, Gemini 2.5 Pro, and Grok 3 showed varying degrees of significant bias, particularly in "Asian vs. Black" and "Asian vs. White" comparisons.

Experiment II: Agentic Workflow Impact

DeepSeek V3 was selected for the agentic workflow test.

Improvements: The workflow improved most explicit bias metrics compared to the standalone DeepSeek V3:
- Mean p-value: Increased from 0.7859 to 0.8207.
- Median p-value: Increased from 0.8834 to 1.0000.
- Mean Difference: Decreased from 0.2588 to 0.1639.
Limitation: The bias detection rate slightly increased (0.0000 to 0.0167), indicating that while overall bias decreased, the improvement was not uniform across every specific metric.

5. Significance and Conclusion

Clinical Safety: The study confirms that while LLMs inherently possess racial biases, retrieval-based agentic workflows can serve as a technical mitigation strategy, reducing explicit bias in diagnostic ranking tasks.
Regulatory Alignment: The findings support the EU AI Act's requirement for high-risk AI systems to implement robust risk management and traceable mitigation strategies throughout the system lifecycle.
Future Directions: The authors suggest future work should include:
- Human-in-the-loop components for realistic clinical validation.
- Broader performance metrics (accuracy, recall) beyond just bias detection.
- Extending agentic workflows to synthetic text generation tasks.

Final Verdict: DeepSeek V3 demonstrated the lowest overall bias in this study, and its integration into an agentic workflow further reduced explicit racial bias in differential diagnosis, offering a promising pathway for safer medical AI deployment.