🤖 AI

Causal Bias Detection in Generative Artifical Intelligence

This paper establishes a unified theoretical framework for causal fairness in generative AI, deriving new decomposition methods and estimators to quantify how generative models' internal causal mechanisms contribute to demographic biases across different pathways.

Original authors: Drago Plecko

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Drago Plecko

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand why two groups of people are treated differently in society. Maybe one group gets lower salaries, or is more likely to be diagnosed with a certain disease, or uses certain substances more often.

Now, imagine we build a super-smart computer (an AI) to help us make decisions or tell stories about these people. The big worry is: Does the computer just copy the unfairness of the real world, or does it accidentally make it worse?

This paper introduces a new way to check the computer's "brain" to see exactly how and why it might be biased. Here is the breakdown using simple analogies:

1. The Old Way vs. The New Way

The Old Way (Standard AI):
Think of a traditional AI like a judge. The judge looks at a person's resume (their age, education, job history) and decides if they get a loan or a job. The judge uses the real world's rules for how age and education work, but the judge applies their own specific rule for the final decision.

The problem: We usually only check if the judge's final decision is fair. We don't check if the judge misunderstood how the world works.

The New Way (Generative AI):
Think of a Generative AI (like the chatbots you know) as a storyteller. This storyteller doesn't just look at a resume and give a "Yes/No." It invents entire lives. It decides what a person's job is, what their salary is, what their hobbies are, and whether they get sick, all based on who they are.

The problem: Because the storyteller invents everything, it might have its own weird ideas about how the world works. It might think, "Oh, people of Group A usually have low incomes," even if that's not true in reality. It's not just judging; it's rewriting the rules of the universe.

2. The "S-Node": The Reality Switch

The authors created a special tool called the S-SFM (Selection-Standard Fairness Model). Imagine a giant switchboard labeled "S".

When the switch is set to Real World, the computer uses actual facts from human history (like real census data).
When the switch is set to AI World, the computer uses the AI's own made-up beliefs.

By flipping this switch for different parts of a person's life (their background, their job, their health), the researchers can isolate exactly where the AI goes wrong.

3. The Three Paths of Bias

The paper breaks down unfairness into three "roads" the AI can take:

The Direct Road: The AI treats Group A differently than Group B just because of who they are (e.g., "Because you are X, you get Y").
The Indirect Road: The AI treats Group A differently because it thinks they have different middleman traits (e.g., "Because you are X, the AI thinks you have less education, so you get less money").
The Spurious Road: The AI gets confused by background noise (e.g., "Group A is younger on average, and younger people get sick less, so the AI thinks Group A is healthier, even if that's not the whole story").

4. The "Waterfall" Experiment

The researchers didn't just look at the final result. They built a waterfall to see where the water (the bias) changes.

They started with real-world data.
Then, they swapped out the AI's "outcome" brain (how it decides if someone smokes or gets sick) while keeping the real world's facts for everything else.
Next, they swapped the AI's "mediator" brain (how it decides someone's income or education).
Finally, they swapped the AI's "background" brain (how it decides someone's age or race).

By watching the water level rise or fall at each step, they could pinpoint: Is the AI biased because it thinks minorities use more drugs? Or is it biased because it thinks minorities have lower incomes?

5. What They Found

They tested 10 different popular AI models on three real-world topics: marijuana use, diabetes, and salaries.

The "Mirror" Effect: Sometimes, the AI was a perfect mirror of reality.
The "Distortion" Effect: Sometimes, the AI took a small real-world bias and made it huge.
The "Reversal" Effect: Sometimes, the AI got it completely backward. For example, in the real world, minority groups actually used less marijuana than the majority group (when controlling for other factors). But one AI model (Gemma 3) decided the opposite, inventing a stereotype that minorities use more.
Family Matters: You might think AI models from the same company (like "Llama" siblings) would think alike. The study found that wasn't always true. Two models from the same family could have very different "personalities" regarding bias, while models from different families sometimes thought alike.

The Bottom Line

This paper gives us a microscope for AI bias. Instead of just saying "This AI is unfair," we can now say, "This AI is unfair because it has a specific, incorrect belief about how education relates to income for a specific group."

It's like a doctor diagnosing a patient. Instead of just saying "You are sick," the doctor can now say, "You have a fever because of this specific virus, not that one." This helps us fix the specific part of the AI's "brain" that is broken, rather than just throwing the whole computer away.

Technical Summary: Causal Bias Detection in Generative Artificial Intelligence

Problem Statement

As automated systems based on artificial intelligence (AI) are increasingly deployed in high-stakes domains (e.g., hiring, lending, healthcare), concerns regarding fairness and the perpetuation of demographic disparities have intensified. While prior work in causal fairness has established a principled framework for the standard machine learning (ML) setting—where a decision-maker constructs a single predictive function $f_{\hat{Y}}$ while inheriting causal mechanisms for all other covariates from the real world—this framework is insufficient for Generative AI.

Generative models differ fundamentally from standard ML predictors:

Mechanism Construction: Unlike standard ML, which learns a single mapping from covariates to an outcome, generative models implicitly construct their own beliefs about all causal mechanisms (including those for covariates $X, Z, W$ ), not just the outcome $Y$ .
Data Granularity: Generative models often operate on lower-level modalities (text, images) rather than structured tabular data, requiring inference of variable relationships that may differ significantly from real-world correlations.
Bias Propagation: Existing methods fail to distinguish whether observed disparities in generative outputs arise from the model's learned outcome mechanism, its learned covariate mechanisms, or the interaction between them.

The paper addresses the need for a new methodology to detect, quantify, and decompose causal bias in generative AI, specifically analyzing how generative models alter real-world disparities through their internal causal mechanisms.

Methodology

The authors propose a unified theoretical framework that extends the Standard Fairness Model (SFM) to the generative setting using Structural Causal Models (SCMs).

1. The S-Standard Fairness Model (S-SFM)

The core innovation is the introduction of a selection node $S$ that indicates the source of the data generation mechanism:

$S = rw$ (or $0$): The real-world mechanism ( $f^{rw}$ ).
$S = gm$ (or $1$): The generative model's mechanism ( $f^{gm}$ ).

In this framework, the selection node $S$ points to all variables ( $X, Z, W, Y$ ), allowing for the independent replacement of mechanisms. This contrasts with standard SFM where $S$ typically only affects the outcome mechanism. The model defines potential outcomes $Y^{s_y}_{x_y, W^{s_w}_{x_w} | X=x_z, S=s_z}$ , where superscripts denote the environment (real world or model) from which the specific mechanism is drawn, and subscripts denote the values of variables along specific causal paths.

2. S-Modified Causal Effects

The authors define S-modified x-specific effects (Direct, Indirect, and Spurious) that account for the source of the mechanism. For example, the S-modified direct effect is:
$x\text{-}DE^{s_z, s_w, s_y}_{x_0, x_1}(y | x) = E[Y^{s_y}_{x_1, W^{s_w}_{x_0}} | X=x, S=s_z] - E[Y^{s_y}_{x_0, W^{s_w}_{x_0}} | X=x, S=s_z]$
This allows for the isolation of how the direct effect changes when specific mechanisms (e.g., $f_Y$ , $f_W$ , or $f_{X,Z}$ ) are swapped from the real world to the generative model.

3. Decomposition of Disparity Differences

The paper derives a Theorem 1 that decomposes the difference in total variation (TV) between the real world and the generative model ( $\Delta TV$ ) into contributions from changes in direct, indirect, and spurious pathways. Crucially, each pathway contribution is further decomposed into three terms representing the stepwise replacement of mechanisms:

T1: Replacement of the outcome mechanism ( $f_Y$ ) only.
T2: Replacement of the mediator mechanism ( $f_W$ ), with $f_Y$ already replaced.
T3: Replacement of the covariate mechanism ( $f_{X,Z}$ ), with $f_W$ and $f_Y$ already replaced.

This granular decomposition enables researchers to pinpoint exactly which part of the generative model's "belief system" drives the bias.

4. Identification and Estimation

The paper establishes identification conditions (Proposition 3) showing that these potential outcomes can be estimated using samples from mixed environments. The estimation strategy involves:

Generator ( $\Gamma$ ): An LLM prompted to generate narratives based on specific conditions (e.g., "Write a story about a [Race] person of [Sex] age [Age]...").
Annotator ( $A$ ): A separate LLM that extracts structured values (e.g., income, education, outcome) from the generated narratives.
Datasets: Construction of four specific datasets ( $D_{s_0}$ , $D_{s_0,s_0,s_1}$ , $D_{s_0,s_1,s_1}$ , $D_{s_1}$ ) representing progressive replacement of mechanisms, allowing for the calculation of the decomposition terms.

Key Contributions

Formalization of Causal Fairness in Generative AI: The paper unifies the standard ML fairness setting and the generative AI setting under a single theoretical umbrella (S-SFM), explicitly modeling the generative model's construction of its own causal mechanisms.
New Causal Decomposition Results: It introduces a novel decomposition (Theorem 1) that quantifies fairness impacts along two dimensions:
- Different causal pathways (Direct, Indirect, Spurious).
- The specific mechanism being replaced (Real-world vs. Model).
Identification and Estimation: It provides identification conditions and efficient estimators for these causal quantities, demonstrating how to generate the necessary counterfactual data using LLMs.
Empirical Application: The methodology is applied to analyze racial and gender bias in 10 large language models (LLMs) across three datasets:
- NSDUH: Racial bias in substance abuse (marijuana use).
- BRFSS: Racial bias in chronic disease burden (diabetes).
- ACS: Sex bias in income levels.

Results

The authors analyzed 10 instruction-tuned open-weight models (including Llama 3, Qwen, Gemma, DeepSeek, and Phi families) across the three datasets.

Bias Signatures: The study produced a 27-dimensional "bias signature" for each model. Hierarchical clustering revealed that bias similarity does not strictly correlate with model family or parameter scale; distinct families often exhibited closer bias signatures than siblings within the same family.
Amplification and Reversal:
- Reversal: Models frequently reversed the sign of real-world disparities. For instance, in the NSDUH dataset, real-world data showed minorities use marijuana at lower rates (controlling for covariates), but models like Gemma 3 27B reversed this to show minorities using it at higher rates.
- Amplification: In the BRFSS dataset, models often amplified the direct effect of race on diabetes risk.
- Dampening: Some models dampened spurious advantages (e.g., the age-related protection against diabetes for minorities in BRFSS).
Mechanism Attribution: The decomposition revealed that bias is not monolithic.
- In the Gemma 3 27B case (NSDUH), the reversal of the direct effect was driven primarily by the replacement of the mediator mechanism ( $f_W$ ), which encoded a stereotyped distribution of education/income for minorities, rather than just the outcome mechanism.
- In the Qwen 3.5 27B case (BRFSS), the reversal of the direct effect was dominated by the outcome mechanism ( $f_Y$ ), while the indirect effect reversal was driven by the mediator ( $f_W$ ) and covariate ( $f_{X,Z}$ ) mechanisms.

Significance and Claims

The paper claims that its primary significance lies in providing a granular, causal lens for understanding bias in generative AI, moving beyond aggregate statistical disparities.

Beyond Counterfactual Fairness: Unlike prior work that focuses solely on counterfactual fairness in classification, this framework models the latent causal process and quantifies which mechanisms (outcome, mediator, or covariate generation) contribute to disparities.
Generalization: The framework strictly generalizes existing causal fairness literature for standard ML (where only the outcome mechanism is replaced) to the more complex generative setting where all mechanisms are potentially altered.
Diagnostic Utility: The decomposition allows practitioners to diagnose whether a model's bias stems from its understanding of the outcome (e.g., "minorities are more likely to use drugs") or its understanding of the world's structure (e.g., "minorities have lower education/income"), offering targeted insights for mitigation.

The authors remain modest regarding limitations, noting that the framework assumes the ability to query models for arbitrary conditionals (which may be difficult in vision models), requires human elicitation of causal assumptions (S-SFM specification), and was restricted to open-weight models due to cost constraints for closed models. They also note that the framework does not currently cover inferences made in the reverse causal order (e.g., inferring race from salary).