🤖 AI

A Systems Thinking Approach to Algorithmic Fairness

This paper proposes a systems thinking framework that integrates machine learning, causal inference, and system dynamics to model algorithmic fairness, thereby enabling policymakers to navigate complex trade-offs and design AI regulations aligned with democratic values.

Original authors: Chris Lam

Published 2026-04-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Chris Lam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fix a leaky faucet in a house where the water pipes are hidden inside the walls. Most engineers (AI researchers) have been trying to fix the leak by only looking at the water coming out of the spout, measuring the flow, and trying to adjust the handle. They assume that if the water looks clean, the system is fair.

But this paper argues that you can't fix the leak just by looking at the water. You need to understand the whole plumbing system, the history of the house, and why the pipes got clogged in the first place.

Here is the paper's argument, broken down into simple concepts and analogies.

1. The Core Problem: The "Black Box" Trap

For years, we've built AI systems like "black boxes." You put data in (like a resume or a credit score), and a decision comes out (hire or loan). If the AI treats people unfairly, we usually just try to tweak the math inside the box to make the numbers look equal.

The author, Chris Lam, says this is like trying to fix a car engine by painting the hood. It doesn't work because fairness isn't just a math problem; it's a story problem. The data the AI learns from is already stained by history, bias, and social structures.

2. The Three Lenses: How to Look at the Problem

The paper suggests we need to look at AI fairness through three different "lenses" or tools, like using a microscope, a telescope, and a time machine all at once.

Lens A: The Machine Learning View (The Snapshot)

The Analogy: A photograph.
What it does: It takes a picture of the data right now. It sees: "Here is the input, here is the output."
The Limit: A photo is static. It shows you what happened, but not why it happened or what will happen next. It misses the story behind the picture.

Lens B: Causal Inference (The Detective)

The Analogy: A detective's flowchart.
What it does: Instead of just looking at the photo, the detective draws arrows to show cause and effect. "Did the person get rejected because they are poor (the cause), or because they have bad credit (the effect)?"
The Insight: This helps us see if the AI is making a decision based on a protected trait (like race) directly, or if it's using a "proxy" (like zip code) that secretly stands in for race. It helps us decide: Is this unfair because of the person's identity, or because of the system they live in?

Lens C: System Dynamics (The Time Machine)

The Analogy: A video game with a feedback loop.
What it does: This is the most important part of the paper. It asks: "What happens tomorrow because of what we did today?"
The Insight: If an AI denies loans to a specific group today, that group has less money tomorrow. Because they have less money, their credit score looks worse next year. The AI sees the bad score and denies them loans again.
- This creates a vicious cycle (a "feedback loop"). The system gets stuck in a loop of getting worse and worse.
- System dynamics helps us see these loops so we can break them before they destroy the system.

3. The Political Divide: The "Blame Game"

The paper explains why politicians argue so much about fairness. It comes down to where we think the problem starts.

The "Internal" View (Often Right-Wing): "The problem is the individual."
- Analogy: If a student fails a test, it's because they didn't study hard enough.
- Solution: Don't look at race or gender. Just look at the test score. (This is called "Fairness through Unawareness").
- Risk: If the test itself was rigged against them, this approach just ignores the unfairness.
The "External" View (Often Left-Wing): "The problem is the system."
- Analogy: If a student fails a test, it's because the school didn't give them a teacher or a quiet place to study.
- Solution: We need to actively help the disadvantaged group to fix the imbalance. (This is called "Affirmative Action").
- Risk: If we help too much without fixing the underlying skills, we might set people up for failure later (like giving a loan to someone who can't afford it).

4. The "System Archetypes" (The Traps)

The author uses three famous patterns from systems thinking to explain why AI fairness is so hard:

"Success to the Successful": If Group A gets a little advantage, the AI gives them more advantages. Group B gets fewer. Over time, Group A gets rich and Group B gets poor, and the AI just keeps reinforcing this gap.
"Limits to Success": You can't just keep giving resources to Group A forever; eventually, the system hits a wall (like a budget cap or a market limit).
"Shifting the Burden": This is the most dangerous trap.
- Scenario: We see Group B failing, so we force the AI to give them loans (External Intervention).
- Short Term: It looks great! More loans are approved.
- Long Term: Because the loans weren't based on actual ability, Group B defaults on the loans. They look even worse next time. Now, we feel we need to intervene even more. We become addicted to the "fix" and stop trying to fix the root cause (the actual skills or opportunities).

5. The Big Picture: The "Systems Map"

The paper concludes that we need a new map to navigate this.

Old Map: Just look at the code and the math.
New Map: A bridge connecting Sociology (how society works), Politics (what we value), and Computer Science (how the code works).

The author suggests we need a "Causal Hierarchy":

Level 1 (Prediction): What does the data say? (Machine Learning)
Level 2 (Intervention): What happens if we change the rules? (Causal Inference)
Level 3 (Simulation): What happens over 10 years? (System Dynamics)

The Takeaway

To build truly fair AI, we can't just be programmers. We have to be plumbers, detectives, and historians all at once.

We need to stop asking, "Is the math fair?" and start asking, "How does this math change the world over time, and are we accidentally trapping people in a loop of poverty or privilege?" By using Systems Thinking, we can design AI that doesn't just look fair on paper, but actually helps society grow in a healthy, sustainable way.

1. Problem Statement

The paper addresses the persistent failure of purely technical approaches to solve the algorithmic fairness problem. Current methods relying on statistical correlation have failed to align mathematical definitions of fairness with legal definitions (e.g., disparate treatment vs. disparate impact) or societal values.

The Core Conflict: There is a fundamental ideological divide in how society attributes the causes of disparities (e.g., between race or gender groups).
- Internal Attribution (Right-wing/Libertarian): Disparities are caused by individual traits/choices. This supports "Fairness through Unawareness" (ignoring protected attributes).
- External Attribution (Left-wing/Progressive): Disparities are caused by systemic bias. This supports "Affirmative Action" (using protected attributes to correct bias).
The Gap: Existing causal models (Causal Bayesian Networks) are limited because they cannot model feedback loops (time dynamics) and often fail to incorporate the sociopolitical context required to link machine learning (ML) to law and policy.

2. Methodology

The author proposes a Systems Thinking framework that integrates three distinct analytical techniques into a unified hierarchy:

Machine Learning (Block Diagrams): Modeling the system as a black box where data $X$ enters and decisions $D$ are made, leading to outcomes $Y$ .
Causal Inference (Causal Bayesian Networks - CBNs): Using Directed Acyclic Graphs (DAGs) to encode assumptions about where bias exists (e.g., between protected attributes $A$ , mediators $W$ , and data $X$ ).
System Dynamics (Causal Loop Diagrams - CLDs): Modeling feedback loops and time delays to understand how fairness policies evolve over time, creating reinforcing (virtuous/vicious) or balancing loops.

The Framework:
The paper constructs a New Causal Hierarchy for fairness:

Layer 1 (Association): Supervised Machine Learning (Prediction).
Layer 2 (Acyclic Causation): Causal Inference (Control/Intervention).
Layer 3 (Cyclic Causation): System Dynamics (Simulation/Emergence).

The methodology involves mapping different political worldviews and legal concepts onto these technical representations to visualize how specific interventions (like removing a protected attribute or adding affirmative action) propagate through the system.

3. Key Contributions

A. Unification of Social Science and Technical Models

The paper bridges the gap between social science concepts (Agency vs. Structure, Internal vs. External Attribution) and technical ML models. It demonstrates that different political ideologies correspond to specific causal graph structures:

Fairness through Unawareness: Maps to the "Right-wing" view; assumes data is unbiased (internal attribution).
Fairness through Affirmative Action: Maps to the "Left-wing" view; assumes data is biased (external attribution) and requires intervention.
Fairness through Supremacism/Lottery: Represents extreme ends of the spectrum.

B. Formalizing the "Is-Ought" Problem in Fairness

The author distinguishes between:

Normative Effects (What ought to be): Modeled via CBNs. These depend on prior beliefs about bias. For example, a policy might be "fair" to one group (zero total effect) but "discriminatory" to another based on their attribution of the cause.
Positive Effects (What is): Modeled via CLDs. These analyze the actual system behavior over time, independent of moral judgment.

C. Identification of System Archetypes in Fairness

The paper applies three classic System Dynamics archetypes to algorithmic fairness, revealing counterintuitive long-term consequences:

Success to the Successful: A reinforcing loop where ML models favor historically successful groups, creating a "vicious cycle" of disparity (equivalent to Fairness through Unawareness in the short term).
Limits to Success: A balancing loop where constraints eventually halt the growth of the advantaged group, potentially allowing the disadvantaged group to catch up.
Shifting the Burden / Addiction: A critical finding regarding Affirmative Action. While it may equalize decisions in the short term (balancing loop), it can create a long-term dependency on external intervention. If the underlying mediator (e.g., merit/qualifications) is not improved, the disadvantaged group may suffer worse actual outcomes (e.g., loan defaults), reinforcing the need for even stronger intervention.

D. A New Causal Hierarchy

The paper proposes an expanded hierarchy (Figure 8) that explicitly includes System Dynamics as a third layer above Pearl's existing hierarchy. This allows for the modeling of cyclic causation (feedback loops), which is essential for understanding how AI systems interact with society over time.

4. Results and Findings

Mapping to Law: The paper successfully maps technical causal structures to US/EU legal concepts:
- Disparate Treatment (Direct Discrimination): Modeled as a direct edge from Protected Attribute ( $A$ ) to Decision ( $D$ ).
- Disparate Impact (Indirect Discrimination): Modeled as an indirect path ( $A \to W \to X \to D$ ) where bias flows through mediators.
Short-term vs. Long-term Trade-offs:
- Fairness through Unawareness: Short-term negative total effect (widens disparity); Long-term neutral effect (due to "Limits to Success" balancing loops).
- Affirmative Action: Short-term neutral total effect (equalizes decisions); Long-term negative total effect (due to "Shifting the Burden" creating dependency and worsening actual outcomes).
The "Blind Spot" of Static Models: Static Causal Bayesian Networks (DAGs) cannot capture the feedback loop where outcomes ( $Y$ ) become future data ( $X$ ). This omission leads to an incomplete understanding of why disparities persist. System Dynamics (CLDs) are required to model this temporal evolution.

5. Significance

Policy Design: The framework provides a "common causal language" for policymakers, data scientists, and lawyers. It allows stakeholders to simulate the long-term consequences of fairness policies before implementation, moving beyond static compliance checks.
Sociotechnical Alignment: It offers a path to build AI systems aligned with democratic values by explicitly encoding the diverse mental models of bias held by different political groups.
Legal Compliance: By visualizing how bias flows through a system, businesses can better identify "redlining" proxies and design models that are robust against both disparate treatment and disparate impact claims.
Theoretical Advancement: It challenges the dominance of purely statistical or static causal approaches in AI fairness, arguing that emergent behavior (feedback loops) is the primary driver of persistent inequality, and thus must be modeled using System Dynamics.

In conclusion, the paper argues that solving algorithmic fairness requires moving "outside the black box" of the ML model to model the sociotechnical system in which the model operates. Only by combining machine learning, causal inference, and system dynamics can we understand the complex, dynamic trade-offs between different definitions of fairness.