🤖 AI

Quantifying Harm

This paper expands on a previous qualitative definition of harm by developing a quantitative framework for measuring individual and societal harm under uncertainty, demonstrating that simple aggregation methods can yield counterintuitive results and proposing decision-theoretic alternatives, particularly in the context of precision medicine.

Original authors: Sander Beckers, Hana Chockler, Joseph Y. Halpern

Published 2026-05-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Sander Beckers, Hana Chockler, Joseph Y. Halpern

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: From "Did it hurt?" to "How much did it hurt?"

Imagine you are a judge trying to decide if a new AI system caused harm. In the past, the authors (Beckers, Chockler, and Halpern) had a simple rule: Yes or No. Did the AI cause harm? If the answer was "Yes," that was it.

But in the real world, we need to be more precise. We don't just want to know if harm happened; we want to know how bad it was so we can choose the best option. This paper is about building a mathematical ruler to measure the "amount" of harm, rather than just a light switch that says "on" or "off."

1. The Baseline: What is "Normal"?

To measure harm, you need a starting point. Think of it like a thermostat.

The Default Utility: This is the "normal" temperature of a room.
The Outcome: This is the actual temperature after the heater or AC runs.

If the room is supposed to be 70°F (the default) and the heater makes it 75°F, that's a benefit. If the AC makes it 60°F, that's a harm. The amount of harm is simply the difference between where you should be and where you actually ended up.

The Twist: The paper argues that "normal" isn't always zero. Sometimes, "normal" is a range.

Analogy: Imagine tipping a waiter.
- The Range: A tip between 15% and 20% is "normal." It's neither good nor bad; it's just expected.
- Harm: If you tip 5%, you have caused harm (you are below the floor).
- Benefit: If you tip 50%, you have created a benefit (you are above the ceiling).
- The Point: You can't just say "more money is always better." There is a "sweet spot" where nothing happens.

2. The Dice Roll: Dealing with Uncertainty

Life is rarely certain. Sometimes a doctor's surgery cures a patient; sometimes it kills them. How do we measure harm when the outcome is a gamble?

The paper looks at how people actually think about risk, which is often weird.

The "Driverless Car" Problem: Imagine a self-driving car.
- Option A: Drive at the speed limit. There is a 1 in a million chance of a fatal crash.
- Option B: Drive 20% slower. There is a 1 in 2 million chance of a fatal crash.
- The Math: Option B is safer. If you just do the math (Expected Utility), you should always choose B.
- The Reality: People often prefer Option A. Why? Because our brains treat a 1-in-a-million chance as "basically zero." We ignore tiny risks.

The authors suggest we use Probability Weighting. Instead of treating a 1% risk and a 0.0001% risk linearly, we apply a "weight" to them.

Analogy: Think of a magnifying glass.
- Sometimes we use a magnifying glass that makes tiny risks look huge (like fearing a terrorist attack after hearing about it).
- Sometimes we use a "dimmer switch" that makes tiny risks disappear (like ignoring the risk of a car crash because we drive every day).
- To measure harm accurately, we must account for how humans actually perceive these odds, not just the raw numbers.

3. The Group Problem: Fairness and Aggregation

What happens when a policy hurts 1,000 people? Do we just add up the pain?

The "Sum" Trap: If Policy A hurts 1,000 random people by a tiny bit, and Policy B hurts 1 specific person by a lot, a simple math sum might say they are equal.
The Fairness Issue: Intuitively, we feel differently about these two. Hurting 1,000 random people feels different from targeting 1 specific person (or a specific group, like a minority community).

The paper proposes a Fairness Penalty.

Analogy: Imagine a school cafeteria.
- If the cafeteria accidentally gives a bad lunch to 100 random students, that's annoying.
- If the cafeteria only gives bad lunches to the students sitting at Table 5, that feels like bullying.
- The authors suggest that our "harm calculator" should add a massive penalty if a policy disproportionately hurts a specific, identifiable group. It's not just about the total number of hurt people; it's about who gets hurt.

4. The Precision Medicine Debate

The paper connects these ideas to a recent argument in medicine about "Precision Medicine" (tailoring treatments to specific genes).

The Conflict: Some experts say, "Treat the patient if the average benefit is positive." Others say, "No, we must prioritize avoiding harm to the individual, even if the average benefit is positive."
The Authors' View: They show that this debate is actually just a specific version of the problems they already solved.
- The "Average Benefit" approach ignores the "Default" (what happens if we do nothing).
- The "Avoid Harm" approach often relies on a specific definition of causality (the "But-For" test: "Would they have died but for the treatment?").
- The authors argue that the medical debate is missing the nuance of context. What is "harm" depends on what the patient's life was like before the treatment. If a patient is already dying, a risky treatment might not be "harmful" even if it kills them, because the alternative was death anyway.

5. The Hard Part: The Math is Tricky

Finally, the paper admits that calculating this is computationally very hard.

Analogy: Imagine trying to solve a massive Sudoku puzzle where every time you move a number, the rules of the puzzle change slightly.
The authors prove that figuring out exactly "how much" harm occurred is a problem that takes a super-computer a very long time to solve in the worst-case scenario.
However: They argue that in real life, the puzzles aren't usually that big. Most decisions involve a manageable number of variables, so we can still use these definitions in practice.

Summary

This paper builds a sophisticated tool to measure harm. It moves beyond simple "Yes/No" answers to ask:

How much worse is the outcome compared to the "normal" baseline?
How do we adjust for how humans perceive risk (ignoring tiny risks vs. fearing them)?
How do we ensure we aren't unfairly targeting specific groups?

By answering these questions, the authors hope to help AI systems, doctors, and policymakers make decisions that align better with human intuition about what is truly "harmful."

Technical Summary: Quantifying Harm

Problem Statement
While Artificial Intelligence systems increasingly influence critical domains such as healthcare, hiring, and autonomous driving, defining and measuring "harm" remains a significant challenge. Existing regulatory frameworks, such as the European AI Act, require the assessment of both the severity and probability of harm, yet the philosophical literature often describes the concept of harm as a "Frankensteinian jumble" of conflicting accounts. Previous work by the authors established a qualitative definition of harm (determining whether harm occurred or not) based on causal models and a default utility. However, practical applications require a quantitative notion of harm to compare interventions, aggregate harm across populations, and account for uncertainty. The paper addresses the gap between qualitative causation and the quantitative metrics required for policy-making and ethical decision-making.

Methodology
The authors develop a quantitative framework grounded in structural causal models and the Halpern-Pearl definition of actual causation. The methodology proceeds through several stages:

Quantitative Harm in a Deterministic Context:
The authors define quantitative harm ($QH$) for a single agent in a fixed context. It is calculated as the difference between the utility of the actual outcome and a baseline, bounded by a "default utility" ( $d$ ). Specifically, if an action $\vec{X}=\vec{x}$ causes an outcome $O=o$ rather than a contrastive outcome $O=o'$ , the harm is $\max(0, \min(d, u(o')) - u(o))$ . This formulation ensures harm is only registered if the actual utility is lower than both the contrastive utility and the default utility.
Handling Uncertainty (Probability Weighting):
To address uncertainty regarding contexts, the paper moves from simple expected harm to Weighted Expected Quantitative Harm (WEQH). Recognizing that human decision-making often deviates from strict expected utility maximization (e.g., overweighting small probabilities or underweighting them based on experience), the authors incorporate a probability weighting function $w$ . The WEQH is the sum of the weighted probabilities of contexts multiplied by the quantitative harm in those contexts. This allows the model to capture phenomena such as the preference for avoiding rare catastrophic events (overweighting) or ignoring negligible risks in daily activities (underweighting).
Aggregating Societal Harm and Fairness:
The paper critiques the "obvious" approach of simply summing individual harms, noting that it fails to account for fairness and the disproportionate impact on specific subpopulations. The authors propose a Collective Utility Model that introduces a penalty term ( $\alpha$ ) if a predefined identifiable group ( $G$ ) suffers an average harm significantly higher ( $\beta$ ) than the population average. This mechanism allows the framework to penalize policies that concentrate harm on specific groups, even if the total aggregate harm is low.
Harm vs. Benefit Asymmetry:
Unlike standard cost-benefit analyses that treat benefit as the symmetric opposite of harm, the authors propose a default interval $D = [d_h, d_b]$ . Outcomes below $d_h$ constitute harm, outcomes above $d_b$ constitute benefit, and outcomes within the interval are neutral. This captures the intuition that there is a "safe" range of outcomes where neither harm nor benefit is incurred.
Complexity Analysis:
The appendix analyzes the computational complexity of determining and computing harm. Deciding whether harm occurs (qualitative) is shown to be DP-complete, while computing the extent of quantitative harm is $FP^{NP[\log n]}$ -complete. The authors note that while these classes suggest intractability in the worst case, practical applications with small variable sets or symmetry considerations may render the problem feasible.

Key Contributions and Results

Formal Definition: The paper provides the first formal, quantitative definition of harm that integrates causal models, utility theory, and a default baseline.
Refutation of Simple Aggregation: The authors demonstrate that summing expected harms can lead to counterintuitive results, particularly regarding fairness and the distribution of risk. They show how probability weighting and group-based penalties can resolve these paradoxes (e.g., Norcross's cycle of harm).
Comparison with RBT: The paper offers a detailed comparison with the approach of Richens, Beard, and Thompson (RBT). The authors argue that RBT's reliance on "but-for" causation and a single default action (no treatment) leads to flawed conclusions, such as implying that not treating a patient causes no harm. The authors' use of a more general causality definition and a flexible default utility yields more nuanced and medically plausible results.
Precision Medicine Context: The framework is applied to a recent debate in precision medicine (involving Dawid, Senn, Sarvet, Stensrud, Mueller, and Pearl). The authors show that the debates regarding treatment rules (e.g., maximizing Average Treatment Effect vs. weighing benefit against harm) are essentially special cases of the broader issues of default utility and probability weighting addressed in their framework.

Significance and Claims
The paper positions itself as a foundational step toward a formal approach for determining harm in practice, particularly for AI systems and public policy. The authors claim modestly that this work constitutes a "first step" rather than a complete solution. They emphasize that:

The definition of quantitative harm for a single agent in a fixed context is straightforward, but subtleties arise with uncertainty and aggregation.
The proposed approach for fairness (penalizing disproportionate harm) is a sketch that requires further empirical validation to ensure it aligns with human heuristics.
The handling of probability weighting (overweighting vs. underweighting) is complex and context-dependent, requiring policymakers to weigh normative and descriptive considerations rather than assuming a universal weighting function.
The framework integrates harm and blame, suggesting a path toward a full theory of moral responsibility, though the paper focuses primarily on the mechanics of quantifying harm.

Ultimately, the paper argues that a rigorous, causality-based quantitative definition of harm is critical for the ethical deployment of AI and the formulation of regulations like the European AI Act, moving beyond the "mess" of qualitative philosophical debates to actionable metrics.