💻 computer science

Bridging Predictions and Interventions: An Integrated Framework for Automated Decision-Systems

This paper proposes an integrated framework for automated decision systems that shifts the focus from prioritizing predictive accuracy to an intervention-oriented approach, emphasizing how predictions reshape organizational workflows and require a re-evaluation of design and deployment to better anticipate downstream societal consequences.

Original authors: Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou, Luke Guerdan, Jessica Hullman, Daniel Malinsky, Bryan Wilder, Simone Zhang, Hammaad Adam, Amanda Coston, Ben Laufer, Ezinne Nwankwo, Michael Zanger-Ti

Published 2026-06-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou, Luke Guerdan, Jessica Hullman, Daniel Malinsky, Bryan Wilder, Simone Zhang, Hammaad Adam, Amanda Coston, Ben Laufer, Ezinne Nwankwo, Michael Zanger-Tishler, Eli Ben-Michael, Avi Feller, Talia Gillis, Shion Guha, Daniel Ho, Lily Hu, Kosuke Imai, Sayash Kapoor, Joshua Loftus, Razieh Nabi, Juan Carlos Perdomo, Matthew Salganik, Mark Sendak, Berk Ustun, Suresh Venkatasubramanian, Angelina Wang, Ashia Wilson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: It's Not Just About the Crystal Ball

Imagine you have a crystal ball that can predict the future with 90% accuracy. You might think, "Great! If I can see the future, I can make perfect decisions."

This paper argues that in the real world, having a perfect crystal ball isn't enough. In fact, just handing a crystal ball to a decision-maker (like a judge, a doctor, or a teacher) often changes the whole game in ways the crystal ball itself didn't predict.

The authors say we are currently obsessed with making the crystal ball more accurate. But they argue we should stop focusing only on the prediction and start focusing on the intervention (the action taken because of the prediction).

The Problem: The "Crystal Ball" Paradox

The paper points out a frustrating reality:

In the lab: Automated systems (like those predicting if a criminal will re-offend, if a patient will get sick, or if a student will drop out) often look great. They have high accuracy scores.
In the real world: When these systems are actually used, they often fail to improve outcomes. Sometimes they make things worse.

Why? Because a prediction doesn't change the future by itself. It only changes the future if a human being sees it and does something different because of it.

The New Framework: The "Traffic Light" System

To understand how these systems actually work, the authors break them down into four parts, like a traffic light system:

The Data (X): The information you have (e.g., a patient's history, a student's grades).
The Prediction (R): The computer's guess (e.g., "High risk of sepsis").
The Assessment (Ŷ): How we translate that guess into a simple label for humans (e.g., turning the complex data into a red "ALERT" light or a "High Risk" sticker).
The Decision (D): What the human actually does (e.g., the doctor orders a test, the judge denies bail, the teacher assigns a tutor).
The Outcome (Y): What actually happens (e.g., the patient recovers, the student graduates).

The Missing Link:
Most people study the link between Data and Prediction. They ask, "Is the crystal ball accurate?"
The authors say we need to study the link between Prediction and Decision. They ask, "Does seeing the red light actually make the doctor act differently? Does it make them better at their job?"

The "Policy Change" (The Rulebook)

The paper introduces a concept called Policy Change (Z). Think of this as the rulebook of the organization.

When a hospital or a court introduces an automated system, they aren't just adding a tool; they are changing the rules of the game.

Before: A judge decides based on their gut feeling.
After: The judge sees a "Risk Score" and the rulebook says, "If the score is high, you must detain the person unless you write a special note explaining why."

The paper argues that you cannot judge the success of the system without understanding this new rulebook. If the rulebook forces a judge to act in a specific way, the outcome depends on the rule, not just the prediction.

Two Ways to Use the Crystal Ball

The authors explain that organizations usually use predictions in one of two ways, and they often confuse them:

The "Sick Person" Approach (Baseline Risk):
- Question: "Who is most likely to get sick if we do nothing?"
- Action: We give help to the sickest people.
- Analogy: Giving life jackets to the people who are already drowning.
- Problem: This helps the people who need it most, but it might not save the most lives overall if the resources are limited.
The "Rescue Mission" Approach (Intervention Effect):
- Question: "Who will benefit the most if we give them help?"
- Action: We give help to the people who will improve the most because of it.
- Analogy: Giving a life jacket to the person who is just starting to sink but can easily be saved, rather than the person who is already underwater and beyond help.
- Insight: The paper argues that for limited resources, we should often focus on the "Rescue Mission" approach (who benefits most) rather than just the "Sick Person" approach (who is worst off).

Why "Accuracy" Isn't the Whole Story

The paper uses a few metaphors to explain why high accuracy doesn't equal success:

The Alert Fatigue: If a fire alarm goes off every 5 minutes, even if it's 99% accurate, the firefighters will eventually stop listening. If an automated system flags too many people as "risky," the humans might ignore it.
The Zombie Prediction: Sometimes, the data the computer learns from is "zombie" data. For example, if a judge has been detaining people for years, the computer learns that "people who are detained don't re-offend." But that's only because they were locked up! The computer thinks locking them up caused them to be safe, but it's just a trick of the data.
The Human Element: A prediction is just a suggestion. If the human decision-maker doesn't trust the computer, or if they are too busy to act on it, the prediction is useless.

The Solution: A New Way to Build and Test

The authors propose a new "Integrated Framework" for building these systems. Instead of just asking "Is the math right?", we should ask:

Design: Are we trying to predict who is sick, or are we trying to figure out who will get better if we help them?
Evaluation: Don't just check if the prediction was right. Check if the decision changed and if the outcome improved. Did the system actually help the patient/student/defendant?
Implementation: Look at the whole environment. Who are the people involved? What are the rules? Do they have enough resources to act on the prediction?

The Bottom Line

The paper concludes that prediction is just one tool in a toolbox. You can have the most accurate prediction in the world, but if the organization doesn't change how it works, or if the humans don't know how to use the tool, nothing changes.

To make these systems work, we need to stop treating them like magic crystal balls and start treating them like policy changes that require careful planning, testing, and a deep understanding of how humans actually behave in the real world.

Technical Summary: Bridging Predictions and Interventions in Automated Decision Systems

Problem Statement

Automated Decision Systems (ADS) utilize statistical pattern recognition on historical data to predict individual future outcomes (e.g., recidivism, sepsis, student dropout) to inform bureaucratic decision-making. Despite the ubiquity of ADS in public sectors, there is a substantial gap between high predictive accuracy and improved real-world social outcomes.

The core problem identified is a "myopic focus" on standalone model properties, particularly predictive accuracy ( $R$ ), which fails to account for how predictions function as bureaucratic interventions. In practice, introducing predictions into operational workflows alters organizational processes, assessment categorizations, and decision-making behaviors. Current evaluation frameworks often treat ADS as isolated artifacts rather than components of a dynamic system where predictions ( $R$ ) influence assessments ( $\hat{Y}$ ), which in turn drive decisions ( $D$ ) under specific policy regimes ( $Z$ ), ultimately affecting outcomes ( $Y$ ). Consequently, optimizing for prediction accuracy does not necessarily translate to optimizing for downstream societal or organizational utility.

Methodological Framework: An Integrated View

The paper proposes an integrated framework that shifts the paradigm from a purely prediction-based approach to an intervention-oriented view. This framework expands the traditional $(X, R, Y)$ pipeline (Covariates, Risk Score, Outcome) to include the intermediate mechanisms of assessment and decision, as well as the institutional policy context.

The Expanded Causal Chain

The authors define the ADS lifecycle through the following components:

Covariates ( $X$ ): Individual-level information (e.g., demographics, history).
Risk Score ( $R$ ): The predictive model output, $R(X) = E[Y|X]$ .
Assessment ( $\hat{Y}$ ): The mapping of $R$ into categorical labels (e.g., "high-risk" vs. "low-risk") via thresholding ( $\hat{Y} = I[R(X) > t]$ ). This step determines how information is sorted and presented to decision-makers.
Decision ( $D$ ): The intervention actions taken by human decision-makers (e.g., detention, release, resource allocation) based on the assessment.
Outcome ( $Y$ ): The realized result (e.g., recidivism, recovery).
Policy Change ( $Z$ ): The institutional context defining the workflow, including whether ADS is deployed, which cases are subject to it, and the available set of actions for decision-makers.

Theoretical Tools

To analyze this chain, the paper integrates tools from causal inference and decision theory:

Potential Outcomes Framework: Used to distinguish between baseline risk and the causal effect of decisions. The paper defines the Average Treatment Effect (ATE) and Conditional Average Treatment Effect (CATE) to measure how decisions $D$ causally affect outcomes $Y$ for specific covariates $X$ .
Interventional vs. Predictive Targeting: The framework contrasts two prioritization strategies:
- Prediction View: Targeting based on baseline risk ( $\pi_{risk}(X) = I[E[Y(0)|X] > t]$ ).
- Policy Intervention View: Targeting based on the heterogeneous treatment effect ( $\pi_{ITR}(X) = I[E[Y(1) - Y(0)|X] > t]$ ), which prioritizes individuals who benefit most from an intervention.
Causal Inference for Evaluation: The paper advocates for moving beyond predictive benchmarking to evaluate the causal impact of ADS on decisions and outcomes. It suggests utilizing quasi-experimental methods, randomized controlled trials (RCTs), and event studies to estimate the effects of the policy change $Z$ on the decision process $D$ and subsequent outcomes $Y$ .

Key Contributions

The paper does not present new empirical results or a specific algorithmic model. Instead, its primary contribution is a conceptual and technical framework for studying ADS in social systems.

Reframing ADS as Interventions: The authors argue that ADS should not be viewed merely as prediction tools but as bureaucratic interventions that modify organizational workflows. The impact of an ADS is mediated by how organizations operationalize the tool (display, interpretation, and action).
Integrated Lifecycle Stages: The paper structures the ADS lifecycle into three interrelated stages, proposing specific shifts for each:
- Model Design: Organizations should explicitly choose between predictive targeting (baseline risk) and interventional targeting (treatment effect) based on domain context and values, rather than implicitly adopting tools designed for the former. It also highlights the need to address "selective labels" and "zombie predictions" where historical decisions censor or distort outcome data.
- Evaluation Science: Evaluation must move beyond predictive accuracy (e.g., AUC, FPR) to decision-centric metrics. This includes measuring the causal impact of ADS on decision-making processes (e.g., does the tool change who gets detained?) and the downstream outcomes of those decisions. The paper notes that standard evaluation often fails to account for the "fundamental problem of causal inference" in non-randomized settings and suggests methods to handle overlap violations.
- Implementation Science: The framework emphasizes the importance of the "context of deployment," including organizational constraints, resource limitations, and legal/regulatory environments. It argues that the interaction between humans and ADS is a central design choice, not a side effect, and that the "loop" structure (automation vs. decision support) must be calibrated to the specific institutional context.
Unified Vocabulary: The paper provides an expanded technical vocabulary to bridge gaps between computer science (model properties), behavioral science (human response to predictions), and social science (organizational and policy impacts).

Results and Claims

As a perspective paper, it does not report experimental results. Its "results" are the logical deductions and synthesis of existing literature (e.g., failures of the Epic Sepsis Model, mixed results of pretrial risk assessment tools) to support the following claims:

Accuracy is Insufficient: High predictive accuracy does not guarantee improved downstream outcomes because the relationship between prediction and outcome is mediated by human decisions and organizational constraints.
Context is Determinative: The same predictive model can have vastly different impacts depending on the policy regime ( $Z$ ), the available interventions ( $D$ ), and the organizational capacity to act on predictions.
Evaluation Gaps: Current evaluation practices are fragmented. Studies often focus on isolated links (e.g., $R \to Y$ or $Z \to Y$ ) rather than the full causal chain ( $R \to \hat{Y} \to D \to Y$ ), leading to an incomplete understanding of ADS efficacy.
Design Implications: To improve social welfare, developers and policymakers must consider "universal allocation," "resource expansion," and "intervention design" alongside predictive modeling. Improvements in predictive accuracy are often outweighed by improvements in policy levers such as expanding access to resources.

Significance

The paper claims its significance lies in bridging the gap between model properties and real-world consequences. By shifting the priority from a purely prediction-based paradigm to an intervention-oriented view, the framework aims to:

Improve the understanding of how ADS manifest in practice.
Provide a more meaningful basis for anticipating downstream societal and organizational consequences.
Inform new methods for the design, evaluation, and implementation of ADS that are grounded in the pragmatic context of organizational decision-making.
Enable a more rigorous assessment of whether ADS-informed decisions actually improve social system outcomes, including their procedural effects.

The authors maintain a modest stance, acknowledging that while the framework offers a new vocabulary and structure, the actual implementation requires navigating complex sociotechnical constraints and that the field currently underestimates the layers of context in which ADS operate.