🤖 AI

Causal Explanations for Image Classifiers

This paper introduces ReX, a novel black-box framework grounded in formal actual causality theory that computes efficient, minimal, and high-quality explanations for image classifiers, outperforming existing state-of-the-art tools in both speed and explanation size.

Original authors: Hana Chockler, David A. Kelly, Daniel Kroening, Youcheng Sun

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hana Chockler, David A. Kelly, Daniel Kroening, Youcheng Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, but secretive, robot that looks at a picture and says, "That's a ladybug!" You ask, "Why?" The robot can't tell you because it's a "black box"—its internal gears and wires are hidden, and even the people who built it don't fully understand how it makes its decisions.

This paper introduces a new way to ask that "Why?" question and get a clear, honest answer without needing to open the robot's hood. The authors call their method rex.

Here is the breakdown of how it works, using simple analogies:

1. The Problem: The "Heat Map" vs. The "Truth"

Most current tools try to explain the robot's decision by drawing a "heat map" on the picture. They color the parts of the image they think are important in red and the unimportant parts in blue.

The Flaw: These heat maps are often fuzzy. They might highlight the whole ladybug, but also the grass, the sky, and a random leaf nearby. It's like a detective pointing at a crime scene and saying, "The culprit was somewhere in this whole neighborhood." It's not very helpful if you want to know exactly who did it.

2. The Solution: The "Minimal Ingredient" Test

The authors, led by Hana Chockler and colleagues, decided to use a concept from logic called causality. Instead of guessing what is important, they ask a simple question: "What is the absolute minimum amount of this picture I need to keep to still get the same answer?"

Think of it like a recipe:

If you are baking a cake and the robot says, "This is a chocolate cake," a heat map might say, "The flour, sugar, eggs, and the bowl are all important."
The rex method is like a chef who says, "Actually, if I remove the bowl, the cake is still a cake. If I remove the sugar, it's not. If I remove the flour, it's not. But if I keep just the flour, sugar, and eggs, and throw away the rest, it's still a chocolate cake."

rex finds the smallest possible group of pixels (the "ingredients") that are strictly necessary for the robot to say "Ladybug." If you cover up everything else, the robot still sees a ladybug.

3. How It Works: The "Blindfolded Detective"

Since the robot is a black box, rex can't look inside. Instead, it plays a game of "cover and uncover":

The Partition: It cuts the image into four random chunks (like slicing a pizza).
The Test: It covers up different combinations of these chunks and asks the robot, "Is it still a ladybug?"
The Refinement: If the robot says "Yes" even when a chunk is covered, that chunk wasn't important. If the robot says "No" when a chunk is covered, that chunk is vital.
The Iteration: It keeps cutting the "vital" chunks into smaller and smaller pieces, over and over again, until it finds the tiniest possible pixels that matter.
The Average: To make sure it didn't get lucky with one specific way of cutting the pizza, it repeats this process many times with different cuts and averages the results.

4. The Results: Smaller, Sharper, and Faster

The authors tested rex against other popular tools (like Grad-CAM, LIME, and SHAP) using thousands of images. Here is what they found:

Tiny Explanations: rex produces explanations that are incredibly small. In many cases, the "explanation" covers less than 5% of the image. Other tools often highlight 10% to 50% of the image, including a lot of "noise" (irrelevant background).
Precision: Because rex only highlights what is strictly necessary, it avoids pointing at irrelevant things. For example, if a picture of a bus has a person standing in front of it, rex correctly ignores the person and focuses only on the bus parts that prove it's a bus.
Speed: Even though it does a lot of testing, it is surprisingly fast. It is the most efficient "black box" tool tested, often beating tools that are allowed to peek inside the robot's brain (white-box tools).

5. A Twist on "Deleting"

The paper also challenges a common way of measuring quality called the "deletion curve."

The Old Way: People thought a good explanation should make the robot's confidence drop fast if you start deleting the highlighted pixels.
The New Insight: The authors argue this is misleading. Sometimes an image has multiple reasons for a classification (e.g., a picture of a starfish might be recognized by its left arm OR its right arm). If you delete the left arm, the robot still sees the right arm and stays confident.
The Analogy: Imagine a security system that opens if you have a key OR a fingerprint. If you remove the key, the system still opens because of the fingerprint. A "bad" deletion score doesn't mean the explanation is wrong; it means the system found multiple valid reasons. rex is smart enough to find all these different "keys," whereas other tools usually only find one.

Summary

The paper presents rex, a tool that treats an AI image classifier like a mystery to be solved by elimination. Instead of guessing what the AI sees, it systematically covers up parts of the image until it finds the tiny, essential "fingerprint" that proves the AI's decision. The result is an explanation that is smaller, more precise, and more reliable than any other black-box method currently available.

Technical Summary: Causal Explanations for Image Classifiers

Problem Statement
Neural networks (NNs) function as complex, non-linear "black boxes," making it difficult to trace how pixel data is interpreted to produce a classification. This opacity is particularly problematic in proprietary systems (e.g., healthcare) where internal layers are inaccessible. Existing Explainable AI (XAI) techniques for image classifiers typically rely on pixel rankings or heat maps derived from gradients, propagation, or perturbation. However, these methods lack a principled approach grounded in formal definitions of cause and explanation. Furthermore, existing tools often produce explanations that are large, contain extraneous information, or rely on arbitrary initial segmentations (e.g., LIME) or unguided random occlusions (e.g., RISE). The paper addresses the need for a formal, rigorous definition of explanation suitable for image classifiers and an efficient method to compute it without opening the model's black box.

Methodology
The authors propose a novel black-box approach grounded in the theory of actual causality, specifically adapting Halpern's [2019] definitions to image classification.

Formal Definition: The paper defines an explanation as a minimal subset of pixels with their original values that is sufficient to obtain the same classification as the original image, provided all other pixels are masked (replaced with data from a predefined dataset). The image classifier is modeled as a depth-2 causal model where input pixels are exogenous variables and the classification is the output.
Intractability: The authors prove that computing the precise minimal explanation is intractable (exponential in the number of pixels). Consequently, they propose an approximation algorithm.
The rex Algorithm: The proposed tool, rex, computes approximate explanations through two main stages:
- Causal Ranking: The algorithm partitions the image into rectilinear "superpixels" (initially 4). It calculates the "degree of sufficient responsibility" for each superpixel by testing combinations of masked/unmasked regions against the model. This process is iterative: high-responsibility superpixels are recursively subdivided and re-evaluated. To mitigate the bias of any single partition, the algorithm runs multiple times with random initial partitions, averaging the resulting responsibility scores to create a pixel-level ranking.
- Explanation Extraction: Using the ranked list of pixels, a greedy algorithm constructs the explanation. It adds pixels from the highest-ranked list to an empty set until the model's output on this subset matches the original classification.
Complexity: The authors prove the algorithm terminates and demonstrate that the number of model calls is linear in the size of the image ( $O(2^s n N)$ , where $s$ is the partition size, $n$ is the number of pixels, and $N$ is the number of iterations), making it computationally feasible.

Key Contributions

Formal Framework: The paper establishes a formal definition of explanation for image classifiers based on actual causality, distinguishing it from heuristic or gradient-based importance measures.
Algorithm and Tool (rex): It introduces an efficient, black-box algorithm for computing approximate minimal explanations and implements it in an open-source tool.
Theoretical Guarantees: The authors provide proofs for the termination and complexity of their algorithm, as well as a discussion on the approximation ratio compared to the precise definition.
Quality Metrics: The paper critically evaluates standard XAI metrics, arguing that while insertion curves and explanation size are robust, deletion curves may be misleading for models with multiple valid explanations (as high deletion AUC can result from multiple distinct explanatory regions rather than poor quality).

Experimental Results
The authors evaluated rex against state-of-the-art white-box (e.g., Grad-CAM, LRP), grey-box (e.g., Integrated Gradients, GradientSHAP), and black-box (e.g., LIME, RISE, KernelSHAP) tools across four datasets (ImageNet-1k-mini, VOC2012, ECSSD, and a custom "Photobombing" dataset) and three models (ResNet50, ViT, ConvNext).

Explanation Size: rex consistently produced the smallest explanations, averaging between 3% and 5% of the image size. This is significantly smaller than other black-box tools and often outperforms white-box methods like Grad-CAM.
Efficiency: rex is the most efficient black-box tool, with execution times comparable to other perturbation-based methods (approx. 4–6 seconds including extraction) and significantly faster than RISE.
Quality Measures:
- Insertion Curves: rex achieved the highest Area Under Curve (AUC) for insertion, indicating that its identified pixels are highly effective at restoring classification confidence.
- Segmentation Overlap: On datasets with human-provided segmentation, rex explanations showed high overlap with the relevant object and minimal overlap with irrelevant background or occlusions.
- Deletion Curves: The authors note that rex often has higher deletion AUC than some competitors. They argue this is not a failure but a feature: rex identifies multiple distinct explanatory regions (e.g., multiple parts of an object), and deleting pixels from one region does not immediately destroy the signal from the others, unlike tools that identify a single diffuse region.

Significance and Claims
The paper claims that rex represents a significant advancement in black-box XAI by providing explanations that are not only computationally efficient but also theoretically grounded in causality. The authors assert that rex produces the smallest, most precise explanations among black-box tools, often outperforming white-box methods in terms of explanation size and concentration.

Crucially, the paper argues that the small size of rex explanations (often <5% of the image) implies that further reduction is not useful, as the identified pixels are sufficient for the classification. The authors also highlight that their approach does not rely on pre-existing image segmentation, making it robust across different datasets and model architectures where segmentation masks may be unavailable or inaccurate. The work demonstrates that black-box methods can achieve explanation quality comparable to, or better than, white-box methods when guided by formal causal definitions.