🤖 machine learning

Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses

This paper introduces DECAF, a novel framework that decomposes perturbation-based model explanations into distinct evidence, contradiction, and fragility components, demonstrating that this trajectory-aware approach significantly outperforms traditional magnitude-based attribution methods in accuracy, efficiency, and interpretability across diverse vision and tabular tasks.

Original authors: Lei You

Published 2026-08-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lei You

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand why a friend made a specific choice, like picking a red shirt over a blue one. You might ask, "How much did they react to the color?" If they grabbed the red shirt instantly, you'd say, "Wow, huge reaction!" But here's the tricky part: a huge reaction doesn't always mean the same thing. Maybe they loved the red shirt (a good reason). Maybe they hated the blue shirt so much they grabbed the red one just to get away from it (a bad reason). Or maybe they grabbed the red shirt because they were confused, only to realize halfway through that they actually preferred the blue one all along (a fleeting, fragile reaction).

This is the world of "AI explainability," a field where scientists try to peek inside the "black box" of computer models to see what they are thinking. For a long time, the main tool for this was measuring the "magnitude" of a reaction—basically, how big the change in the AI's answer was when you tweaked the input. It's like measuring how loud a door slams. But as this new paper points out, knowing the door slammed hard doesn't tell you if it was slammed in anger, in joy, or by a gust of wind that didn't even stick. The author, led by Lei You from the Technical University of Denmark, argues that we've been ignoring the meaning behind the noise, and they've built a new way to sort it out.

The paper introduces a method called DECAF (which stands for Decomposition of Evidence, Contradiction, and Fragility). Think of DECAF as a super-smart detective that doesn't just listen to how loud the door slammed, but watches the whole story of how the person approached the door.

Here is how DECAF works, using a simple story: Imagine you are watching a magic trick where a magician changes a card from a King to a Queen.

Evidence: As the magician reveals the card, if the change feels like it's building up to the final result (the Queen), DECAF calls this Evidence. It's the part of the reaction that supports the final decision.
Contradiction: Sometimes, the card might look like a King for a split second, then a Queen, then back to a King, before finally settling on the Queen. If the reaction goes against the final result at any point, DECAF calls this Contradiction. It's like the AI saying, "Wait, I thought it was a King!" before changing its mind.
Fragility: Finally, imagine the magician waves a card that looks like a Queen, but when you look closely at the end, it's actually just a blank piece of paper. The card seemed to change, but the final result was "nothing." If the AI reacts strongly during the trick but the final answer is basically zero, DECAF calls this Fragility. It's a reaction that exists only because of the path taken, not because of a real difference.

The author tested this idea in a few different ways. First, they created controlled video games and puzzles where they knew exactly what the AI should be doing. They found that DECAF could spot when an AI was relying on a "shortcut" (like guessing based on the background color) versus the actual object. If the AI was using a shortcut, DECAF's "Evidence" score would match that behavior perfectly. If the AI was confused or changing its mind, the "Contradiction" score would spike.

Then, they took this to the real world with a massive test involving 72 different AI models looking at images from the ImageNet-9 dataset (a collection of pictures with different backgrounds). They compared cases where the AI had the exact same "loudness" of reaction but different underlying behaviors.

The Result: If you just looked at the loudness (magnitude), you would only guess the correct behavior 35.0% of the time. It was basically a coin flip.
The DECAF Fix: When they used DECAF to look at the type of reaction (Evidence vs. Contradiction vs. Fragility), they got the right answer 96.4% of the time.

The paper also discovered something surprising about how we reveal information to the AI. They tested two ways of showing an image: one where the whole picture slowly fades in (like a blend), and another where the image is revealed patch by patch (like a puzzle). They found that changing how the image is revealed changed the total "loudness" of the reaction by nearly 80%. However, the "Evidence" (the good, solid reasons) barely changed at all. Instead, the "Fragility" (the confusing, path-dependent noise) exploded, growing by more than 4 times. This proves that the total size of a reaction can be misleading; it often just tells you how sensitive the AI is to the method of showing the picture, not the picture itself.

Finally, the author showed that DECAF is incredibly efficient. On a huge AI model with 1 billion parameters (a DINOv2 model), DECAF could do its job with 4.75 times less time and 2.36 times less memory than other popular methods, while still getting the job done just as well.

In short, the paper suggests that we've been too focused on how much an AI reacts and not enough on what that reaction means. By breaking reactions down into Evidence (support), Contradiction (opposition), and Fragility (fleeting noise), DECAF gives us a much clearer, more honest picture of what these digital brains are actually thinking. It's a tool that doesn't just measure the slam of the door, but helps us understand why it slammed in the first place.

Technical Summary: DECAF – Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses

1. Problem Statement

Perturbation-based explanation methods interpret model decisions by measuring prediction changes under altered inputs. However, a fundamental semantic gap exists: the standard metric, response magnitude (the absolute difference in predictions), quantifies how much a model reacts but fails to explain what that reaction means.

The paper identifies that identical response magnitudes can arise from three distinct behavioral regimes:

Alignment: The reaction supports the final factual–counterfactual difference.
Opposition: The reaction opposes the final difference (e.g., an intermediate step suggests one outcome, but the final state suggests another).
Fragility: The reaction is strong along the perturbation path but vanishes at the endpoint, indicating sensitivity to the reveal process rather than a meaningful final effect.

Treating these distinct behaviors as equivalent obscures critical interpretive information. Existing methods often reduce complex trajectories to a single scalar or rely on gradient-based internal signals, missing the semantic role of intermediate responses relative to the final contrast.

2. Methodology: DECAF

The author introduces DECAF (Decomposition of Evidence, Contradiction, And Fragility), a model-agnostic framework that decomposes paired perturbation responses into three semantic components.

Formal Setup

Given a model score function $q$ , a factual input $x^+$ , and a counterfactual $x^-$ , the final contrast is $d = q(x^+) - q(x^-)$ . A paired reveal trajectory $(x^+(t), x^-(t))$ for $t \in [0, 1]$ starts from an uninformative state $x^0$ and progresses to the endpoints. The signed stage response is $r(t) = q(x^+(t)) - q(x^-(t))$ .

Semantic Routing

DECAF uses the final contrast $d$ as a semantic reference to route intermediate responses $r(t)$ :

Gate ( $a$ ): A binary gate determines if the final contrast is meaningful ( $|d| \ge \epsilon$ ).
Orientation ( $s$ ): The sign of $d$ ( $s = \text{sign}(d)$ ) defines the "correct" direction.
Decomposition: The response is routed into three components:
- Evidence ( $E$ ): Responses aligned with the final direction when the final contrast is active ( $a=1, s \cdot r(t) > 0$ ).
- Contradiction ( $C$ ): Responses opposing the final direction when the final contrast is active ( $a=1, s \cdot r(t) < 0$ ).
- Fragility ( $F$ ): Responses occurring when the final contrast is negligible ( $a=0$ ), representing sensitivity that does not persist to the endpoint.

The decomposition is lossless: $|r(t)| = e(t) + c(t) + f(t)$ . The aggregate metrics are the expected values of these components over the trajectory and dataset.

Theoretical Foundations

The paper proves that under three operational axioms—Conservation (no magnitude is lost), Endpoint Gating (fragility is isolated to negligible endpoints), and Directional Support (evidence/contradiction are separated by sign)—this routing is unique. Furthermore, DECAF is a strict refinement of ordinary magnitude; while magnitude can be derived from DECAF ($Abs = E + C + F$), the reverse is impossible, as distinct behavioral profiles (e.g., pure evidence vs. pure contradiction) can yield identical magnitudes.

3. Key Contributions

Problem Identification: Explicitly defining the semantic gap in perturbation explanations where magnitude fails to distinguish between alignment, reversal, and path-dependent sensitivity.
Paired Reveal Interface: Introducing a framework to track factual–counterfactual contrasts along matched trajectories as information is progressively revealed.
DECAF Algorithm: A lossless, unique decomposition of responses into Evidence, Contradiction, and Fragility based on endpoint-relative semantics. It requires no gradients, internal activations, or learned explainer models.
Behavioral Validation: Demonstrating that these components track independently measured behaviors (shortcut reliance, effect reversal, and off-path sensitivity) across neural, linear, and tree-based models.
Practical Utility: Showing that DECAF recovers response semantics even when magnitudes are matched and provides stable cross-protocol diagnostics.

4. Experimental Results

Controlled Validation (3D Shapes & Covertype)

Evidence: In vision tasks with correlated shortcuts (e.g., wall color vs. object shape), the Evidence component ( $E$ ) strongly correlates (Spearman $\rho = 0.936$ ) with a model's vulnerability to shortcut reversal.
Fragility: In tasks designed to induce sensitivity to intermediate states (e.g., floor color changes), the Fragility component ( $F$ ) tracks the rate of prediction changes on endpoint-null pairs ( $\rho = 0.974$ ).
Contradiction: In tasks where an effect reverses direction under context change, Contradiction ( $C$ ) tracks the label-swap rate ( $\rho = 0.961$ ), whereas ordinary magnitude fails ( $\rho = -0.036$ ).
Tabular Transfer: On the Covertype dataset, DECAF components successfully track preservation, inversion, and endpoint-null changes across 135 classifiers (linear, tree, and neural), outperforming SHAP and other baselines in mechanism identification.

Real-World Audit (ImageNet-9)

Semantic Recovery: On a 72-model zoo (24 off-the-shelf ImageNet-1k models and 48 fine-tuned ImageNet-9 models), DECAF correctly identifies the dominant behavioral role (Evidence, Contradiction, or Fragility) in 96.4% of cases where models have nearly identical response magnitudes. In contrast, magnitude alone achieves only 35.0% accuracy.
Path Sensitivity: Changing the reveal path (e.g., from a global blend to a patch-by-patch reveal) increased the total response magnitude by nearly 80%, but this increase was driven almost entirely by Fragility and Contradiction, while Evidence remained stable. This demonstrates that higher magnitude does not necessarily imply stronger evidence.
Ranking Stability: Model rankings based on ordinary magnitude were highly unstable across different reveal paths ( $\rho \approx 0.17$ ), whereas rankings based on Evidence and Contradiction remained stable ( $\rho > 0.77$ ).

Attribution Benchmarks & Scaling

Performance: On FunnyBirds and ImageNet-1k, short forward-only DECAF trajectories (3–9 stages) outperformed tested general-purpose attribution baselines (DeepLIFT, Integrated Gradients, SmoothGrad, RISE).
Efficiency: On a 1B-parameter DINOv2 model, a short DECAF trajectory matched the quality of a strong gradient-based baseline (Integrated Gradients) with 4.75× lower wall time and 2.36× lower peak memory, as DECAF requires only forward passes.

5. Significance and Claims

The paper claims that DECAF addresses a semantic ambiguity inherent in perturbation methods. By decomposing the response trajectory relative to the final contrast, it reveals what kind of response produced a given magnitude.

Interpretability: It distinguishes between a model that consistently relies on a feature (Evidence), one that is confused or reverses its decision (Contradiction), and one that is sensitive to the specific path of revelation but not the final state (Fragility).
Robustness: The components provide more stable diagnostics across different perturbation protocols than raw magnitude.
Efficiency: It offers a high-quality, model-agnostic explanation method that scales efficiently to large models without requiring gradients or internal access.

The author emphasizes that these roles are relative to the chosen counterfactual pair, score, and reveal path; they do not infer latent causal mechanisms but rather describe the observable behavior of the model's response to a specific intervention. The work suggests that future explanation efforts should move beyond scalar magnitudes to consider the semantic composition of the response trajectory.