🤖 machine learning

FRInGe: Distribution-Space Integrated Gradients with Fisher--Rao Geometry

The paper proposes FRInGe, a novel attribution method that replaces heuristic input baselines and straight-line paths with a maximum-entropy reference and Fisher-Rao geodesic interpolation in predictive distribution space to produce more robust and calibration-oriented explanations for deep learning models.

Original authors: Gabriele Martino, Sebastian Tschiatschek

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gabriele Martino, Sebastian Tschiatschek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Why Current AI Explanations Are "Brittle"

Imagine you have a very smart AI that looks at a picture of a cat and says, "That's a cat!" You want to know why. You ask the AI to point out the pixels that made it decide "cat."

One popular method for doing this is called Integrated Gradients (IG). Think of IG as a hiker trying to explain a mountain view. The hiker starts at the bottom of the mountain (a blank, boring image with no information) and walks in a straight line to the top (the actual photo of the cat). Along the way, they measure how much the view changes at every step to figure out which parts of the path were most important.

The problem? The hiker's explanation depends entirely on:

Where they start: If they start at a black image vs. a gray image, they might tell a different story.
The path they take: Walking in a perfectly straight line through a mountain range often means walking through rocks, cliffs, or areas where the view doesn't change much (saturation). This makes the "importance" calculation messy and unreliable.

The Solution: FRInGe (The "Smart Navigator")

The authors propose a new method called FRInGe. Instead of walking through the "pixel world" (the image itself), FRInGe decides to navigate through the "Prediction World" (the AI's internal confidence).

Here is how it works, step-by-step:

1. The Starting Point: The "Blank Slate"

In the old method, you had to guess what a "blank" image looked like (e.g., all black pixels). This is like guessing what a "blank" feeling is.

FRInGe's approach: Instead of guessing a blank image, it starts with a blank prediction. Imagine the AI is 100% confused, saying, "It could be a cat, a dog, a car, or a toaster—all with equal chance." This is the "Maximum Entropy" state. It's a mathematically perfect definition of "I know nothing," without needing to invent a fake image.

2. The Path: The "Curved Highway"

In the old method, the hiker walked in a straight line. But in the world of AI predictions, a straight line is often a bad path. It might go through areas where the AI is confused or where the view doesn't change.

FRInGe's approach: It uses a Fisher–Rao Geodesic.
- The Analogy: Imagine you are flying a plane. If you fly in a straight line on a flat map, you might crash into a mountain. But if you fly along a great circle (the shortest path on a globe), you follow the natural curve of the earth.
- FRInge flies along the "curved highway" of the AI's confidence. It moves smoothly from "total confusion" to "total certainty" in a way that respects how the AI actually thinks, avoiding the rocky, uninformative areas.

3. The Translation: The "Translator"

Here is the tricky part: The AI lives in the "Prediction World," but we need to show you the explanation in the "Image World" (pixels).

FRInGe's approach: It uses a Pullback Metric.
- The Analogy: Imagine you are walking on a bumpy, uneven terrain (the prediction world). You want to know how your steps translate to a flat map (the image). FRInGe acts like a smart translator. It says, "To take one smooth step in the prediction world, I need to take a specific, slightly curved step in the image world."
- It also uses a Trust Region (a safety leash). If the translation tries to jump too far in the image world (which would look like a glitchy, weird image), the leash pulls it back to keep the steps small and stable.

Why Is This Better?

The paper tested this on six different famous AI models (like ResNet and VGG) using thousands of images.

Better Calibration (The "Truth Meter"): The biggest win for FRInGe is that its explanations match the AI's confidence better.
- The Analogy: If the AI is 90% sure it's a cat, a good explanation should show 90% of the "cat-ness" in the picture. Old methods often showed too much or too little. FRInGe's "mass" of explanation is perfectly calibrated to the AI's confidence.
Less Noise: Because it avoids the "straight line" through bad areas, the resulting explanation maps are cleaner. They focus on the actual object (the cat) rather than random noise.
No More Guessing Baselines: You don't need to guess what a "blank" image looks like anymore. The method defines "nothing" mathematically in the prediction space.

The Trade-off (The "Cost")

There is one downside mentioned in the paper: Speed.

The Analogy: The old method (IG) is like taking a bus on a straight road. It's fast. FRInGe is like hiring a private pilot who has to calculate the perfect curved flight path and check the weather at every second. It takes about 10 to 20 times longer to compute the explanation.
However, the authors note this is fine for "offline" use—like when you want to deeply understand a model after it's already trained, rather than needing an explanation instantly while the AI is running.

Summary

FRInGe is a new way to explain AI decisions. Instead of walking a straight, boring line through an image, it navigates the AI's internal "confidence map" along the smoothest, most natural curve possible. It translates this smooth journey back into the image, giving us explanations that are more accurate, better calibrated, and less dependent on arbitrary guesses.

Technical Summary: FRInGe – Distribution-Space Integrated Gradients with Fisher–Rao Geometry

1. Problem Statement

Gradient-based attribution methods, such as Integrated Gradients (IG), are widely used for explaining deep neural network predictions due to their model-faithfulness and scalability. However, standard IG suffers from several practical limitations that affect the stability and interpretability of explanations:

Baseline Sensitivity: The choice of input-space baseline (e.g., zero images or noise) is heuristic and can lead to qualitatively different explanations.
Path Artifacts: The standard straight-line interpolation in input space often traverses regions where gradients are poorly conditioned, dominated by artifacts, or saturated.
Saturation: Along standard paths, model outputs may change rapidly early on and then plateau, concentrating informative gradients in a narrow segment while later steps contribute noise.
Discretization Error: Approximating the path integral with finite Riemann sums can introduce substantial errors when gradients vary sharply.

Existing attempts to address these issues often involve designing trajectories directly in input space or relying on explicit generative models to define data manifolds, which can conflate attribution quality with the quality of the generative model.

2. Methodology: FRInGe

The authors propose FRInGe (Fisher–Rao Integrated Gradients), a method that redefines the reference point and the interpolation schedule within the predictive distribution space rather than the input space. The method consists of three core components:

A. Predictive Distribution Reference

Instead of selecting a heuristic input baseline, FRInGe defines the reference in the space of predictive distributions. It adopts a maximum-entropy reference ( $q^\star = \frac{1}{C}\mathbf{1}$ ), representing a uniform distribution over classes. This operationalizes "no information" directly at the model output level, removing dependence on data-type-specific baseline designs.

B. Fisher–Rao Geodesic Schedule

The interpolation between the model's predictive distribution $p(x)$ and the maximum-entropy reference $q^\star$ is defined as a Fisher–Rao geodesic on the probability simplex.

Using the square-root embedding $\psi(p) = \sqrt{p}$ , the simplex is mapped to the positive orthant of a unit sphere.
The geodesic is computed via spherical linear interpolation (slerp) in this embedded space, ensuring the path follows the intrinsic geometry of the predictive distributions.

C. Input-Space Trajectory via Pullback Metric

To realize this distribution-space trajectory in the input space, FRInGe uses the pullback Fisher metric.

Geometry-Aware Steps: The method minimizes a spherical alignment loss between the current prediction and the next waypoint on the geodesic. The descent direction is computed using the pullback Fisher metric $G(x) = J_F(x)^\top S(x) J_F(x)$ , where $J_F$ is the Jacobian of logits and $S(x)$ is the Fisher information matrix of the softmax distribution.
Trust-Region Stabilization: To ensure numerical stability and prevent large, non-perceptual input displacements, the method employs a dual trust-region rule:
1. KL Control: Limits the step size based on the local KL divergence (approximated by the Fisher norm), ensuring constant progress in predictive space.
2. Euclidean Cap: Imposes a maximum Euclidean step size to prevent instability in regions where the predictive geometry is flat.
Spatial Regularization: To discourage adversarial, high-frequency perturbations, a spatial smoothing operator (e.g., discrete Laplacian) is added to the linear system solving for the update direction.

D. Attribution Computation

Feature attributions are obtained by integrating input gradients along the realized input-space trajectory. Since the trajectory moves from the input toward the reference, the accumulated integral is negated to restore the standard baseline-to-input orientation. The method satisfies completeness with respect to the realized endpoint of the trajectory.

3. Key Contributions

Distribution-Space Definition: FRInGe is the first attribution method to define both the IG reference and the interpolation schedule entirely within the predictive distribution space, using a maximum-entropy reference.
Intrinsic Geometry: It constructs the integration schedule as a Fisher–Rao geodesic, avoiding arbitrary Euclidean straight lines in input space.
Pullback Realization: It induces the corresponding input-space trajectory via the pullback Fisher metric, creating a model-aware direction field without requiring an explicit generative model of the data manifold.
Stabilization Mechanism: The integration of KL-based trust regions and Euclidean caps ensures stable numerical realization of the path, addressing issues of saturation and discretization error.

4. Experimental Results

The authors evaluated FRInGe on 1,000 ImageNet validation images across six architectures (Inception-v3, ResNet-18/50/101/152, VGG-19).

Calibration Metrics (MAS): FRInGe demonstrates the most significant improvement on Magnitude Aligned Scoring (MAS) metrics (both Insertion and Deletion). This indicates that FRInGe attributions are better calibrated to the model's confidence dynamics; the accumulated attribution mass closely tracks the actual change in model confidence.
Perturbation Metrics: FRInGe remains competitive on standard perturbation-based AUC (Insertion/Deletion) and Infidelity metrics, though it does not uniformly dominate all baselines (e.g., SmoothGrad sometimes performs better on Deletion AUC).
Comparison to Geometric Baselines: FRInGe outperforms other geometry-aware methods like GGIG and GeoIG on MAS and Infidelity, suggesting its gains stem from the combination of predictive-space scheduling and pullback geometry rather than path design alone.
Ablation Studies: Removing the pullback geometry (Euclidean tracking) or spatial regularization leads to noisier, high-frequency trajectories and lower calibration scores, confirming that all three components (predictive reference, Fisher geometry, and regularization) are necessary for optimal performance.
Computational Cost: FRInGe is computationally more expensive than standard IG (requiring conjugate gradient solves at each step), with runtimes ranging from 4x to 22x slower depending on the architecture. However, the authors note this is acceptable for offline analysis and auditing.

5. Significance and Claims

The paper claims that FRInGe offers a principled alternative to Euclidean Integrated Gradients by shifting the definition of "progress" from pixel-space distance to changes in the model's predictive state.

Alignment with Model Dynamics: The primary benefit is the alignment of attribution magnitude with the model's internal confidence dynamics. By following the Fisher–Rao geodesic, the method avoids the saturation and artifacts common in straight-line paths, producing explanations that are more faithful to how the model actually processes information.
Modality Independence: By defining the reference in distribution space (maximum entropy), the method reduces the need for hand-designed, data-modality-specific baselines.
Limitations: The authors acknowledge the computational overhead as the main practical limitation. They suggest future work could focus on cheaper structural approximations of the inverse pullback metric or amortized predictors to accelerate the method.

The paper concludes that FRInGe successfully demonstrates that explanations following the model's own predictive dynamics (via Fisher–Rao geometry) yield higher-quality, better-calibrated attributions than those following arbitrary Euclidean paths.