Using the Path of Least Resistance to Explain Deep Networks

This paper introduces Geodesic Integrated Gradients (GIG), a novel attribution method that replaces the straight paths of standard Integrated Gradients with geodesics under a model-induced Riemannian metric to eliminate feature-wise cancellation and produce more faithful explanations for deep neural networks.

The Gist

The Big Problem: The "Straight Line" Trap

Imagine you are trying to explain to a friend how a complex machine (a Deep Learning Model) decided to identify a picture of a jet plane.

The most popular way to do this right now is called Integrated Gradients (IG). Think of IG like a hiker who insists on walking in a perfectly straight line from a "blank canvas" (a black image) to the "jet plane" image.

The Problem:
In the real world, the "landscape" of a neural network isn't flat. It has hills, valleys, and cliffs.

• The Straight Line Flaw: If the hiker walks in a straight line, they might accidentally walk right over a steep cliff (a region where the model is very confused or changes its mind rapidly) or through a swamp (a region that looks like a jet but isn't).
• The Result: The hiker blames the wrong features. In the paper's example, the straight-line method looked at a jet and said, "The wings don't matter because the straight line passed through a confusing area." It gave a misleading explanation.

The Solution: The "Path of Least Resistance"

The authors propose a new method called Geodesic Integrated Gradients (GIG).

Instead of forcing a straight line, GIG asks: "What is the easiest, smoothest path to get from the black image to the jet image, avoiding the cliffs and swamps?"

• The Analogy: Imagine you are a river flowing from a mountain spring (the black image) to the ocean (the jet image). A river never walks in a straight line if there is a mountain in the way; it curves around the mountain to find the path of least resistance.
• The Magic: GIG calculates a "map" of the model's sensitivity (where it is sensitive and where it is flat). It then finds the path that flows smoothly through the "flat" areas and only crosses the "steep" areas when absolutely necessary.

The New Rule: "No Cancellation"

The paper introduces a new rule for good explanations called No-Cancellation Completeness (NCC).

The Analogy:
Imagine you are balancing a checkbook.

• Old Rule (Completeness): "The total sum of your transactions must equal your final balance."
  • The Loophole: You could have a huge deposit of $1,000 and a huge withdrawal of $1,000. The math adds up perfectly, but it hides the fact that you actually spent a lot of money. In AI, this means the model might say "Feature A is super important (+100)" and "Feature B is super unimportant (-100)," canceling each other out. The total is right, but the individual explanations are lies.
• New Rule (NCC): "Not only must the total balance be right, but you cannot hide a massive withdrawal behind a massive deposit."
  • GIG ensures that if a feature is important, it gets a high score, and if it's not, it gets a low score. It doesn't let them cancel each other out to hide the truth.

How They Do It (The Two Tools)

Since finding the perfect "river path" is mathematically hard, the authors built two tools to approximate it:

1. The "Neighborhood" Map (k-Nearest Neighbors):

   • Best for: Simple, low-dimensional data (like a 2D graph).
   • How it works: Imagine dropping thousands of pins on a map between the start and end points. You connect each pin to its closest neighbors. You then look for the path that requires the least "energy" to walk. It's like finding the shortest walking trail through a dense forest by hopping from tree to tree.

2. The "Magnetic" Path (Stochastic Variational Inference):

   • Best for: Complex, high-dimensional data (like real photos).
   • How it works: Imagine a rubber band stretched between the start and end points. You place magnets around the rubber band that repel it if it gets too close to a "high gradient" (dangerous) zone. The rubber band naturally snaps into a curved shape that avoids the magnets. The computer simulates this snapping process to find the best path.

Why This Matters

• More Honest Explanations: In tests with real images (like identifying animals in the Pascal VOC dataset), GIG was much better at pointing out the actual parts of the image that mattered (like the eyes of a cat) compared to the old straight-line method.
• The Cost: The trade-off is speed. Finding the "river path" takes more computing power than drawing a straight line. It's like taking a scenic, safe detour versus driving straight through a dangerous shortcut. It's slower, but the destination is reached more reliably.

Summary

The paper argues that to understand AI, we shouldn't just draw a straight line from "nothing" to "something." We should follow the path of least resistance, curving around the confusing parts of the model's brain. This gives us a truer, more honest explanation of why the AI made its decision, ensuring that important features aren't hidden by mathematical tricks.

Technical Summary

1. Problem Statement

Deep learning models are often "opaque," making it difficult to understand their decision-making processes. Integrated Gradients (IG) is a widely used axiomatic method for explaining these models by attributing importance scores to input features. IG calculates these scores by integrating model gradients along a straight line path from a baseline (e.g., a black image) to the input.

The authors identify a critical flaw in this approach:

• Flawed Attributions via Straight Paths: In Euclidean space, straight paths often traverse regions of high gradient (decision boundaries) or areas where the model is flat, leading to misleading attributions.
• The "Cancellation" Problem: Standard IG satisfies the Completeness axiom (the sum of attributions equals the difference in model output). However, it allows for feature-wise cancellation, where a feature receives a large positive attribution and another receives a large negative attribution. These cancel out in the sum, preserving Completeness but distorting the individual importance scores, making the explanation unfaithful to the model's actual behavior.
• Baseline Sensitivity: IG results are highly sensitive to the choice of baseline because the straight path may pass through irrelevant high-gradient regions depending on the baseline's location.

2. Methodology: Geodesic Integrated Gradients (GIG)

The authors propose Geodesic Integrated Gradients (GIG), a generalization of IG that replaces straight lines with geodesics (the shortest path between two points) on a Riemannian manifold.

A. Model-Induced Riemannian Metric

Instead of using the standard Euclidean metric, GIG equips the input space with a metric tensor derived from the model's Jacobian ($J_x$).

• Metric Definition: $G_x = J_x^T J_x$.
• Geometric Interpretation: This metric defines the "cost" of traversing the input space. Regions with high gradients (high sensitivity) have high "resistance" (longer path lengths), while flat regions have low resistance.
• Path of Least Resistance: The geodesic path minimizes the accumulated local resistance. Consequently, the path naturally avoids high-gradient regions (where the model is unstable or making decisions) and stays in flat regions where the model's output is constant, unless it must cross a decision boundary.

B. Approximation Techniques

Computing exact geodesics is computationally intractable for high-dimensional data. The authors propose two approximation methods:

1. k-Nearest Neighbors (kNN) Approach:
   • Suitable for low-dimensional inputs (e.g., synthetic data, tabular data).
   • Samples points between the baseline and input, constructs a weighted graph where edge weights represent the local geodesic length (approximated via IG integration), and uses Dijkstra's algorithm to find the shortest path.
2. Stochastic Variational Inference (SVI) Approach:
   • Suitable for high-dimensional inputs (e.g., images).
   • Defines an energy function $E(\gamma)$ with two terms: a distance term (keeping the path near the straight line) and a curvature penalty (pushing the path away from high gradients).
   • Uses SVI to optimize a probability distribution over paths, sampling paths that minimize this energy.

C. Theoretical Foundation: No-Cancellation Completeness (NCC)

The authors introduce a new axiom, No-Cancellation Completeness (NCC):

• Definition: $\sum_i |A_i(x)| = |f(x) - f(x')|$.
• Significance: Unlike standard Completeness (which sums signed attributions), NCC requires that the sum of absolute attributions equals the absolute change in output. This rules out the possibility of positive and negative attributions canceling each other out.
• Theorem 1: The authors prove that for path-based attributions under the model-induced metric, NCC holds if and only if the integration path is a geodesic. This provides a theoretical justification for why GIG is the unique path-based method satisfying this stronger axiom.

3. Key Contributions

1. Identification of the "Straight Path" Flaw: Demonstrated that Euclidean straight paths lead to unfaithful attributions due to misalignment with the model's gradient landscape.
2. Geodesic Integrated Gradients (GIG): Proposed a novel attribution method that integrates gradients along geodesics defined by a model-induced Riemannian metric ($J^T J$).
3. New Axiom (NCC): Introduced No-Cancellation Completeness to prevent feature-wise cancellation and proved that GIG is the unique method satisfying NCC under the proposed metric.
4. Approximation Algorithms: Developed practical algorithms (kNN and SVI) to approximate geodesic paths for both low and high-dimensional data.
5. Empirical Validation: Showed that GIG produces more faithful explanations than existing methods (IG, Guided IG, SHAP, etc.) on both synthetic and real-world datasets.

4. Experimental Results

The authors evaluated GIG on two datasets:

• Synthetic Half-Moons Dataset:

  • Setup: A simple 2D classification task where the model is flat everywhere except the decision boundary.
  • Metric: Purity (measuring if high-attribution points belong to the correct class).
  • Result: GIG (kNN) achieved the highest AUC-Purity (0.531), significantly outperforming IG (0.487) and other baselines. IG showed artifacts where attributions varied wildly based on the path, while GIG correctly assigned uniform importance to points in the same class region.

• Real-World Pascal VOC 2012 (Image Classification):

  • Setup: Used a ConvNext model. Evaluated using Comprehensiveness (drop in probability when top features are masked) and Log-Odds (evidence removal).
  • Result: GIG (SVI) significantly outperformed all other methods.
    • Comprehensiveness (AUC): 0.27 (GIG) vs. 0.21 (IG).
    • Log-Odds (AOC): 1.44 (GIG) vs. 1.25 (IG).
  • Qualitative: Visualizations (e.g., Fig. 1) showed that IG often attributed importance to background artifacts or irrelevant black regions, whereas GIG correctly focused on the object of interest.

5. Significance and Limitations

Significance:

• Theoretical Rigor: The paper bridges differential geometry and explainable AI, providing a formal proof (Theorem 1) that geodesic paths are necessary to satisfy NCC.
• Faithfulness: By following the "path of least resistance," GIG avoids the artifacts caused by traversing high-gradient noise, leading to more reliable explanations for debugging and auditing models.
• Complementarity: The approach complements existing methods like Manifold Integrated Gradients (MIG); while MIG focuses on staying on the data manifold, GIG focuses on the model's gradient landscape.

Limitations & Trade-offs:

• Computational Cost: The SVI-based method is computationally expensive (approx. 840x slower than standard IG). While acceptable for auditing/debugging, it is currently too slow for real-time applications.
• Hyperparameter Sensitivity: The SVI method requires tuning hyperparameters (e.g., the trade-off parameter $\beta$).
• Degenerate Metrics: For scalar-valued functions, the metric $J^T J$ is rank-1 degenerate (zero in flat regions), meaning geodesics are not uniquely defined in those areas, though the authors argue this is handled naturally by their approximations.
• Scope: Experiments were limited to image classification (ConvNext) and synthetic data; validation on NLP or other architectures is future work.

In conclusion, the paper argues that geometry matters in explainability. By respecting the curvature of the model's decision landscape via geodesics, GIG provides a theoretically sound and empirically superior alternative to standard path-based attribution methods.