Beyond $L_2$: Generalizing Abductive Latent Explanations to Diverse Prototype-Based Architectures

Technical Summary: Beyond L2: Generalizing Abductive Latent Explanations to Diverse Prototype-Based Architectures

Problem Statement
Prototype-based neural networks (PBNs) are recognized as "interpretable-by-design" architectures that justify predictions by referencing similar training examples (prototypes). Recently, Abductive Latent Explanations (ALE) were introduced to provide formal, mathematically guaranteed explanations for these networks by computing tight bounds on latent space distances. However, existing ALE formulations are rigidly confined to Euclidean latent spaces using $L_2$ distances. This creates a critical gap: modern state-of-the-art PBNs increasingly utilize non-Euclidean representations, including spherical metrics (cosine similarity), Gaussian densities, and dimensional projections (softmax). Current formal explanation methods are incompatible with these diverse geometric structures, preventing rigorous interpretability analysis across the full spectrum of prototype-based architectures.

Methodology
The authors generalize the ALE framework to support non-Euclidean prototype architectures by deriving specific bounding algorithms for different geometric variants. The core objective remains the same: to compute subset-minimal sets of prototypes that guarantee a model's prediction by iteratively tightening bounds on activation values.

1. Cosine Similarity (Spherical Geometry): For architectures like TesNet that use dot-product similarity on a unit sphere, the authors adapt the reasoning to spherical geometry. Since cosine similarity does not satisfy the standard triangle inequality, they utilize the Angular Distance ($d_\angle$) to derive bounds. They introduce a Spherical Cap Intersection Approximation, where the intersection of two spherical caps (defined by a patch and two prototypes) is bounded by a minimal-radius enclosing spherical cap. This allows for the derivation of tight lower and upper bounds on the similarity between a latent patch and unobserved prototypes.
2. Dimensional Projection (Simplex Geometry): For architectures like PIP-Net that map activations to a probability simplex via a softmax function, the geometric reasoning shifts from spatial distances to the conservation of probability mass. The authors propose a Simplex Explanation where adding a $\langle \text{patch}, \text{prototype} \rangle$ pair to the explanation "consumes" a portion of the probability mass, strictly limiting the upper bound of remaining prototypes. They also introduce a Sparse-Weight Explanation for models with sparse, non-negative classification heads, which allows for exact score calculation for the predicted class and tight bounding for competing classes.
3. Isotropic Gaussian Similarity: For probabilistic networks (e.g., ProtoGMM) where prototypes are Gaussian distributions, the authors map the problem back to a universal Euclidean space. By assuming isotropic covariance, they recover a "true" Euclidean distance from the similarity score. They apply the standard Hypersphere Intersection Approximation (HIA) in this mapped Euclidean space and then project the resulting geometric bounds back into the model's specific activation space.
4. Focal Similarity: For architectures like ProtoPool that use focal pooling to suppress background noise, the authors demonstrate that existing spatial ALE bounds can be aggregated to bound the aggregate statistics (maximum and expected values) required for the focal pooling operation, without altering the underlying geometric solver.

Key Contributions

• Generalization of ALE: The paper extends the ALE framework beyond Euclidean $L_2$ spaces to support spherical metrics, Gaussian densities, and simplex-based projections.
• Novel Bounding Algorithms: The authors systematically derive how to map diverse architectures to existing bounds or construct new, architecture-specific bounding algorithms (e.g., Spherical Cap Intersection, Simplex mass conservation).
• Unified Framework: By unifying these diverse models under a single formal framework, the work enables the first rigorous, cross-architecture comparison of interpretability.
• Empirical Validation: The authors validate these theoretical constructions by computing subset-minimal formal explanations on fully trained image classifiers across multiple datasets (Oxford Flowers 102, Oxford IIIT Pet, CUB200) and architectures (ProtoPNet, PIP-Net, TesNet, Gaussian ProtoPNet).

Results
The experiments reveal significant trade-offs between explanation size (interpretability) and computational cost across different paradigms:

• PIP-Net: Models with sparse, non-negative linear heads (PIP-Net) yield the smallest absolute explanation sizes and fastest computation times (approx. 0.07s), suggesting they are inherently more compatible with formal verification.
• Simplex Paradigm: The Simplex explanation paradigm (for PIP-Net) achieves the smallest relative explanation size (0.1%–0.2%), indicating excellent scalability with latent space dimensions.
• Gaussian Models: Scaled HIA provides the smallest relative explanation sizes for Gaussian models (4.6%–6.2%) but incurs the highest computational overhead (up to 142s), with some configurations timing out on larger datasets.
• Cosine/Spherical Models: While Spherical HIA achieves competitive relative sizes, both Cosine TI and Spherical HIA suffer from high variance and scalability issues, often timing out on the CUB200 dataset.
• Metric: The authors introduce a "Relative Size" metric to normalize explanation sizes across different paradigms, providing a robust indicator of inherent architectural interpretability independent of the total number of prototypes.

Significance
The paper claims to provide the first quantitative comparison of formal interpretability across diverse prototype-based networks. By extending ALE to non-Euclidean spaces, it bridges the gap between the theoretical guarantees of Formal XAI and the practical reality of modern, diverse prototype architectures. The work highlights specific design principles—such as the use of sparse, non-negative heads—that lead to better formal interpretability. It establishes that while geometric precision (e.g., in Gaussian or spherical models) can yield tight bounds, it often comes at a significant computational cost, underscoring the need for careful architectural design to balance interpretability guarantees with scalability.