Technical Summary: MM++ (Multilayer Mahalanobis++)

Problem Statement

Deep neural networks (DNNs) often exhibit overconfident predictions on out-of-distribution (OOD) inputs, a behavior attributed to neural collapse, where in-distribution (ID) representations are compressed into low-dimensional class centroids. While this improves classification accuracy, it reduces representational diversity, causing OOD samples to project near ID structures.

Existing post-hoc OOD detection methods face a trade-off between scale invariance and hierarchical expressivity:

1. Single-layer methods (e.g., Mahalanobis++, ReAct) operate on the terminal (penultimate) layer. While scale-invariant approaches like Mahalanobis++ project features onto a unit hypersphere to mitigate magnitude variations, they discard mid-level structural cues critical for detecting near-OOD samples.
2. Multilayer methods (e.g., X-Mahalanobis) aggregate scores from intermediate layers but typically treat layer-wise representations as independent marginals. They rely on additive fusion of individual scores, often requiring variance-based weighting or classifier fine-tuning. This approach ignores cross-layer conditional dependencies and may propagate high-variance noise from early layers, failing to detect hierarchical inconsistencies where an anomaly mimics ID features at individual layers but violates the expected evolutionary trajectory between them.
3. Strictly post-hoc limitations: Many advanced methods require auxiliary OOD validation sets or fine-tuning, violating the strictly post-hoc assumption and biasing the detector toward specific auxiliary geometries.

Methodology: MM++ Framework

The authors propose MM++ (Multilayer Mahalanobis++), a fully unsupervised, strictly post-hoc, and scale-invariant framework. Instead of additive score fusion, MM++ constructs a principled joint feature space by fusing selected intermediate layers with the terminal representation.

1. Layer Selection via Entropy Density Drops

MM++ identifies discriminative intermediate layers without auxiliary data by measuring entropy density drops:

• Covariance Entropy: For each layer $l$, the Shannon entropy $H_l$ of the normalized, Ledoit–Wolf shrunk covariance eigenvalue spectrum is computed. This quantifies representational richness.
• Entropy Density: Defined as $\rho_l = H_l / D_l$ (entropy per feature dimension), providing a scale-invariant metric.
• Entropy Density Drop ($\Delta_l$): The difference between consecutive layers, $\Delta_l = \rho_{l-1} - \rho_l$. A large positive $\Delta_l$ indicates a sharp drop in relative intrinsic dimensionality, marking a boundary of semantic compression.
• Selection Strategy: The penultimate layer $L$ is always included as an anchor. The top $K-1$ intermediate layers with the largest positive $\Delta_l$ are selected. This focuses the model on representations undergoing the most significant structural transitions while suppressing early-layer noise.

2. Feature Fusion and Joint Modeling

Selected layers are $\ell_2$-normalized and concatenated into a unified representation $\phi(x)$.

• Lossless Concatenation: Unlike additive fusion, this preserves full information from each selected layer without explicit weighting.
• Joint Precision Matrix: A single class-conditional Mahalanobis distance is computed in this unified space. The tied covariance matrix $\hat{\Sigma}_K$ is estimated from the concatenated features.
• Ledoit–Wolf Shrinkage: To handle the high dimensionality of the fused space (where feature dimension $D_K$ often exceeds sample count $N_c$), MM++ applies Ledoit–Wolf shrinkage to the covariance matrix. This ensures a well-conditioned, positive-definite precision matrix $\hat{\Sigma}_K^{-1}$.
• Cross-Layer Dependencies: The off-diagonal blocks of the joint precision matrix explicitly model cross-layer covariance. This allows the detector to penalize samples that are plausible under individual layer distributions but exhibit anomalous joint configurations (violating the learned evolutionary trajectory of features).

3. OOD Scoring

The final OOD score is the negative minimum Mahalanobis++ distance over all classes in the joint space:
$$S_{MM++}(x) = -\min_{c} (\phi(x) - \hat{\mu}_c^K)^\top \hat{\Sigma}_K^{-1} (\phi(x) - \hat{\mu}_c^K)$$
Higher scores indicate proximity to ID clusters. The method requires no fine-tuning, no proxy OOD data, and introduces only a single hyperparameter $K$ (number of fused layers).

Key Contributions

1. Unified Multilayer Framework: Shifts the paradigm from additive score aggregation to joint feature-space modeling, capturing latent cross-layer correlations.
2. Principled Layer Selection: Introduces a layer selection mechanism based on entropy density drops to identify boundaries of sharp semantic compression, avoiding heuristic weighting.
3. Strictly Post-Hoc & Scale-Invariant: Extends $\ell_2$-normalization (from Mahalanobis++) to a multi-layer setting with Ledoit–Wolf regularization, requiring no auxiliary data or architectural modifications.
4. Architecture Independence: The framework operates on normalized intermediate features, making it applicable to heterogeneous architectures (Transformers, CNNs, Hierarchical models) without dimension-specific adaptations.

Empirical Results

The paper evaluates MM++ across diverse backbones (ViT-B/16, Swin-T, ConvNeXt-T, EVA02) and datasets (ImageNet-1K, ImageNet-LT).

• Balanced Regime (ImageNet-1K): MM++ achieves competitive performance with state-of-the-art (SOTA) methods. While X-Mahalanobis (which requires fine-tuning) achieves the highest overall performance on far-OOD benchmarks, MM++ outperforms it on specific near-OOD datasets (Texture, SUN) without any fine-tuning.
• Long-Tailed Regime (ImageNet-LT): MM++ demonstrates superior robustness, achieving the highest average AUROC (83.91%). It significantly outperforms baselines on challenging near-OOD shifts (ImageNet-ES, -R, -V2), where single-layer methods and additive multilayer fusions struggle with subtle semantic deviations in imbalanced regimes.
• Cross-Architectural Consistency: MM++ consistently outperforms baselines on near-OOD benchmarks across Vision Transformers, hierarchical attention models (Swin), and modernized convolutions (ConvNeXt). Notably, X-Mahalanobis is often inapplicable to non-homogeneous architectures (like Swin or ConvNeXt) due to its reliance on uniform feature dimensions and fine-tuning.
• Ablation Studies:
  • Layer Selection: Enforcing penultimate layer anchoring and selecting layers via entropy drops is critical; removing anchoring or using pseudo-inverse estimation degrades performance significantly.
  • Shrinkage: Ledoit–Wolf shrinkage provides substantial gains (e.g., +8.26% AUROC) over pseudo-inverse estimation in high-dimensional fused spaces.
  • Hyperparameter $K$: Performance peaks at small $K$ (typically $K=2$), balancing the complementarity of intermediate information with the statistical noise of expanding dimensionality.

Significance and Claims

The paper claims that MM++ addresses the fundamental dichotomy between the stability of terminal layers and the depth of multilayer architectures. By modeling the hierarchical trajectory of feature evolution through a unified joint distribution, MM++ detects OOD samples that violate cross-layer evolutionary dynamics, a failure mode undetectable by single-layer or additive methods.

The authors emphasize that MM++ offers a rigorous, generalizable path toward reliable deep learning systems. It achieves robust near- and far-OOD detection across heterogeneous architectures and long-tailed regimes without the computational cost of fine-tuning or the data requirements of auxiliary OOD datasets. The work suggests that robust OOD detection can be achieved through generalizable principles (information-theoretic layer selection and geometric joint modeling) rather than architecture-specific tuning.